These four pre-install scanners work in one of three ways, usually combining two: static rules, LLM judgment, or actually connecting to the MCP server. Cisco's own documentation shows rules alone catch about 8% of malicious samples they have not seen, and third-party tests show the numbers shift a lot when the sample mix changes. Use scanners as a first filter; "no findings" does not mean safe.
I sort agent sandboxes into four kinds: shared-kernel OS isolation, container orchestration, local microVMs (lightweight virtual machines) and remote microVMs. Claude Code's sandbox is off by default and wraps only shell commands; Codex CLI's is on by default locally. Most public escapes go around the boundary through config files or check scripts rather than through its core, and a sandbox answers how far a fooled agent can get, not whether it gets fooled. On Claude Code, turn the sandbox on, put ~/.ssh and ~/.aws on the deny list, and block the Read tool separately.
Guardrails intercept an action before the agent carries it out. On Claude Code the main attachment point is the hook, and the official docs say that on most events a hook that errors or times out lets the action go ahead; blocking needs onFailure: block. Other tools have their own failure behavior and defaults: the Agent Governance Toolkit library allows everything when no policy is loaded, NeMo Guardrails lets requests through when its jailbreak detector is down, and llm-guard and rebuff are archived. Ask three things of any guardrail: where it intercepts, what happens when it fails, and who can change its rules.
Detection and audit tools run along a spectrum: the open-source release of Uber ADR reads records purely after the fact, numbat only monitors by default, and for most of the rest the selling point is that they can block before an action, yet they start out observing only. Whether a record can serve as evidence depends on who can change it: when the signing key sits on the same machine and user as the audited agent, tampering can be noticed but not prevented.
The two representatives take opposite approaches: Cloudflare's security-audit skill is a pure-LLM multi-stage process, and Cloudflare says that when its internal harness wired in Semgrep, the hunting agents did not call it once in a month; Trail of Bits combines Semgrep and CodeQL scripts with LLM judgment. I found no checks specific to Workers, D1, KV or Astro in the rules or supporting files of either, and neither publishes accuracy figures from real projects. This post is based on documentation only and was not run against any project.
Two reports that both say "attack success rate 5%" cannot be compared directly: behind the number sit who set the questions (the attacker model), who judged (the judge, a model or rule that decides), and what the denominator counts. garak's own FAQ says its scores have no scientific validity, and in HackAgent's white paper the order of two models flips with the judge threshold. I found no independent accuracy evaluation of any of the four.
AI penetration-testing tools use an LLM as the attacker against ordinary software. They share two problems: the tool is itself an attack surface (a target can take it over through prompt injection, and HexStrike's server has no authentication), and once open-sourced, its use cannot be controlled (the same tool serves authorized testing and attacks alike, and the anti-abuse settings of ARTEX and HexStrike are defaults the operator can edit). When you read news that a tool was abused, ask who said so and whether the evidence is independent. The ARTEX incident has several vendor analyses, but some of them examine overlapping attacker infrastructure or cite each other, so they are not independent pieces of evidence.
When you see 'covers OWASP 10 of 10', 'merged into Microsoft and Cisco' or '9X% detection', ask five things: who said it, what the term means, who set the questions, what is being measured, and where the primary source is. The earlier posts and a few cases added here each happen to show one: Microsoft's toolkit itself says 7 full and 3 partial; of ATR's (an open-source set of agent threat-detection rules) merged PRs into Microsoft and Cisco, the three checked were all opened by its author; Cisco's own detection rate and an independent study's differ several-fold; common benchmarks score models or agents, not tools.