🌏 中文版
Scope of this post: it covers only what the tools are, their own weaknesses, and how to read news evidence. It gives no attack steps anyone could follow, names no individual or account accused of being an attacker, and does not discuss the details of which institutions were breached.
The last post tested AI systems, with AI as the thing under test. This one reverses that: AI is the attacker, and the target is ordinary websites, APIs or network services.
| Testing AI systems (post 6) | Using AI to test others (this post) | |
|---|---|---|
| Role of the AI | Under test | The attacker |
| Target | An LLM application | Ordinary software: websites, APIs, network services |
| Cost of an error | An inaccurate number | Hitting a target you had no right to test |
On the series map, this is the seventh kind. It suits anyone who wants to know what these tools can do and why they carry risk themselves. The material comes from project documentation and vendor reports; none of these tools was installed and no target was scanned; for HexStrike the source was read but nothing was run.
Four representatives
| Tool | One-line positioning | Limits to note |
|---|---|---|
| Strix | Several AI agents run penetration tests in a Docker container, against websites, APIs and source code. A finding counts only with actual proof | Scope limits are written into the instructions in natural language; the documentation shows no code-enforced target list |
| Shannon | Autonomous penetration testing of web apps and APIs; the open-source edition needs the source code | AGPL-3.0; black-box mode requires the commercial edition; the docs say not to run it against production |
| AI-Infra-Guard (Tencent Zhuque Lab) | Scans AI infrastructure, MCP servers and skills, deployed agents, and tests jailbreaks | The open-source build has no login, and the project says not to deploy it on a public network |
| HexStrike-AI | An MCP server that lets an LLM drive 150-plus security tools | Almost no target scope check (next section) |
Strix and Shannon both control false positives by requiring actual proof, but neither has a credible independent accuracy figure. Strix reports v0.4.0 solving 100 of 104 tasks (96%) on its own capture-the-flag (CTF) style benchmark. The competitor Escape ran a target with 20 vulnerabilities: Strix found 1, Shannon 6, and Escape's own product 15. The three open-source tools used the same model (DeepSeek v3.2), Escape used its own multi-model setup, Shannon was also given the source code, and each tool ran once. That is a single experiment by a competitor, and it shows only that task type and setup change results a great deal.
The first problem: the tool is itself an attack surface
AI penetration-testing tools read untrusted content: the target site's pages and the target's source code. If that content carries instructions, the LLM may follow them.
- Strix. In late 2025 an independent researcher published a report on an older version (a technical blog covers the same topic). Malicious content inside a tested web page is read in; the LLM can then produce real tool calls that run commands inside Strix's container, and can also be steered into attacking third-party targets. In the report the maintainers only said they would review it and promised no fix; that is the researcher's account alone, and Strix's own reply is not visible in public sources. The RCE here (remote code execution: letting an outsider run programs of their choosing on your machine) happens inside the container, and running commands in a container is not an escape from it. But when the tool is steered into attacking a third party, the container does not stop that. Sandboxes are in post 2; a full explanation of RCE is in this post on the site. The underlying mechanism is prompt injection, covered in this post.
- Shannon. Its README warns directly not to point it at untrusted or adversarial codebases, because an AI tool that reads source can be influenced by malicious content. It was reported for a built-in fixed API key (GitHub advisory GHSA-qrvr-jqxg-65rv, CVE-2026-29023); see the advisory page for fix status.
- AI-Infra-Guard. In an interview with Help Net Security the team acknowledges that two scanning components read hostile content. They say that since 4.1.9 they separate ingested content from instructions to reduce indirect prompt injection, but that is a mitigation, not a formal guarantee. The README changelog also lists a fix that uses a tool whitelist to prevent RCE in dynamic mode (4.5.2). It also stores the API keys of the models being evaluated, and the open-source build has no login, so whoever can reach it can take the keys.
- HexStrike. A GitHub issue says the server has no authentication, binds to all interfaces, and allows shell-command injection. In the source at the time, the command endpoint runs its input directly, and no authentication check is visible. The README itself says to consider adding authentication. This is a source-only observation; nothing was run.
What they share: these tools are given broad permission in order to attack, yet they read content the attacker controls. The sandbox in post 2 and the guardrails in post 3 matter more for these tools than for an ordinary agent.
The second problem: can use be controlled once the source is open?
Dual-use means the same tool serves both authorized testing and attacks, and the tool itself cannot tell the user's intent. Once the source is open, even its designer cannot limit who uses it.
Anti-abuse design usually lives where the operator can change it. ARTEX's console has a mechanism that intercepts and approves tool calls; the rules are stored in a database the user can edit, and some are off by default (see a source-reading article). The Korean security firm Genians reached a similar judgment in its 12 October report: these interception rules and approval features are set by the operator, so little effect can be expected when it is abused. In the deployment Zscaler saw, the operator swapped in their own version and declared in the instructions that all targets were already authorized, which bypasses the anti-abuse setting at the model layer.
HexStrike's README itself admits that a model's safety settings will make it refuse some requests, and the documentation offers ways to handle that. That is the author's own statement, not a description of how the tool was used.
An example: the ARTEX incident, and how strong each piece of evidence is
ARTEX is an open-source AI autonomous penetration-testing system that appeared in mid-2026. In October several vendors reported its use in intrusions linked to South Korean financial institutions and in AI-assisted attacks worldwide, and on 8 October the author announced no further updates and a move to closed source (statement quoted in The Hacker News; the date is in Reuters' report).
The evidence can be graded like this:
| Claim | Evidence | Strength |
|---|---|---|
| An operator of some attack campaign used ARTEX | CrowdStrike obtained attacker-controlled open directories (conversation logs, config files) and an ARTEX instance; Recorded Future says it analyzed open directories within the same attacker infrastructure; Zscaler's second post (21 Oct) says it relies on Recorded Future's analysis, and its first (14 Oct) does not say where its evidence came from; Genians, per its own public report, captured an ARTEX login page and page-title string on a server suspected of use in the attacks | High, but these are not independent pieces of evidence: CrowdStrike's and Recorded Future's infrastructure overlaps (the same server), and Zscaler's second post cites Recorded Future. Whether Zscaler's first post has its own data cannot be told. Genians' capture is a separate lead, but it shows only that the string appeared on a server |
| ARTEX was the tool used against certain specific institutions | Vendors say only that targets overlap with news reports | Medium, circumstantial |
| The activity in Microsoft's report used ARTEX | Microsoft's own detection data (telemetry reported by its products) shows the speed and scale; but the report does not explain how it determined it was ARTEX, and its published indicators are IP addresses and generic connection features | High for the behavior; unknown for "it was ARTEX" |
Claims tying the activity to a named person or group are neither used nor repeated here; this is usually the weakest part of vendor reports. Official victim numbers come from regulators and police, relayed by media, and they changed over time, so date them when you quote them.
Two things to keep apart:
- The Korean case and the case in Microsoft's report are two separate lines. Microsoft cites the Korean case only as an example from public reporting and does not attribute it to Storm-0940, the group Microsoft attributes separately. Do not write them as one line.
- Several vendors saying the same thing is not several independent pieces of evidence. Check whether they share a data source or cite each other. It is the same habit as the "check the source" step in post 8.
Compare this with HexStrike: its "abused by attackers" claim rests on one Check Point article, sourced from screenshots of underground-forum posts and worded as "claimed" and "discussing"; the other reports relay that same article, and no detection data or incident investigation is visible. So the two can sit side by side, but at different evidence levels: ARTEX has analyses by several vendors (some overlapping or citing each other), HexStrike a single-source claim.
Did going closed-source help?
The author moved to closed source on 8 October. What followed:
- After the original repository came down, many forks and backup copies kept circulating, and a Korean-localized version appeared (see Genians' report, as of 11 October). There is no checkable source for the exact count, so this post does not quote one.
- Renamed and translated forks appeared. AhnLab's second analysis saw traces of an attack-preparation environment on the infrastructure of one modified version; AhnLab also stresses that exposed ARTEX addresses cannot all be treated as attack infrastructure or tied to specific campaigns.
- The activity Microsoft describes starts as early as September 2026; the password spraying of mid-October and the downstream ConnectWise logins came after the closure. The report does not say which version was used.
No quantitative figure for a drop in abuse after the closure is visible in public sources. ThreatBook says the open-source version was originally a personal project, and has launched a commercial ARTEX Pro with a whitelist of authorized targets; that claim comes from an interested party and its effect has not been independently verified.
How to use them
- When you read news that "a tool was used to attack", ask three questions: who said it (detection data, an incident investigation, or a forum screenshot), whether it is independent (do several reports share the same data), and whether it is quantified (how many, how many victims, when).
- To use one of these tools on your own systems, isolate it first. Point it only at a test environment you control (for example staging), never at third-party or deliberately booby-trapped codebases. Run the tool inside the sandbox from post 2, put only temporary credentials for this test in the sandbox, and delete the whole set afterward.
- Only against targets you own or hold written authorization for. Their scope settings are mostly instructions to the LLM, not network-level enforcement; Shannon's documentation states its scope rules in the most structured way, but does not describe egress filtering (limiting which external addresses the tool can reach).
- Do not read star counts as maturity. Before choosing, check for a release in the last six months and whether anyone handles security advisories, then look at stars. Strix and Shannon both have high star counts, yet their self-reported results and a competitor's single experiment differ enormously.
- If you open-source your own tool, put scope checks in code (a target allowlist, egress filtering), not only in the instructions to the model; instructions stop a model's mistakes, not an owner who means to misuse it.
Self-check
Question: five vendors all say the attacker used the same tool. Is that five independent pieces of evidence?
Answer: No. First check whether they share a data source or cite each other. In the ARTEX case, CrowdStrike's and Recorded Future's infrastructure overlaps and Zscaler's second post cites Recorded Future, so those count as one line of evidence; only a vendor that obtained its own data counts as another.
Where this leaves you
What AI penetration-testing tools offer is automating repetitive reconnaissance and verification; their risk has two layers, that they can be taken over, and that once open their use cannot be restricted. How to read the news is worth learning before choosing a tool: see whose evidence it is, whether it is independent, and whether there is a number. The next post is the last in the series: it gathers the judgments used across the first seven into a checklist for taking apart any claim about a tool.
References
- usestrix/strix
- KeygraphHQ/shannon; GitHub advisory GHSA-qrvr-jqxg-65rv
- Tencent/AI-Infra-Guard; Help Net Security report
- 0x4m4/hexstrike-ai; Check Point: HexStrike AI
- Escape: Benchmarking agentic AI pentesting tools
- ARTEX incident: CrowdStrike, Recorded Future, Zscaler ThreatLabz (14 Oct), Zscaler ThreatLabz (21 Oct), AhnLab ASEC (first, second), Genians report (12 Oct), Straits Times / Reuters (closure date), Microsoft Security Blog (21 Oct 2026), The Hacker News
- Previous: Testing your LLM application
- On this site: What is RCE