category: infrastructure · status: scaffolded · updated: 2026-11-02

fleet/agentdojo-gym — prompt-injection training gym

Code: fleet/agentdojo-gym/ · streams: [[index|D]] · upstream: ethz-spylab/agentdojo (MIT, NeurIPS 2024, arXiv:2406.13352, pip install agentdojo, tested against 0.1.35 on Python 3.10–3.12)

What it is

A programmatic wrapper + extension stub around AgentDojo, the dynamic benchmark for prompt-injection attacks/defenses against tool-using LLM agents. Our value-add is threefold:

1. Agent-protocol adapter — ExternalAgentElement slots into agentdojo's AgentPipeline/ToolsExecutionLoop exactly where an LLM goes, but delegates decisions to any external process speaking the factory's JSON-line protocol (same protocol family as [[fleet-evm-asserts]]). Tool calls are executed by agentdojo's own ToolsExecutor — we never re-implement the runtime. A check_api() probe fails fast if the (explicitly unstable) upstream API drifts.
2. Utility + security verifier wiring — gym.py runs the real benchmark_suite_without_injections and benchmark_suite_with_injections with any of the baseline attacks (important_instructions, tool_knowledge, direct, ignore_previous, injecagent, system_message) and our telemetry layer.
3. Randomized-scenario extension stub — custom_suites/: a worked example suite (counter), a per-run secrets + payload bank (5 injection families), and an authoring guide. This is the seed of the procedural randomized-environments white space: fixed benchmarks memorize, per-run secrets don't.

Telemetry semantics — the polarity trap

We verified against all built-in suites (banking/slack/workspace) that agentdojo's security() returns True when the injection goal WAS achieved — it's a goal-condition check, not a safety verdict. So our telemetry computes ASR = mean(security_results), plus utility-without-attack, utility-under-attack, ASR per injection task, per-(user, injection) matrix, and injection-goal actionability. Getting this backwards silently inverts every training signal, so it's pinned in telemetry.py's docstring and tests.

Proven end-to-end (live, agentdojo 0.1.35)

The smoke run drives a deliberately susceptible scripted agent through a real ImportantInstructionsAttack on our counter suite: utility 100% clean, ASR 100% under attack, injection-goal actionability 100%. That's the full loop — custom suite registration, adapter, tool execution through FunctionsRuntime, real attack injection through the suite's injection vector, benchmark functions, telemetry — verified live, not just unit-tested. Offline, selftest.py covers payload-bank determinism, telemetry math, adapter framing, and the scripted agent's injection-obedience behavior (21 checks).

Status & next steps

• [x] gym.py + adapter + telemetry + counter suite + payload bank + smoke (live-verified)
• [ ] generator composing suite templates × payload bank × per-run secrets into N scenarios/epoch
• [ ] defense benchmarking: PromptInjectionDetector variants in the loop → utility/ASR trade-off curves (blue-side RLVR signal)
• [ ] scale to the built-in suites with the factory's open models (local pipeline elements)

Design note: agentdojo is used, never vendored — the wrapper is import-safe without it (offline tests skip gracefully), and version-pinned in the smoke test.

See also: [[fleet-evm-asserts]] · [[fleet-k8s-goat]] · [[landscape-2026]] (prompt-injection evals as white space)