The anti-contamination core of the fleet: seeded, randomized challenge instances across five classes — crypto, web, forensics, recon, privesc — each with a machine-checkable win condition and a strict task/solution split. Code: fleet/procedural-generator/ (generator.py, verifier.py, smoke_test.py). stdlib-only, generates and verifies entirely offline.
Static cyber benchmarks are contaminated twice over: models memorized public CTF writeups in pretraining (measured by CTFusion, arXiv 2605.11504 / ICML 2026, which showed static CTF benchmarks unreliable for agent assessment and moved to continuously generated "Live CTFs"), and fine-tuning itself overfits fixed instance sets. Random-Crypto (arXiv 2506.02048, aielte-research/Random-Crypto) proved the fix trains: pick a scheme subtype, randomize its parameters, encrypt a random flag → "near infinite variations", GRPO-trainable with exact rewards (their agent comes from the EnIGMA lineage, arXiv 2409.16165). We generalize that doctrine beyond crypto so the same property covers web, blue-side, and privesc behavior.
Every instance is a pure function of (class, seed, difficulty). The seed drives random flags, values, ports, paths, positions and timings; artifacts are constructed from the secret, so each instance is solvable by construction (the smoke test proves it — 15 instances, all solved from their own solution.json, all verified, 0 failures). Each instance emits task.json (agent-visible, provably free of solution material) and solution.json (private, verifier-side). verifier.py checks answers with per-class normalization: exact flags for crypto/web, alias-tolerant technique names + timestamp windows for forensics, exact CVE sets (alias-tolerant) for recon, binary+vector for privesc.
| Class | d=1 → d=5 gradient |
|---|---|
| crypto | Caesar/Atbash/base64 → XOR/add-shift → Vigenère-with-crib / encoding chains → 4-step chains / repeating XOR → RSA small-e (Håstad-style cube root, ± base64 wrap) |
| web | one vuln (SQLi / traversal / simulated command injection) → two chained → three chained with decoys; flags fragment and dash-join across vulns |
| forensics | obvious SSH brute force → any of 5 techniques at a random position → d5 injects a decoy technique and asks for the primary one |
| recon | 1 vulnerable banner → 3 vulns + 5 patched decoys; answer = exact CVE set |
| privesc | GTFOBins sudo → decoys → tar wildcard injection → env_keep LD_PRELOAD → writable sudo script |
Safety by construction: the web service is a throwaway in-memory app — SQLi hits only a sandbox SQLite DB, traversal is confined to a service-created temp root, and command injection is simulated (no subprocess ever sees attacker input).
Regenerate, don't accumulate: each week draws a fresh seed window (e.g. seed = iso_week * 1_000_000 + k), publishes that week's task.json set, and archives the old one. Solutions never leave the verifier side; held-out eval uses disjoint seed ranges; determinism per seed keeps every episode replayable and auditable. This is the refresh loop the factory doctrine (see [[index]]) claims as moat.
queue/ + a trainer-side sampler[[fleet/pwn-college-dojo]] · [[index]] · Random-Crypto: arXiv 2506.02048 · EnIGMA: arXiv 2409.16165 · CTFusion: arXiv 2605.11504