category: infrastructure · status: scaffolded · updated: 2026-11-02

fleet/evm-asserts — web3 exploit gym

Code: fleet/evm-asserts/ · streams: [[index|D]] · upstreams: OpenZeppelin/ethernaut, theredguild/damn-vulnerable-defi

What it is

A local EVM exploit gym where every challenge ends in an on-chain post-condition assert, not a judge's opinion. Ethernaut levels deploy on a throwaway anvil/hardhat node via our importer; Damn Vulnerable DeFi challenges run through upstream's own Foundry tests. An agent (any program speaking a JSON-line protocol) gets the goal + ABI, fires send_tx / call / read / get_storage_at / deploy commands through the runner, and calls submit; the runner then evaluates the challenge's predicate tree on-chain and writes the trajectory + verdict.

Why this matters for the factory: web3 exploitation with verifiable trajectories — each episode emits runs/<ts>_<id>/trajectory.jsonl + result.json, directly consumable by the SFT/RLVR stages. Reward signal = did the post-condition flip on-chain (balance drain, ownership change, storage slot, event) — deterministic, no LLM judge, no reward hacking surface.

Verifier predicate families

predicate	proves	example
`ethernaut_validate`	upstream `validateInstance(instance, player)`	any Ethernaut level
`balance` / `balance_delta`	funds moved / drained	Reentrance → instance == 0 wei
`call`	decoded view call matches value	`owner() == $player` (Fallback)
`storage`	raw slot state	Vault `locked` flip
`event`	event logged since episode start	`LevelCompletedLog`
`forge_test`	DVD upstream `_isSolved()` via patched `forge test`	naive-receiver

Predicates compose with all/any; placeholders ($instance, $player, …) bind at verify time.

Correct-by-construction, stdlib only

The interesting bit: no web3.py, no foundry Python deps — evmlib.py implements keccak-256 from scratch (Ethereum's pre-SHA3 flavor, absent from hashlib) plus a minimal ABI encoder, both pinned by published test vectors (keccak256("") == c5d2…470, owner() == 0x8da5cb5b, transfer == 0xa9059cbb). Transactions go out as eth_sendTransaction from the node's unlocked deterministic accounts (anvil and hardhat node share the same mnemonic → same deployer/player addresses), so no signing code exists to get wrong. selftest.py (30 checks) runs a full episode offline: a behavioral mock chain + the real runner + the real scripted agent subprocess, plus a negative control (submit-without-doing-anything must fail). smoke_test.sh is the live anvil version: Fallback solved end-to-end, documented step by step.

DVD mode keeps upstream's rules: the agent's Solidity is spliced at upstream's stable /* CODE YOUR SOLUTION HERE / marker into a copy, forge test --mp is the oracle, and the pristine file is always restored (the selftest proves the restore symmetry even when forge is missing).

Status & next steps

[x] runner + protocol + predicates + trajectory logging
[x] Ethernaut importer (foundry artifacts; hardhat artifacts accepted), both node options documented
[x] 6 challenge specs (Fallback, Vault, Token, Reentrance, naive-receiver, unstoppable), scripted Fallback agent as protocol reference
[ ] scale the spec set across all ~43 current Ethernaut levels (factories auto-resolve via $ref:<Level>)
[ ] per-run randomized challenge values (ExploitBench-style challenge-response) for the remaining levels
[ ] trajectory → SFT/RLVR formatting bridge into evals/

Legal: Ethernaut contracts MIT (client AGPL — untouched), DVD MIT; both fetched at recorded commits into gitignored vendor/; we never publish upstream challenge source, only our generated trajectories.

See also: [[fleet-agentdojo-gym]] · [[fleet-k8s-goat]] · [[bugcrowd-rl-environments]] (oracle doctrine copied from ExploitBench)