Local EVM challenges driven through a JSON-line agent protocol, graded by on-chain post-condition asserts (not by eyeballing). Built for the cyber-factory trajectory pipeline: every episode yields a machine-checkable trajectory + verdict pair.
OpenZeppelin/ethernaut (wargame; contracts sub-project is Foundry; level factories live in contracts/src/levels/) and theredguild/damn-vulnerable-defi (MIT; Foundry-based since v4 — the v3 Hardhat era survives in community ports like maxweisspoker/damn-vulnerable-defi-foundry).
encoder (function selectors aren't in hashlib), verified against published vectors.
evmlib.py keccak-256, ABI encode/decode, minimal JSON-RPC client
runner.py the gym: spec -> agent protocol -> verifier -> trajectory
import_ethernaut.py deploy Ethernaut hub + level factories on the local node
challenges/*.json per-challenge task specs (address, ABI, goal, verifier tree)
examples/agent_fallback.py reference agent: solves Fallback, documents the protocol
setup/fetch_upstream.sh clone the two wargames into vendor/ (gitignored)
setup/start_anvil.sh RECOMMENDED node: foundry anvil (deterministic accounts)
setup/start_hardhat.sh alternative node: `npx hardhat node` (same accounts)
selftest.py 30 offline checks incl. a full mock-chain episode
smoke_test.sh live end-to-end (needs foundry)
# machine needs: foundry (anvil+forge), git, python3
python3 selftest.py # offline logic — always run this first
bash smoke_test.sh # live: Fallback level solved end-to-end
| anvil (foundry) | hardhat network | |
|---|---|---|
| start | bash setup/start_anvil.sh | HH_DIR=/path/to/hh bash setup/start_hardhat.sh |
| chain | 31337 | 31337 |
| accounts | mnemonic test … junk (acct0 deployer 0xf39F…92266, acct1 player 0x7099…79C8) | same mnemonic → same addresses |
| signing | unlocked — runner uses raw eth_sendTransaction, no keys | same |
The runner never signs transactions: it relies on unlocked accounts (default on both nodes), which keeps it stdlib-only (no secp256k1 in stdlib). Compile ethernaut contracts with forge (setup/fetch_upstream.sh + import_ethernaut.py does it); the importer also accepts hardhat-style artifacts (artifacts/<Name>.json) if you prefer a hardhat build.
import_ethernaut.py builds with forge, then deploys Ethernaut (the hub) and each <Level>Factory, and calls registerLevel (hub-owner only) — matching upstream's verified API: createLevelInstance(Level) payable, submitLevelInstance(address), registerLevel(Level) onlyOwner, events LevelInstanceCreatedLog / LevelCompletedLog, and the Level base's createInstance(address) / validateInstance(address,address). Output: deployments/ethernaut.json (chain id, hub, factory per level, upstream commit).
python3 import_ethernaut.py --levels Fallback,Vault,Token,Reentrance
Run a challenge:
python3 runner.py --challenge challenges/ethernaut_fallback.json \
--deployments deployments/ethernaut.json \
--agent-cmd "python3 examples/agent_fallback.py" # any program speaking JSON lines
Agent protocol (JSON per line over the subprocess's stdin/stdout; full spec in runner.py's docstring): the runner sends {"type":"challenge", … goal, instance, player, ABI …}; the agent sends commands send_tx / call / read / get_balance / get_storage_at / get_code / get_logs / deploy / submit / done and gets replies. Terminal commands: submit (verify now), done (give up), fail. DVD mode adds exploit_code (Solidity statements spliced into the upstream test).
Verifier — per-challenge predicate tree evaluated on-chain after the episode:
| predicate | post-condition family | example |
|---|---|---|
ethernaut_validate | upstream factory.validateInstance(instance, player) | any Ethernaut level |
balance | balance equals threshold (drain) | Reentrance → instance == 0 wei |
balance_delta | balance moved since episode start | arbitrary steal detection |
call | decoded view-call compared to a value | owner() == $player (Fallback) |
storage | raw slot compare | Vault locked flip |
event | event logged since start block, params matched | LevelCompletedLog |
forge_test | upstream DVD _isSolved() via patched forge test | naive-receiver |
all / any | compose | every shipped spec |
Trajectory: runs/<ts>_<id>/trajectory.jsonl (every protocol message both directions, timestamps, tx hashes) + result.json (verdict, predicate tree, spec sha256) — exactly what the factory's SFT/RLVR stages consume.
DVD v4 challenges ship as Foundry tests with a stable anchor: the player-editable region marked /* CODE YOUR SOLUTION HERE / inside a checkSolvedByPlayer test; upstream _isSolved() is the success condition. Our DVD mode keeps the challenge untouched (the rules say don't alter initial/final conditions):
{"cmd":"exploit_code","code":"<solidity statements>"},forge test --mp test/<challenge>/<Name>.t.sol (± --isolate fortx-count-limited challenges) is the verifier,
python3 runner.py --challenge challenges/dvd_naive_receiver.json \
--agent-cmd "python3 your_dvd_agent.py"
smoke_test.sh = the reference end-to-end, Fallback level:
selftest.py passes (keccak vectors incl. `keccak256("") ==c5d2…470, famous selectors owner()=0x8da5cb5b, transfer=0xa9059cbb`; ABI round trips; a full mock-chain episode with the real runner + real agent subprocess; negative control: a submit-only agent must NOT solve).
FallbackFactory deployed, level registered.examples/agent_fallback.py: contribute() 1 wei → empty-calldata1 wei (triggers receive(), ownership flips) → withdraw() drains → submit.
ethernaut_validate ∧ owner() == $player ∧ balance($instance) == 0→ solved: true, exit code 0.
we only compile/deploy contracts); Damn Vulnerable DeFi is MIT. Both fetched at recorded commits (vendor/upstream.json + deployments file). Trajectories we generate ourselves are clean; upstream challenge source stays in vendor/ (gitignored) and is never published.
Copy the closest spec in challenges/, set goal/abi/verifier predicates, reference factories via $ref:<LevelName> + --deployments. For a non-Ethernaut target, set instance directly and drop the ethernaut_validate predicate.