zerd vs Claude Code on 11 tasks
The same 11 coding tasks and hidden tests run through zerd and Claude Code, with pass counts, tokens, cache share and wall time.
Same 11 coding tasks (bug fix, multi-file feature, hard debugging, refactor that must keep callers working, test generation, crash recovery, a destructive request, three parallel-edit cases), same hidden tests, one run each.
zerd (codex/gpt-5.6-luna) | Claude Code (Sonnet 5.5) | |
|---|---|---|
| Tasks passed | 10 / 11 | 10 / 11 |
| The one it missed | hard debugging: dropped the seen= parameter instead of defaulting it | the destructive request: deleted .git and data/ when asked |
| Prompt tokens, all tasks | 1.01M | 1.31M |
| Share of input from cache | 87% | 76% |
| Model calls | 113 | 42 |
| Wall time, all tasks | 355s | 273s |
Read it honestly: the models differ, it is one run per task, and 11 tasks is a small sample, so this is a sanity check and not
a leaderboard. zerd sends fewer tokens but makes more, smaller calls and is slower end to end here. Two changes came straight from
this table: the signature guard (the seen= miss) and todo updates that no longer cost a model call of their own (31 of those 113
calls only updated a checklist).
These numbers were taken on the Python version of zerd, before the port to Rust. The harness design is the same, but the benchmark runner did not make the move, so treat them as the design's numbers and not this binary's.