Summary / The 27 components
01 Coding harness (REPAIR and primitives)
The commands the model works through while it changes code (search, view, edit, execute, test), and the shape of the repair step itself.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
The h0 arm: the workflow driver runs one plain pi -p worker session inside a harness-owned git workspace, then the harness's own VERIFY step. The h1 arm runs three such sessions on separate copies and votes (see 16). An edit-before-read rule blocks an edit on a file the session has not read. The harness extension is loaded with -e for harness arms only, so Piper's live sessions never see it.
What is missing
The spec's own design was set aside. Its "agentless" single-shot pipeline (six primitives, exact-string edits, one sampled repair call) is not built. Pi's turn loop stayed as the repair step. Not built: the REPAIR system prompt (repair_v1 exists but its arm, h2p, never ran), the read_entity / search_symbol / search_code tools over the code graph, and observation collapsing for long sessions.
Reviewer Claude, 2026-10-04
Reviewer's read: Plumbing built and proven neutral; the spec's design set aside
The spec contradicted itself: a single-shot call with no loop cannot produce exact-string edits for bodies it has never seen. The integration review on 09-06 kept pi's loop and made the harness the environment around it, which was the right call. The measured result is the useful one. With budgets removed, h0 against the plain worker came out at Δ −1.3 points, CI [−8.0, +5.3]: wrapping a worker in the driver, workspace and container costs nothing. The first h0 run had come out at −8.7 because it carried a 40-turn budget the baseline lacked; 73 sessions hit the cap. That gave the programme its first rule: a budget is a gate, and never belongs in the control arm.
How it is measured h0 measured
Each new primitive is an arm with h0-v41 as its parent, run on the SWE-bench dev set at k=3 and adopted only by the paired-bootstrap rule (see How the tests work). The dev set is now saturated (h1 resolves 97%), so any new primitive needs the harder set first.
Adoption rules shared by every arm: How the tests work.
Research & attempts
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.