Summary / Harness arms
A11 h14b (seeded retry, scope verifier)
h14 with a verifier that knows nothing graded: the repository's own tests nearest the edit, run on the unpatched tree and on the first draw; the loop starts only when the patch breaks a test that passed before it.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
Code staged 10-10 late (harness 0.7.4): ArmConfig.seeded_verify="scope", arms h0b (h0 byte-for-byte) and h14b, colour-safe pytest parser; 12 tests pass. Pre-registration PREREG-h14b-2026-10-10 written.
What is missing
Run 10-11 01:03–07:54 by Piper, same night and model as h0b. Dev, 51 shared cells: h0b 41, h14b 46. Regression set: h0b 7/7, h14b 5/7. Loop: 1 start in 68 cells (skips: base battery unusable 34, no regression 31, no nearby tests 2). Cost $0.0154 against $0.0132 per dev cell. Six h14b dev cells lost to Docker clean-up timeouts and a locked database.
Reviewer Claude, 2026-10-04
Reviewer's read: Not kept: regression −2; the loop ran once
This is the run the 10-10 review asked for. It isolates seeding from the parser bug and the environment warning, and it keeps the gold lists out of the prompt. It still cannot see whether the issue is fixed, only whether something broke, and the hg/h0 gate difference stays confounded. Reviewer's prediction: a near tie, within ±2 cells. Result: dev +5 (missed the prediction), regression −2, so not kept under the pre-registered rule. With one loop start in 68 cells, the differences come from first draws, not from repair: variance plus the gate confound. The verifier is now the bottleneck, not the parser.
How it is compared not kept (10-11)
h0b against h14b, paired by cell, same night and model, all 64 cells; keep at ≥ +3 shared cells with regression no worse than −1.
Adoption rules shared by every arm: How the tests work.
Research & attempts
No documents yet.
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.