Summary / Harness arms
A10 h14 (seeded retry)
hg plus component 14: one draw, then, if its verifier test fails, repair the failed workspace with the prior diff and failing output in the prompt instead of drawing again.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
10-10, pre-registered pair against h0 on 19 dev tasks × 3 seeds plus 7 regression tasks. Same-night cells (21): h0 21, h14 19, cost $0.013 against $0.043 per cell, wall 460 s against 703 s. The reported 90% against 65% compared tonight's h14 with h0 rows from 09-08, before the model change.
What is missing
A valid run: verifier fixes, both arms on the same night over all 64 cells. Staged as h14b (A11), not yet run.
Reviewer Claude, 2026-10-04
Reviewer's read: Keep rule not met; the loop barely ran
The loop fired on a genuine failure in 3 of 62 cells; the other 15 starts were a pytest parser bug (9) and an environment warning (6). So this run says nothing about seeding either way, except that a stray retry costs about three times as much. Piper's headline +25 came from mixing model generations; her own caveat about old baseline rows was right, and the same-night rows already settled it.
How it is compared invalid run (10-10)
Paired by cell, same night and model; then against h1 at matched cost.
Adoption rules shared by every arm: How the tests work.
Research & attempts
No documents yet.
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.