Summary / Harness arms
A2 h0 (plumbing)
The baseline worker inside the harness: driver, git workspace, container execution, VERIFY. No gates, no budgets.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
v4-flash era (09-08): 115/150, against baseline Δ −1.3, CI [−8.0, +5.3], undetermined. Plumbing costs nothing. Current model (09-15): 128/150 (85.3%), the parent of every later arm. Held-out plus regression (09-29): 93/100.
What is missing
Nothing; it is the parent arm.
Reviewer Claude, 2026-10-04
Reviewer's read: Neutral, as intended
Its first version was rejected at −8.7 points because of a 40-turn budget the baseline did not have. Relabelling those 150 runs as h0-v1-budgeted instead of deleting them kept the record honest.
How it is compared measured
Parent of every arm since 09-14.
Adoption rules shared by every arm: How the tests work.
Research & attempts
No documents yet.
Score history
Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.
| Date | Suite | Type | Score | ± | Live use | Cost | Notes |
|---|---|---|---|---|---|---|---|
| 2026-09-29 | v3 | sealed | 93 | — | — | — | h0-v41 on holdout 20 + regression 30, k=2: 93/100. Suite v3 = held-out sets. |
| 2026-09-15 | v2 | sealed | 85.3 | — | — | — | h0-v41 on dev, k=3: 128/150. Model deepseek-flash (V4.1). Suite v2 = same tasks on the new model; not comparable with v1. |
| 2026-09-08 | v1 | sealed | 76.7 | — | — | — | h0 on dev, k=3: 115/150. vs baseline Δ −1.3, CI [−8.0, +5.3], undetermined. deepseek-v4-flash. |
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.