Summary / The 27 components
16 Candidate generation and competing hypotheses
Producing several plausible fixes instead of committing to the first.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
N=3 sessions plus an AST-normalized vote (arm h1): three workers on separate copies, patches grouped after stripping comments and docstrings, the largest group wins, a tie goes to the smallest diff. Also built and run on 15 cells each: per-session "frames" as data (pk4), sampling temperature (pk5) and model swaps (pk6, pk7).
What is missing
The vote is wired into her delegate path (bridge 1.90, dispatch_vote: k attempts, each judged by the envelope's own acceptance, smallest passing diff wins) but switched off, so production still runs one worker per task. It has never been measured on her own tasks. Not built in the harness: the exclusion list on retry ("already tried, do not repeat").
Reviewer Claude, 2026-10-04
Reviewer's read: Unconfirmed since the boxed re-run (10-19)
Boxed re-run (10-18/19, PREREG baseline-boxed-20261018): after the 10-18 leak audit (h1-v41 had network or answer-store access in 61% of its draws), harness 0.8.0 ran every tool inside the task container with no network. On verified-hard (38 tasks × 3 seeds) h1-v41 resolved 63/114 against h0b's 55/114: +7.5 points, CI [−3.1, +18.0], decision undetermined, so the adoption is marked unconfirmed (not reverted). h1 was better on 11 tasks, worse on 6, tied on 21. It cost about 9 times as much per run ($0.19 against $0.021) and 2.8 times the wall time; $6 of its $21 went on 10 tasks neither arm ever solved. Caveat found on 10-19: the vote filters its pool on the graded PASS_TO_PASS list. (A second caveat posted the same day, that h1-v41 and h0b ran different models, was wrong: deepseek-v4-flash is an alias of deepseek-flash, and both returned V4.1-Flash-1017 during the run.) The gold-free re-test, h1c-v41 (early stop + selection on the repository's own nearby tests) against h0-v41, was pre-flighted on 10-19: NO-GO, because the nearby-test check was unusable on 5 of 5 tasks (4 could not run the tests on the unchanged tree, 1 found no nearby tests), so the early stop never fired and h1c cost $0.159/task against a $0.154 bar. The pre-box history: the programme's only adopted change, and it held up on those rows. On dev, h1-v41 resolved 145 of 150 against h0-v41's 128 (+11.3 points, CI [+5.6, +17.0], McNemar p = 0.0002), consistent across seeds and on both sides of an out-of-memory interruption. On the held-out and regression sets (09-29, pre-registered): 99 of 100 against 93 (+6.0, CI [+1, +12]), confirmed by the rule. The cost is about 2.7 times per task. The diversity probes said the same thing from the other side: 15 of 15 cells resolved with frames and with temperature, so on this set more draws, not different draws, are what help.
How it is measured unconfirmed (boxed 10-19: +7.5, CI crosses 0, 9× cost)
Re-establish on boxed rows: a cost-capped h1 (stop the extra sessions when the first patch passes its own checks) is the cheapest next arm. Then run the production dispatch_vote on the replay bench, where it meets her own tasks rather than SWE-bench, before switching it on.
Adoption rules shared by every arm: How the tests work.
Research & attempts
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.