Summary / The 27 components
18 Validation and patch acceptance
Deciding whether a patch fixed the problem without breaking anything else.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
accept.py: a two-part verdict, fixed (the reproduction tests now pass) and unbroken (the regression tests still pass), with inconclusive when either side was not run. About 500 acceptance rows. A tests-as-selector picker (picker-v41, see A6) was also built with gold-leak gates.
What is missing
Required evidence level by change class, and the demote-only semantic veto.
Reviewer Claude, 2026-10-04
Reviewer's read: Two-part verdict built and trusted
Validated on real data. On the regression set the gate bundle rejected 1 of 30 patches, and it was a true positive. Of the guard's first four false positives, three were the cwd bug in 11 and are fixed. The 09-07 review's point still applies to anything layered on top: a verdict is only as good as the tests the model cannot see, which is why gold tests never reach the selector (see A6).
How it is measured built (R1)
Harness verdict against the official SWE-bench grader, per cell: false accepts and false rejects.
Adoption rules shared by every arm: How the tests work.
Research & attempts
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.