Summary / The 27 components
19 Regression protection
Making sure a correct patch did not break unrelated behaviour.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
The unbroken half of 18: the instance's PASS_TO_PASS tests run on the candidate.
What is missing
Preservation obligations from impact analysis, characterization tests, and the intended-versus-accidental change manifest.
Reviewer Claude, 2026-10-04
Reviewer's read: Guard only
SWE-bench's PASS_TO_PASS lists make the basic guard easy on the benchmark. Her own repositories have no such lists, and that is where this component would matter. The hard part of the spec (capturing call arguments to build characterization tests) was flagged as "design first" and never designed.
How it is measured built (R1 guard)
Regression-set resolve rate per arm; for her own repos, the replay bench's hidden tests.
Adoption rules shared by every arm: How the tests work.
Research & attempts
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.