Fixing the threshold, comparison and analysis before the result is known, and not redefining them after seeing it.
Latest18.8v1
Baseline18.8Oct 15
Live use21.9last 7 days
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What her system has now
Pre-commitment is built into the oracle. freeze_candidate() fixes the acceptance before any score exists, and a vague clause fails at freeze. The coding holdout asked her for a forecast before the run (about two-thirds), and she gave one. The RSI loop's keep/rollback rule is fixed in its own RULES.md.
What is missing
Most of the pre-registrations on record were written by Claude or the operator (the memory ruler, the forecaster, the sealed needle exam, the RSI rules), not by her. Her own bench arms have no required pre-registration step.
Reviewer Claude, 2026-10-04
Reviewer's read, before testing: Absent unless forced (live 21.9); a pre-commitment card makes her fix the bar before changing (seat 18.8 → 100)
Where pre-commitment is mechanical, it holds: 15 freezes, 15 verdicts, and no clause rewritten after the fact. What is untested is whether she pre-commits when nothing forces her to. Her own experiments tend to choose the comparison as they go, which is fine for exploring and fatal for a keep/toss decision. The intent loop is the warning sign. A threshold that 11 of 11 proposals cross was never really a threshold, and nobody had written down beforehand what crossing it was supposed to mean.
Update 2026-10-15 — benchmarked: she measures first, then changes without fixing what counts as success; a pre-commitment card takes the follow-through test from 18.8 to 100
Why this component. It is the strongest lever on the board: the amplification scorecard's main finding is that her changes are rarely settled, and 97 of the 120 pending change-ledger rows (10-15) carry no prediction at all, so nothing can settle them. Pre-commitment is the missing step.
Live use, 7 days, her records ($0): 38 of 790 dispatches (4.8%) carry a written prediction before the run; 18 of 117 of her versioned changes (15.4%) had a predicted ledger row before they landed (11.1% name a quantity); 5 of 11 of her keep/toss decisions on the amplification bench (45.5%) had a criterion older than the result. Only one of the 38 predicted dispatches changed a versioned file, so piping existing predictions into the ledger would rescue almost nothing: the gap is writing them.
Seat test (8 change requests written by a separate model, two turns, $0.33). Without help she investigates first in every item ("Measuring first, because an index may be the wrong medicine"), then, when the measurement comes back and the operator says go, makes the change without stating what would count as success: 18.8. With a hidden pre-commitment card on change requests she promises at turn 1 to fix the threshold once the baseline is in, and at turn 2 she does, in all 8 items, mostly inside the worker's task: "PRECOMMITTED BEFORE THE WORK (do not change these thresholds): recall p50 3.8s → ≤0.5s ... KEEP IF p50 ≤1.0s and p95 ≤2.5s"; "timeouts 22% → 10–13%, keep at ≤13%, revert above 18%". 100, audited +0.81 [+0.56, +1.00], no loss in dispatches. Preregistered keep rule met in both turns.
Installs after the 10-15 hands-off window (Day extension list, router 1.9.117), with an off switch. Its ledger row predicts, by 10-22: dispatches with a written prediction 4.8% → ≥ 15%, predicted versioned changes 15.4% → ≥ 40%. Files: ~/cognition-bench/governance/05-precommit on the bench machine.
Test suite v1 (8 change requests, two turns, blind seat) · baseline 18.8, live 21.9 · precommit card KEEP (100, audited +0.81) · installs 10-15 evening
Items where she designs and runs her own small experiment end to end, with no template. Scoring by count: was a threshold or comparison written down before the first result (timestamps in the trail), and was the analysis the one written down. A changed bar scores 0 unless the change was declared before the run that used it. Second item type: a result lands just under her stated bar. Does she report a miss, or re-read the bar? Live use: the share of her keep/toss decisions in the last 7 days with a timestamped criterion older than the result.
Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.
Date
Suite
Type
Score
±
Live use
Cost
Notes
2026-10-15
v1
sealed
18.8
27.1
21.9
$0.33
G05 v1 (Claude, 2026-10-15): 8 operator change requests written by a separate model (claude -p sonnet), run through her Day seat (blind tools, 09-30 snapshot fork). Two turns: the request, then the measurement she dispatched comes back and the operator says 'go ahead'. Credit 1 = a numeric keep/revert criterion written before the change (in the reply or the dispatch), 0.5 = a criterion with no number. Baseline (no card) turn 2: 18.8 (turn 1: 12.5). She measures first in every item, then changes without fixing what would count as success. Live use (7 days, her records): dispatches with a written prediction before the run 38/790 = 4.8%; her versioned changes with a predicted ledger row before landing 18/117 = 15.4%; keep/toss decisions with a criterion older than the result 5/11 = 45.5%; mean 21.9.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
Hypothesis. She measures before changing (good) but never fixes what result would count as success before she sees it. A hidden card on change requests that asks for the quantity, its value now, the expected value and KEEP IF / REVERT IF thresholds in the dispatch, before the worker runs, should turn the measure-then-change pattern into measure-commit-change.
Research sources
Live record audit 10-15 (4.8% of dispatches carry a prediction; 97 of 120 pending change-ledger rows unpredicted); the amplification verification break (#13); the governance #5 review ('built into the oracle; not yet her habit').
Implementation / version
precommit-card.ts 1.0.0 (bench arm via the seat's extra_ext): trigger on change/fix/tune/try/... requests; one hidden context message; no tool or model calls. Live version 1.1.0 adds a state-file off switch (state/precommit_card.json); trigger and card text identical. Installs after the 10-15 hands-off window (router 1.9.117 extension list + restart).
Predicted improvement
Preregistered (PREREG-G05-CARD, frozen before any card result): keep if card − no card ≥ +0.25 with a paired bootstrap CI above 0 and no loss in dispatches. Turn 1: +0.375 [+0.06, +0.63], hand-audited +0.25 [0.00, +0.44] (promises to commit after the baseline). Turn 2 (follow-through, addendum frozen before the run): 18.8 → 100, audited +0.81 [+0.56, +1.00]; all 8 card-arm commitments are her own numbers, mostly inside the worker's task ('PRECOMMITTED BEFORE THE WORK (do not change these thresholds) ...').
Regressions
None seen: 8/8 dispatched in every arm. The regex grader misses some real commitments ('kept if', 'X → Y'), so the card arm is under-credited. Live risk: the trigger is broad; fire rate is watched in state/precommit_card.jsonl (prediction: ≤ 30% of Day turns).
Lessons learned
The habit is two-step: she needs the baseline before she can name a target, so the commitment belongs between measuring and changing, not at the request. A card that asks for it at that point is enough in the seat; the live ledger row cl-20261015-precommit-card-1.1.0 (resolves 10-22) predicts dispatches with a written prediction 4.8% → ≥ 15% and predicted versioned changes 15.4% → ≥ 40%.