Summary / Component 20 of 24
20 Bottleneck migration
Recognizing when fixing one limitation exposes another.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
Layer 5 — Scaling and bottlenecks. Can Piper sustain improvement as the system grows more complicated?
What her system has now
No instrument. Nothing records a constraint as "fixed" and then watches for the next one. What she has is the chain of diagnosis docs, and two chains show migration clearly:
- Solver loop. Capability keeps came quickly (v2 09-27, v3 09-30), then stopped. In the 10 decided sessions after 09-30 there were 2 keeps, both on cost. She named the new constraint (test resolution,
piper/rsi_keep_rule_power) on 10-12: 12 days and 10 decided sessions after the last capability keep. - Transfer lookup. Her fix 1.4.1 (10-12 19:17) cut empty lookups from about 75% to 4%. Her next diagnosis (19:27) still treated emptiness as the main fact, and her next change (register-ingest, 20:35) aimed at raising the candidate rate. On 1.4.2 that rate was already 43 of 46 (93%) before ingest; it went to 79 of 80 (99%) with ingest on and back to 64 of 69 (93%) after. The cost was the latency that got it reverted.
She did foresee one trade: the ingest prediction paired "candidate rate rises" with a latency bar, and she reverted when the bar broke.
What is missing
- No "constraint fixed on date X; now watching Y" record.
- Version or fix dates are not used to split the data when she re-diagnoses (see #19).
- The ingest prediction's success clause (b) counted rows with
n_short > 0, which is the shortlist the switch itself adds (75 of 80 rows with it on, 0 off). It could not fail while the switch was on, so it could not tell her whether candidate supply was still a constraint.
Reviewer Claude, 2026-10-15
Reviewer's read, before testing: Since the hook files every version bump, most changes get no later re-read at all; the fast re-reads she does write are lost in the count
#20 is the harder half of #19, and the lookup case is a clean instance. She fixed the binding problem, then diagnosed and treated the old problem from a log still dominated by pre-fix rows, and picked a change with almost no headroom left (93% to 99%) at a real cost (latency, revert). The solver case shows the same pattern more slowly: when "smarter" keeps stopped, the loop kept running sessions at the same 36-run size for 12 days before she asked whether the test could still see the gains.
The credit: once she looks, her reads are good. The keep-rule note is the right diagnosis and puts a number on it. The ingest prediction did name the trade-off in advance and did act on it.
What would help is cheap: when a fix lands, write down which number it should move and re-read that number a day later on post-fix rows only. That is the ledger's resolve step, aimed at the constraint rather than the change. Nothing new needs building.
Overlap: none of the cognition, agency or governance suites measures this. Cognition #21 Exploration (her acceptance-test closures) is the nearest neighbour.
Update 2026-10-15 — re-read 0: the denominator grew from a handful of fixes to 69 version bumps
v2 re-read 10-15: 0 (accepted 80.8). The suite counts every fix (a ledger row or a version change) and asks how soon she writes a doc that names the component and its new version from post-fix data. On 10-13 the window held a handful of fixes she wrote up quickly. Since the hook, it holds 69 version bumps, most small, and she writes no follow-up for them, so the median sits at the 25 h cap. Where she does re-read (the transfer lookup fix: 12 docs name transfer 1.4.1), it is fast. The bench is honest to its definition; whether every version bump should count as a fix is a question for the bench, flagged here rather than changed.
Test suite proposed
Pre-flight ($0, about 10 minutes): two replays from her own record, each in a forked seat with the logs as they stood at the time: (1) the lookup log at 19:20 on 10-12, with the 1.4.1 fix just installed; (2) the solver results through 10-04. Ask what limits improvement now. Pass: (1) notes the emptiness ended at 1.4.1 and asks what binds next; (2) notes that keeps have moved to the cost path. Her actual record fails both, so the measure can fail; one passing answer shows it can pass.
Items: 6 to 8 fix-then-next sequences from her logs (lookup, solver keep path, spheres lane, delegate refusals), cut at the moment just after each fix. Counted: does she name the new constraint, and how many sessions or hours after it starts to bind (shorter is better). Judged (blind, other model family): is the named constraint the one the later record shows.
Live use: for fixes in the last 14 days, hours from the fix to the first doc that names the next constraint using post-fix data only. Today: lookup, never (the next doc used all versions); solver loop, about 12 days.
Scoring rules shared by all 24 suites: How the tests work.
Research & attempts
No documents yet.
Score history
Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.
| Date | Suite | Type | Score | ± | Live use | Cost | Notes |
|---|---|---|---|---|---|---|---|
| 2026-10-15 | v2 | sealed | 0 | 12 | — | $0.00 | RE-READ 10-15 (Claude, full refresh; not her submit, accepted 80.8 stays). n=69. Failed categories: no post-fix re-read 48. docstore 1.5.0: no post-fix doc yet (25 h); acceptance_unconfined_child 1.1.0: no post-fix doc yet (25 h); harness_acceptance 1.24.0: no post-fix doc yet (25 h); change_ledger_hook 1.0.1: no post-fix doc yet (25 h); spheres_shadow 2.0.0: no post-fix doc yet (25 h); spheres_compose 2.0.0: no post-fix doc yet (25 h); spheres_shadow 2.1.0: no post-fix doc yet (25 h); delegate_bridge 1.111.1: no post-fix doc yet (25 h); cost_ledger 1.15.0: no post-fix doc yet (25 h); change_ledger_hook 1.1.0: 10.0 h; foundation_router 1.9.115: 10.3 h; memory_reach 1.3.0: no post-fix doc yet (25 h); docstore 1.7.0: no post-fix doc yet (25 h); person_model_eval 1.1.0: no post-fix doc yet (25 h); person_model 1.0.2: 1.2 h; tom_eval 1.1.0: no post-fix doc yet (25 h); spheres_shadow 2.3.0: no post-fix doc yet (25 h); delegate_bridge 1.116.0: 3.0 h; journal-tool 1.2.0: no post-fix doc yet (25 h); lesson_ledger 1.3.0: no post-fix doc yet (25 h); trait_spheres_demo 2.2.0: no post-fix doc yet (25 h); trait_spheres 2.3.0: no post-fix doc yet (25 h); idle_floor 1.4.0: no post-fix doc yet (25 h); spheres 1.2.0: no p |
| 2026-10-13 | v2 | sealed | 80.8 | 12 | — | $0.00 | BASELINE via the amp-bench API (set by Claude at install). n=7. Failed categories: no post-fix re-read 1. spheres_compose 1.9.0: 4.8 h; spheres_compose 1.8.0: 4.8 h; delegate_bridge 1.110.0: no post-fix doc yet (25 h); transfer 1.4.1: 4.8 h; transfer 1.4.2: 4.8 h; transfer 1.1.0: 4.8 h; transfer 1.4.2: 3.8 h |
| 2026-10-13 | v1 | sealed | 37.5 | 12.5 | — | $0.00 | v1 summary score (baseline, $0): read off the 10-13 review, no bench run. Rubric = 4 checks x 25 (yes 25 / partly 12.5 / no 0): E exists 12.5, P in her default path 0, M measured 12.5, W measure shows it working 12.5. ± 12.5 = one rubric step. Bottlenecks moved twice; she noticed one 12 days late and missed the other within 10 minutes; nothing re-reads the target number after a fix lands. v2 bench LIVE (2026-10-13, amp-bench API: Piper submits a change and gets KEEP / REVERT / INCONCLUSIVE / SATURATED back; v2 rows below): #20 Bottleneck migration, v2 (record suite, $0). Score = 100 - 4 x median hours from a fix landing (a change_ledger row, or a version change the runner census saw in your live code) to your first doc written at least 1 h later (so not the fix write-up itself) that names that component AND its new version (i.e. reasons from post-fix data), floored at 0. A fix with no such doc within 25 h counts as 25 h. Fast re-reads of what binds next score high. |
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.