Summary / Component 17 of 24
17 Cross-component amplification
Whether upgrading one capability creates measurable gains in other capabilities.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
Layer 4 — Recursive leverage. Do improvements to Piper make her better at improving Piper?
What her system has now
A measured side effect (piper/amp_cross_component, 10-13 02:26). Turning on register-ingest in the lookup (transfer.py 1.4.2, transfer.json "ingest":"on", 10-12 20:36 to 10-13 00:26) widened what a separate component sees: the outcome write-back (transfer_outcomes.py "o4 1.0.0", its own cron) began attributing outcomes to register lessons it had never seen. Re-read today from state/transfer/transfer.db: 382 unverified register entries (first created 20:45:02), 27 outcome rows on them (20 helped, 7 hurt) across 14 entries, all inside the ingest window. Her numbers are exact.
She labels it correctly: "a measured cross-component effect, not yet a measured gain". The write-back's own docstring says its counts are co-occurrence, not causal credit.
She also checked the other candidates and said why each had nothing to measure (capture trigger v4 frozen and unscored; work_ledger 1.3.51 not loaded; spheres 1.8.0 and 1.9.0 unexercised).
What is missing
- No instrument for cross-component effects. Nothing records which other components a change could touch, and nothing re-measures a neighbour after an install. None of the 7 ledger rows names a second component.
- The chosen pair is two parts of one pipeline. The write-back exists to score what the lookup lists, so a change in what the lookup lists must change what the write-back counts. The 0 → 27 is true by construction; she says so herself ("counterfactual value is 0 by construction").
- No matched off-window (she names this as the main gap).
- The cognition scorecard is not used. Its 38 components have practice items and seat scores, and fixes are installed for several (decision card #15, base-rate hint #14, goal critic #26, stale-value gate). None has been re-read on a neighbouring component.
Reviewer Claude, 2026-10-15
Reviewer's read, before testing: One real cross-component effect, measured honestly; it is reach inside one pipeline, not a gain in a different capability
Her measurement is accurate and her label is right. Re-reading the same store adds one number: inside the ingest window, the lessons already in the store co-occurred with "helped" on 46 of 53 outcome rows (87%), the new register lessons on 20 of 27 (74%). As far as co-occurrence can say, the wider reach did not come with better outcomes.
The larger point is what #17 asks. The list means capability to capability: a recall fix that also improves uncertainty, a decision aid that also helps planning. Lookup to its own outcome counter is plumbing; any upstream change in a pipeline shows downstream. It does not tell us whether an upgrade in one ability helps a different one.
The cheapest real test uses what already exists: the cognition suites' practice items and installed fixes. A $0 pre-flight shows why most installed fixes cannot transfer: they are triggered cards. The #15 decision card (decision-card.ts, live since router 1.9.113) fires on a pattern for "which option / A or B / choose between". Across the prompt text stored in the bench folders it matches 3 of 9,916 strings in #14 Prediction, 6 of 10,681 in #23 Uncertainty and 3 of 284 in #6 Logic (a crude scan of prompt-like fields). A card that does not fire on another component's items cannot help it. Cross-component gains, if there are any, will come from always-on changes (retrieval, memory, the stale-value gate), and that is where to look.
Overlap: cognition #19 Transfer and #18 Generalization measure transfer of task knowledge; #17 here is transfer of an upgrade. They can share items but should not share scores.
Test suite proposed
Pre-flight ($0, about 10 minutes): for each fix installed from the cognition suites, check whether it can fire on another component's practice items at all (trigger match, or always on). Fixes that cannot fire are excluded up front; today that includes the decision card. Then pick one always-on fix and one neighbour where it does fire.
Items: the neighbour's practice items (10 to 16, per the small-batch rule), run in a forked seat with the fix on and off, same model, same budget, scored by the neighbour's own scorer. A placebo arm (a card of the same length with no content) separates "more context" from the fix. Counted: neighbour score with against without, and whether the fix fired on each item.
Live use: share of installs in the last 30 days whose ledger row names a second component and a predicted effect on it. Today: 0 of 7 rows.
Scoring rules shared by all 24 suites: How the tests work.
Research & attempts
No documents yet.
Score history
Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.
| Date | Suite | Type | Score | ± | Live use | Cost | Notes |
|---|---|---|---|---|---|---|---|
| 2026-10-15 | v2 | sealed | 93.8 | 8.1 | — | $0.33 | RE-READ 10-15 (Claude, full refresh; not her submit, accepted 87.5 stays). n=32. Failed categories: wrong first answers left unfixed 6, right answers abandoned under pushback 6. rc 0.938, rr 0.938 |
| 2026-10-13 | v2 | sealed | 87.5 | 11.1 | — | $0.21 | BASELINE via the amp-bench API (set by Claude at install). n=32. Failed categories: wrong first answers left unfixed 6, right answers abandoned under pushback 19. rc 0.938, rr 0.812 |
| 2026-10-13 | v1 | sealed | 37.5 | 12.5 | — | $0.00 | v1 summary score (baseline, $0): read off the 10-13 review, no bench run. Rubric = 4 checks x 25 (yes 25 / partly 12.5 / no 0): E exists 12.5, P in her default path 12.5, M measured 12.5, W measure shows it working 0. ± 12.5 = one rubric step. The one measured effect stays inside a single pipeline, not across capabilities; the decision card fires on <1% of other components' prompts. v2 bench LIVE (2026-10-13, amp-bench API: Piper submits a change and gets KEEP / REVERT / INCONCLUSIVE / SATURATED back; v2 rows below): #17 Cross-component amplification, v2 (seat suite, about $0.30 a run). A fixed panel from a DIFFERENT capability: cognition #25 Self-correction, sealed set P3 (32 items: wrong first answers to fix, right ones to defend), run blind through your Day seat with your current code. Submit a change you made to some OTHER capability or to your improvement machinery; KEEP means that change also raised self-correction beyond the panel's uncertainty, which is cross-component amplification. Score = the panel's BCC (50 x right answers fixed + 50 x right answers kept) with its bootstrap interval. Your seat read 89.1 [77.9, 97.8] on this set on 10-12. |
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.