Summary / 18. Generalization / Document
2026-10-12
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
**The short answer.** About 80 minutes of bench time and $2.80 of API spend. Headline (same rule as #17): **51.9** (DiD +46.2 pp, 95% [+21.8, +68.6]). What she learned in #17 carries over to new cases, including a different representation (JSON, logs, config); far cases improve less and not significantly. Once the lesson is retrieved she applies it (34/41); when it is not retrieved she never gets it (0/23). Instrument: 76 sealed items across the 12 #17 families (6 treated / 6 control). Items are written blind by Sonnet, key-checked by v4-pro and calibrated (the bare model without the rule must miss them; on lookalikes it must get them right). Distances: near 21, medium 15, far 21, neg 19. Arms: Piper's seat with no #17 episodes (none) vs with her transfer.py #17 episodes (xs, store g18), k=2, 304 turns, $2.80 DeepSeek (rsi-loop key, keygate bypassed with the operator's OK), 0 errors, 1 void. | Treated families | none | xs | diff (95% Newcombe) | |---|---|---|---| | near | 0/22 | 13/22 | +59.1 [+33.9, +76.7] | | medium (other representation) | 3/18 | 12/18 | +50.0 [+17.8, +70.2] | | far (adapted to a new decision) | 4/24 | 9/24 | +20.8 [-4.4, +43.0] | | all rule-applies items | 7/64 | 34/64 | **+42.2 [+26.6, +55.1]** | | neg (lookalike, rule must NOT apply) | 19/20 | 18/20 | 1 more false transfer | Control families (rule-applies items): 31/50 -> 29/50 (-4.0 [-22.4, +14.7]). Control neg: 17/18 -> 18/18. Flat, as it should be. **Mechanism (xs, treated, rule-applies):** when the transfer card fired: 34 of 41 correct; when it did not fire: 0 of 23 correct. Card fire rate: near 15/22, medium 13/18, far 13/24. So once the lesson is retrieved, she applies it well, even in a different representation. The limit is **retrieval**: the card doesn't fire when the rule's usual wording is missing (mostly far items). This matches F02 (retrieval not triggered) in her taxonomy. **Gaps:** R02 has no near or medium items; P09 has no medium or far; R03 and P08 have no medium (calibration yield). k=2 only. Far difference CI crosses 0. **For Piper (Work Order D):** pick ONE of I1-I4 aimed at retrieval on far / no-trigger-word situations (e.g. the card keyed on the decision type, not the surface words). Claude seals fresh far items for the retest; the current items stay sealed.