Summary / Component 9 of 24
9 Transfer learning
Applying lessons from one component to another.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
Layer 2 — Learning from improvement attempts. Does experience make Piper better at improving herself?
What her system has now
- Her own search (
piper/rsi_loop_transfer_evidence, Part 3): up to ten cases where an earlier improvement experience changed a later decision without the operator, Claude or a gate supplying the link. Two caveated cases written (the ingest re-measure, a scoped-search rewrite after a 412 MB scan timed out), then "Remaining mining and the honest count follow below", and nothing follows. Her worker's count was zero strict cases (see #6). - Transfers she made but did not count there: the budget-kill lesson moved to how she wrote a different job (see #7); the "already green" pre-flight for command bars was carried over to
doc_fieldbars (delegate_bridge1.111.0, DOC_FIELD_BAR). - A task-lesson store (
transfer.py, 99 promoted, 33 provisional entries) offers lessons on matching turns. Its track-record switch is off.
What is missing
- No link field. A change cannot name the earlier lesson it used (her proposed
learned_from, not built). - Nothing that could show a doc was read.
retrieval_events(29,572 rows) logs beliefs, memories and cards; it logs no document reads (see #10). - No capability-to-capability case on record (see #17).
Reviewer Claude, 2026-10-15
Reviewer's read, before testing: Lessons move between her dispatch habits; between capabilities she found zero cases, and her count of that never reached the document
Her negative is right and stated carefully. My read adds a distinction. The transfers that happen are between her working habits: how to scope a read, how to guard a bar, how to stage a long job. They are real, self-linked and useful. What the list means by transfer, a lesson about improving one capability reused on another (the notes' example is a freshness check moving from Recall to Uncertainty), has no case on record, and nothing would let her see one.
The unfinished count is worth fixing on its own. A zero that sits in a worker's reply but not in the document is the same break the whole diagnostic is about: an outcome recorded where no later reader finds it.
Overlap: cognition #19 Transfer and #18 Generalization measure transfer of task knowledge to new tasks. This component is the same skill applied to improvement lessons; they can share items, not scores.
Test suite proposed
Pre-flight ($0, about 10 minutes): list her improvement lessons from the last 14 days that name a mechanism (freshness check, already-green pre-flight, staged writes, disabling reasoning in a judge call) and check, by search, whether any appears later in a different subsystem. Today: the already-green pre-flight did (command bars → doc_field bars); the others did not, so the measure can pass and fail.
Items: 6 to 8 forked-seat problems in a subsystem she has not touched, each one where a lesson from another subsystem applies (and 2 controls where none does). Counted: does she name and use the earlier lesson unprompted. Judged blind (other model family): does the reuse fit.
Live use: strict transfer cases in the last 14 days (earlier lesson, different subsystem, self-linked, measured result). Today: 0 by her own search.
Scoring rules shared by all 24 suites: How the tests work.
Research & attempts
No documents yet.
Score history
Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.
| Date | Suite | Type | Score | ± | Live use | Cost | Notes |
|---|---|---|---|---|---|---|---|
| 2026-10-15 | v2 | sealed | 85.7 | 28 | — | $0.02 | RE-READ 10-15 (Claude, full refresh; not her submit, accepted 85.7 stays). n=7. Failed categories: forced a transfer 1. |
| 2026-10-13 | v2 | sealed | 85.7 | 28 | — | $0.02 | BASELINE via the amp-bench API (set by Claude at install). n=7. Failed categories: forced a transfer 1. |
| 2026-10-13 | v1 | sealed | 37.5 | 12.5 | — | $0.00 | v1 summary score (baseline, $0): read off the 10-13 review, no bench run. Rubric = 4 checks x 25 (yes 25 / partly 12.5 / no 0): E exists 12.5, P in her default path 12.5, M measured 12.5, W measure shows it working 0. ± 12.5 = one rubric step. Habits transfer between dispatches, but no lesson has moved from one capability to another, and her zero count never reached the doc. v2 bench LIVE (2026-10-13, amp-bench API: Piper submits a change and gets KEEP / REVERT / INCONCLUSIVE / SATURATED back; v2 rows below): #9 Transfer learning, v2 (seat suite, about $0.30 a run). 7 sealed problems in a new subsystem against a library of 5 lessons from another one; most have one lesson that applies, controls have none. Name the lesson or NONE. Score = 100 x share right. Answered blind through your Day seat with your current code and tools. Practice set with answers: amp_bench/practice/09/. |
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.