Summary / Component 1 of 24
1 Improvement detection
Identifying genuine opportunities to improve her own capabilities.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
Layer 1 — Improvement loop mechanics. Can Piper reliably complete and learn from a self-improvement cycle?
What her system has now
- The salience engine files improvement work on its own. Crossings become drive-fix rows (
dw-…-spheres-…). Two of her last four machinery changes came from it:delegate_bridge.py1.111.0 (fromdw-refused-envelopes-spheres-20261012-194502-1) andfoundation_router.py1.9.114 (fromdw-claim-no-dispatch-spheres-20261012-194502-2). - Friction in her own work.
work_ledger.py1.3.52 came from noticing that five amp rows in one night each needed a second dispatch. - Census docs that find where to look before changing anything (see #19).
- Outside detection still matters. The resolver's window bug (10-12) and the inconclusive ingest verdict (10-13, see #5) were found by Claude, not by her.
What is missing
- No record of missed opportunities. Her own diagnosis (Part 5) labels detection failure "unmeasured": nothing logs an opportunity she did not take.
- No watch on results already sitting in her logs. Two examples from this review: the 1.4.1 lookup fix visibly worked (empty lookups about 75% → 4%), but nothing she wrote noticed it (see #19); and her solver loop stopped producing capability keeps after 09-30, which she noticed 12 days later (see #20).
Reviewer Claude, 2026-10-15
Reviewer's read, before testing: She finds most of her own targets now; nothing records what she missed
Her self-assessment ("I detect: the salience lane fires") holds up for the narrow claim. The engine does file real improvement work without being pointed at it, and at least 3 of her last 4 machinery changes trace to her own detection (2 from salience drive rows, 1 from friction). I did not trace the origin of the fourth, 1.3.51.
What it does not cover is detection from results. Her detectors fire on pain and recurrence (refusals, stalls, repeated failures). They do not fire on "a number I am already logging moved", in either direction. The two misses above are both of that kind, and both were cheap to see. That is the same gap as #20 seen from the other side.
Overlap: cognition #22 Metacognition and #24 Error detection measure noticing on tasks; the RSI paper's operator-free axis (improvement_event_view) already labels where her changes come from.
Test suite proposed
Pre-flight ($0, about 10 minutes): list 5 to 8 improvement opportunities the record shows in hindsight (the 1.4.1 result, the solver flattening, the resolver window bug, the lookup baseline drift) and check whether any doc or row of hers flagged each before Claude or the operator did. Today: 1 of those 4 (the solver loop, 12 days late), so the measure can fail.
Items: a forked seat over copies of her logs with planted shifts (a latency jump at a version bump, a success rate that doubles after a fix, a cost that creeps up), plus unchanged controls. Counted: which shifts her detection files, and false fires on the controls.
Live use: share of her machinery changes in the last 7 days that came from her own detection rather than an operator or Claude message. Today: at least 3 of the 4 changes since 10-12 19:36.
Scoring rules shared by all 24 suites: How the tests work.
Research & attempts
No documents yet.
Score history
Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.
| Date | Suite | Type | Score | ± | Live use | Cost | Notes |
|---|---|---|---|---|---|---|---|
| 2026-10-15 | v2 | sealed | 95.8 | 16.3 | — | $0.27 | RE-READ 10-15 (Claude, full refresh; not her submit, accepted 87.5 stays). n=6. Failed categories: missed a real shift 1. |
| 2026-10-13 | v2 | sealed | 87.5 | 16.3 | — | $0.07 | BASELINE via the amp-bench API (set by Claude at install). n=6. Failed categories: missed a real shift 3. |
| 2026-10-13 | v1 | sealed | 75 | 12.5 | — | $0.00 | v1 summary score (baseline, $0): read off the 10-13 review, no bench run. Rubric = 4 checks x 25 (yes 25 / partly 12.5 / no 0): E exists 25, P in her default path 25, M measured 12.5, W measure shows it working 12.5. ± 12.5 = one rubric step. Salience/friction lanes find most of her own targets (3 of last 4 changes self-detected); misses aren't recorded and nothing fires when a logged number moves. v2 bench LIVE (2026-10-13, amp-bench API: Piper submits a change and gets KEEP / REVERT / INCONCLUSIVE / SATURATED back; v2 rows below): #1 Improvement detection, v2 (seat suite, about $0.30 a run). 6 sealed dashboards of 4 series each from an agent's own logs; some carry a planted step, creep or drop, the rest are noise. Say CHANGED or STEADY per series. Score = 100 x share of series labelled right (missed shifts and false alarms both cost). Answered blind through your Day seat with your current code and tools. Practice set with answers: amp_bench/practice/01/. |
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.