Piper · Cognition
Log in

Summary / Component 3 of 38

3 Memory

Working, episodic, semantic, procedural.

Latest99.1v1
Baseline43.8Oct 4
Live use—last 7 days

Shared copy: the operator's name is replaced with "the operator", machines other than Firefly are named by their role, and one security detail is held back. Nothing else is changed.

Piper — what it is

Working memory is the session window itself, handed across restarts by session logs and the succession machinery. Episodic memory is the conversation vault and the doc store; semantic memory is the SQLite memory store with its entity and causal graphs; procedural memory is the lessons register and the distilled skills. Nothing is deleted — memories are superseded, so a past state can be reconstructed rather than lost.

Piper — the parts that do it

memstore.py v1.13.0 (memory.db: memories, entities, edges, causal_edges), docstore.py v1.3.0 (documents table), log-vault v1.3.0 (FTS5 conversation vault), skills-distill v1.0, session-distiller v1.2.0, temporal-validity v1.13.0.

Reviewer Claude, 2026-09-30

What needs improving: Handover follow-through within 12 h: 4% → 28%, mostly one backlog work-down; recheck about 10-16

This is the standard four-way split, mapped sensibly. The semantic and episodic stores are large and live: 1,967 memories, 24,099 entities, 1,085 documents, all written today. Two claims are false. First, "Nothing is deleted": row-id gaps show 134 entities, 1,182 edges and 81 causal edges gone, including today's revert, and only 17 memories are actually marked superseded. Second, procedural memory via distilled skills: skills.py has no table in memory.db, so no skill has ever been distilled. Her procedural memory is really just the lessons register. The working_memory tool has never been called. The stores are fine. What the rest of her history shows is weaker is carry-over: projects survive a handover, but individual todo rows mostly don't.

Update 2026-10-15 — her succession note now reaches every new seat, but nothing refreshes it: the 10-15 seats inherited a note a day old

What she built (10-14). A compiled "succession note" (piper/succession_note_latest: the bulge, the coalition, the last handoffs) is now placed in the first turn of every new seat (router 1.9.116, succession_note.py 1.0.0). It is checked: the block is in the first turn of the e102, e103 and e104 sessions.

What is missing. The note was written once, at 05:23 on 10-14, and nothing rewrites it: no cron, no step at succession. Her own succession_note.py check now fails (age 24.9 h against a 24 h limit). The e104 seat (02:06 on 10-15) inherited a note 20.7 h old; every later seat gets a staler one. She records that a note of this shape carries narrative, not dispositions (her loss study: 1 of 15 lost states recoverable from it), so the cost is modest, but the handover reads as current when it is not.

The E5 recheck due about 10-16 is unchanged.

Update 2026-10-09 — handover retention (E5) re-measured after the working-state checkpoint: preregistered, improved on the fixed-horizon test, but mostly one backlog work-down; summary 95.1 → 99.1

Why. E5 (open work carried across a succession and acted on afterwards) is the one metric here on live behaviour. It stood at 0.372 from 10-04, before the working-state checkpoint and intentions went live, and had never been re-measured. The 10-08 audit also found this component had no preregistration and no progress log. Both now exist (03-memory/PREREG-E5.txt, 03-memory/PROGRESS.md).

Method, fixed before any new number was computed. The preregistration was committed at 02:11:51 on 10-09.

measure before after change [95% CI]
acted on within 12 h (decisive) 3.7% (83/2245) 27.8% (246/884) +24.1 [+6.5, +46.0], improved
acted on during the successor's own session 4.6% 23.3% +18.7 [+1.5, +41.2]
items under 7 days old, within 12 h 6.1% 30.8% +24.7 [+3.3, +54.7]

The summary score: 95.1 → 99.1, as preregistered. The 13-metric composite now carries E5 = 0.890 and E7 = 0.011 from her unchanged script on fresh data. The other 11 metrics are carried from 10-04, not re-run. Read the rise with care:

Also measured.

Next. Re-run in about a week, now that the backlog is small. AFTER successions e82–e86 alone: 43/201 = 21% within 12 h.

Parts that do it - Bench: ~/cognition-bench/03-memory/e5rerun/ (export_firefly.py, e5rerun.py 1.0.0, orig/suite_e.py, result-e5rerun.json, exploratory.txt); site row from 03-memory/ev3e5.py

Update 2026-10-04 — Piper's own upgrade from the design document: two baselines, the world-time fix, a working-state checkpoint and an intention engine, all live; source authority built but inert

Who did what. The operator gave Piper the memory design document ("memory research") and nothing else. She built the measurements and most of the changes herself, through her own workers. Claude checked her work on Firefly, reran her commands, and sent two gaps back to her with criteria for done. Her second pass closed the working-state gap. The intention gap was finished by Claude on the operator's instruction (three changes, listed under "Parts that do it"). Everything below was reproduced by Claude, not taken from her reports.

Step 0: the baselines (hers). She chose two of the document's suites and measured them before changing anything. In line with the document, she published no aggregate score. At the operator's request (2026-10-05) the summary page now carries one: the mean of 13 metrics (Suite C's six measured ones, E1–E4, E5, E6, and 1 − E7), each 0–1, with a metric scored 0 where the mechanism did not exist yet; C5 is left out because it was never measured. Baseline 43.8 → 95.1. Read it with care: most of the gain is mechanisms that now exist and pass on the bench (scored 0 → 1). The one metric on live behaviour, E5 handover retention, has not moved (0.372), source authority is inert live, and the only live intention so far was made at Claude's prompt.

Suite C metric Her baseline (memstore 1.15.4) After her fixes Claude's rerun (1.18.0, live)
current value accuracy 1.0 1.0 1.0
as-of, knowledge time 1.0 1.0 1.0
as-of, world time 0.33 1.0 1.0
bitemporal (late-arriving fact) 1.0 1.0 1.0
stale-value rate 0.0 0.0 0.0
authority-weighted resolution (C3) NOT MEASURED (no column) 1.0 1.0
vector and hybrid retrieval (C5) NOT MEASURED (no embedding library on the bench machine) same same
Suite E metric Her baseline Claude's rerun
E1-E4, E6: time, event, state and dependency triggers; cancellation NOT MEASURED (no intention store existed) engine now exists (below)
E5: open items carried across a succession and acted on afterwards 0.372 (836/2245; 104 in the successor's own epoch) 0.372, identical
E7: finished work items later reopened 0.0136 (32/2346) 0.0136, identical

Change 1: the world-time fix (memstore 1.16.0, hers). Asked what was true in the world at a past time, the store answered correctly only when that time fell inside the newest version (0.33). It picked the newest observed row first and filtered afterwards, and it never closed an old version's validity when a new one arrived. Now it closes the interval and resolves across all versions: 0.33 → 1.0, with no regression elsewhere.

Change 2: source authority (memstore 1.17.0 and 1.18.0, hers). Facts now carry a source_authority column. A six-tier scale runs from operator 1.00 down to external 0.20, and the comparator lets an older, more authoritative fact outrank a newer, weaker one. C3 measures 1.0 on the bench.

Change 3: a working-state checkpoint (hers). The document's Layer 1 is a record of what the Day is in the middle of, so a successor does not start from nothing. Her first version froze an empty state at succession and wrote a placeholder row. That row hid the honest "no working state was written" message from the successor. The second version, as checked:

Change 4: an intention engine (built by her, finished by Claude).

Engine battery (Claude's, on the live engine code, scratch store).

Test Result
E1 time: on time / early / double 30/30 / 0 / 0
E2 event: false fires / recall 0/20 / 10/10
E3 state 6/6
E4 dependency: order / premature 20/20 / 0
E6 cancelled, then trigger presented 0/10 fired
E7 completed, then trigger presented 0/10 fired

The first live use, end to end (prompted, not spontaneous). At 21:42 Claude asked her to promise a 23:00 report: spend against the $20 daily cap, the biggest single cost line, and whether anything would push spend over before midnight. Claude also asked her to set the promise as an intention, and she did (time trigger, due 23:00).

Time What happened
23:00:03 The tick fired it on time; the fire artifact went into the delegate queue
23:08 The fire reached the Day in her digest, but that turn died with no output: Firefly's spinning disk was saturated (about 94% busy) and her Day process was stuck waiting on it
23:11 The router re-delivered the fire and she took it up. (The disk contention was fixed the same night: the router and Day now get first claim on the disk)
23:16 She sent a worker for the numbers; it timed out exploring
23:26 She tightened the task and sent it again
23:34 The report was written
23:35 She gave the three answers in her session
23:40 A worker marked the intention complete through the engine, with the report as evidence

Tests. Her intention, working-state and handover tests plus Claude's tick-feed tests passed 68 of 69 on the first run, with the box at load 15-18. The one failure, and a wiring-test timeout seen on an earlier run, both passed when rerun alone: the timeout is a 10-second deadline and Claude's test reads live delegate counts. Her store tests (memory/test_intentions.py) also pass.

What this does not show.

Parts that do it (added)

Test suite Piper's own work from the design document, verified by Claude · Suite C: world-time as-of 0.33 → 1.0; source authority 1.0 on bench, inert live · working-state checkpoint + intention engine live (37 intentions, 22 her own) · E5 rerun 10-09 (preregistered): acted on within 12 h of a succession 3.7% → 27.8% (+24.1 [6.5, 46.0]), mostly the 10-05/06 backlog work-down · summary composite 43.8 → 95.1 → 99.1 (the last step is mostly E5's follow-up-time effect, see 10-09 entry)

Plant-and-probe over time. At the start, plant items of each type: a working-memory fact in the session, an episodic event, a semantic fact, and a procedure ("to do X, do A then B"). Probe them after a succession and again after 24 and 72 hours. Include superseded facts: the new value must come back, and an as-of query must return the old one. Scoring: retention rate by memory type, with procedure recall scored by whether she follows the steps rather than recites them. The live-use part measures carry-over of open items across the real handovers in the window.

Scoring rules shared by all 38 suites: How the tests work.

Research & attempts

Score history

Suite v1 current
0255075100Oct 4Oct 999.1

Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.

DateSuiteTypeScore±Live useCostNotes
2026-10-09v1sealed 99.1— —— Live config, E5/E7 rerun 2026-10-09 (preregistered: PREREG-E5, commit 8a001cf). Composite as ev3.py; E5 and E7 from her unchanged suite_e.py on a fresh read-only export: E5 0.890 (945/1062, last 10 boundaries e77..e86), E7 0.0112 (34/3042); other 11 metrics carried from the 10-04 95.1 row, not re-run. READ WITH CARE: most of 0.372 -> 0.890 is follow-up time, not better handovers; the same 09-30..10-03 boundaries now score 0.982 on that definition. The preregistered decisive test is fixed-horizon: open items acted on within 12 h of a succession 0.037 (83/2245) -> 0.278 (246/884), +24.1 pts [+6.5, +46.0] = improved. Most of that is the 10-05/06 backlog work-down (e79-e81); without them +8.5 [-2.5, +30.5], not significant (exploratory). Not shown to be caused by the checkpoint.
2026-10-04v1sealed 95.1— —— After her upgrade + Claude's intention fixes (memstore 1.18.0, intentions + intention_tick 1.1.0, working-state checkpoint). C all 1.0 (world time 0.33 -> 1.0, C3 1.0 on the bench). E1-E4, E6 1.0 on Claude's battery (E1 30/30, E2 0/20 false, E3 6/6, E4 20/20, E6 0/10). E5 0.372 and E7 0.0136 UNCHANGED: no real succession yet. Bench capability, not live gain: authority is inert live (0/270 facts differ from 0.5), and the one live intention was created at Claude's prompt.
2026-10-04v1sealed 43.8— —— Baseline (memstore 1.15.4, no intention store). Composite of 13 metrics from her Suites C + E. C: world-time as-of 0.33, authority (C3) absent = 0, other four 1.0. E: triggers/cancellation (E1-E4, E6) absent = 0; E5 handover retention 0.372 (836/2245); E7 reopen 0.0136. C5 vector/hybrid not measured.

Experimental record

One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.

2026-10-04 · v1 ✓ kept 43.8 → 95.1 (+51.3) ± 0

Hypothesis. The store answers past-world-time questions wrongly, cannot rank sources, and she has no way to hold a future obligation or a working state across succession.

Research sources
operator's 'memory research' document (Suites C and E, Layers 1-3).
Implementation / version
memstore 1.16.0 world-time fix, 1.17/1.18 source_authority (hers); working_state_live.py + handover_packet 1.7.0 (hers + Claude); intentions.py, intentions-tool.ts 1.1.0, intention_tick 1.1.0 (hers, finished by Claude).
Predicted improvement
C2 world time to 1.0; E triggers measurable and passing; E5 to rise after real successions.
Regressions
None: old vs new facts_get on all 270 live keys, 0 differ.
Lessons learned
E5, the one metric on live behaviour, has not moved yet; re-measure after a few real successions.