4 Recall / retrieval

Update 2026-10-19 — belief gate reranker filter installed: off-topic matches are dropped before they reach her

The operator chose to install the second PREREG-BG arm. It had passed its held-out test, but it was held back because it trades some relevant beliefs for much less junk. Each belief the gate matches is now scored against the message content by a small local reranker, and dropped if it scores below -7, the threshold frozen in the preregistration.

• Held-out, 300 prompts, against the old gate: relevant beliefs per turn 0.39 to 0.67 (+0.14 to +0.43), junk 4.59 to 1.70 (-3.09 to -2.68).
• Against the content seed alone: about a quarter fewer relevant beliefs, and less than half the junk.
• Live checks: the installed scorer reproduces the test's scores (largest difference 0.00005; same keep or drop decision on 96 of 96 items). The gate call takes about 1.5 s per turn, offline. A turn whose every match is dropped injects nothing; a reranker error falls back to the unfiltered matches.

Update 2026-10-19 — auto-recall turned on (about 1 relevant memory per triggered message) and stale beliefs now retire themselves

Auto-recall (PREREG-AR). When one of the operator's messages looks like a claim about the past or about system state, auto-recall puts a short block of her own memories in front of it. It had been off since 29 September, after a timeout problem she later fixed. Tested on her real messages, with the prereg frozen after dev and before any held-out label:

• Held-out, 136 messages from 13 to 18 October: the block carries 1.07 relevant memories per message (95% interval 0.93-1.21) and 0.97 junk. Precision is 52%, against a bar of 40%. KEEP: turned on, with no code change; it takes effect at once.
• Its three beliefs never fit the 700-character budget, so the block is memories only. The belief gate already supplies beliefs on every turn.

Staleness rule. Her invalidation pass already marked beliefs stale every 10 minutes: 1,280 provenance observations past their 2-hour lifetime. Nothing acted on the mark, so they stayed in the active ledger and were rewritten on every pass. The pass now retires a belief once it has been stale for a day past its lifetime (reversible; the row is kept). The first pass retired 1,217. The active ledger is now 8,457 beliefs, down from 10,745 before this round.

Tested and dropped: an age cut at the belief gate. On dev, skipping beliefs older than 14 days removed relevant beliefs about as fast as junk (precision 25% to 30%), so old beliefs are mostly old topics, not stale facts. Not taken to held-out.

Update 2026-10-19 — per-turn belief injection fixed: it now searches on what the message says (relevant beliefs per turn 0.39 -> 0.93, then 1.43 after a ledger cleanup)

Where recall reaches her turns. Her direct memory tools run about 1 to 15 times a day. The retrieval that reaches every turn is the belief gate: before each turn it searches her belief ledger and injects up to five beliefs. It searched on the first 400 characters of the prompt, and most prompts open with machine templates: digest status, lesson counts, the reply-format rule, her own record block, clock and state lines. In 3 days, 138 of 536 injections were one identical set of five.

Test (PREREG-BG). Frozen after the dev split and before any held-out label: 300 of her real prompts from 13 to 18 October. A blind judge rated each injected belief relevant or not, with the arm hidden.

• Today's gate: 0.39 relevant and 4.59 junk beliefs per turn (precision 8%).
• Searching on the message content: 0.93 relevant (+0.41 to +0.68) and 3.85 junk (-0.89 to -0.59). KEEP, installed. The gate change is live; the matching change in her memory tools loads at her next session start.
• A small reranker filter on top also passed (0.67 relevant, 1.70 junk) but gives back about a quarter of the relevant beliefs. Not installed: the operator's call.

Ledger cleanup. Most of the remaining junk came from the ledger itself. An automatic extractor had turned every resolved forecast into belief fragments (1,060 beliefs from 1,000 forecast records), and 11 whole documents had been stored as beliefs. The extractor now skips forecast records, and those 1,071 rows were retired (not deleted, reversible). On the same 300 prompts (exploratory, not preregistered): 1.43 relevant and 2.82 junk per turn (precision 34%).

Closed. The r4route router's shadow week on her real traffic showed no difference: it won 21 of 39 decisive comparisons (54%, interval 39-68%), so it is not installed. Reranking her direct searches with a small cross-encoder gave no gain on the dev split (55.3 against her habit 56.8).

Not changed: the headline score. It measures the direct-search paths on the sealed synthetic set, which this work did not touch.

Update 2026-10-15 — three live fixes for memory she had but did not use, and her own first before/after check of the recall-for-concepts rule (too little data yet)

Live since the last entry (all with off switches; none re-scored on the sealed bench, which stays at 51.3): - 10-13, router 1.9.115: a reply check holds a reply that answers the operator's "you already know / we talked about / do you remember" without any memory search that turn, and has her rewrite it once. On 5,738 replayed turns it held 3, all real no-search answers. - 10-13/14, memory_reach 1.3.0 → 1.4.0: the real cause of the 10-13 "you forgot" complaints was that her reach filter dropped the operator's own rows. 1.4.0 injects an operator thread, last 5 days block (his words from earlier sessions, one line each, secrets redacted) once per session. - 10-14 14:05, her own change: after tracing why recalled items go unused, she wrote a recall-for-concepts rule into her charter.

Her before/after check (10-15 04:52, d-3cb93f), same three-way item trace: before the rule 11.8% used-or-cited (392 of 3,331 items, 493 calls; the standing 11.6% reproduces); after the rule 0.0% (0 of 25 items, 4 calls). Four calls is far too few to read either way. Note: this check followed an idle-time steer from Claude at 04:28 that asked for it; it was not self-started.

Update 2026-10-09 — live use: she could not see her own day's work. Searches crashed on a busy database, never looked in her docs, and the conversation summary dropped the operator's words. Fixed live 23:38; no bench score yet

The operator reported at 22:51 that she could not recall her own work from that morning: the 08:22 "russian plague" finding and whether that day's RSI challenge had succeeded. The memory was there. She could not reach it.

Diagnosis (read-only, reproduced on her machine). 1. Nothing carried her own day's work into her turns. Her docs, journal and RSI result files were found only by exact name. Her session restarted at 13:58, so everything from the morning dropped out of view. 2. Her conversation summary showed digests, not the operator. All 6 summaries that evening showed the same automated digest line. A label filter was missing, and the 120-character cutoff kept the digest and dropped what the operator wrote. 3. A memory search could crash. Every search also writes a usage counter. When a background job held the database for more than 30 s, the search failed outright: 9 failures and 10 timeouts in a week, including the 22:55 "russian plague" search. 4. Her search tools never read her docs. They covered 2,408 short memories but not the 1,196 docs where her findings actually live. 5. The operator's 08:39 "top priority" was lost. Nothing kept it in view, and by 12:47 she was clearing her own self-generated warnings.

The diagnosis went to her doc store as a design for her to own. Within minutes she built her own "my record, last 24 h" block and a digest filter, live 23:21. On the operator's go, Claude's fix was merged on top of her version, not over it.

What changed (live 23:38, router 1.9.108): - memstore 1.24.0. The usage-counter write waits 2 s and, if the database is still locked, is queued instead of failing the read. Tested with the database held locked: the search returned normally. search and fused now also return recency-weighted hits from her docs. "russian plague" and "rsi challenge succeed today" return the right docs in the top 2 (about 0.4 s). Off switch: PIPER_SEARCH_DOCS=0. - memory_reach 1.2.0, on top of her 1.1.0. Four fixes: - Her record block would have been read back as operator speech from her next session on. It is now stripped. - The tail used to repeat every turn. It now shows once. - The summary finds earlier sessions by start time and reads the whole file, so the morning is no longer lost. - The record now also lists RSI result files, journal entries by their own times and open operator priorities. Loop output is left out. - operator_priorities 1.0.0 (new). A message that declares a priority ("top priority", "fix this first") is pinned. It shows as a [priority] line on every turn, including her self-directed work turns. She can close one only with evidence. The 08:39 message was seeded as the first.

What was left alone. Auto-recall has been off since 09-29, with no recorded reason. Its scorer is what the sealed v2 bench measures, so it was not re-enabled or re-ranked. That is the operator's call.

What this does not show. No sealed score: the v2 bench measures retrieval ranking, which this fix does not change. The test is in live use. After a fresh session, ask "did your rsi challenge succeed today?" and "tell me about the russian plague". She should answer both in her first reply, with no lookups. Her own summary check fails on both her 1.1.0 and on 1.2.0: it assumes an empty block, so the check needs updating, not the code.

Parts that do it - Her machine: memory/memstore.py 1.24.0, core/memory_reach.py 1.2.0, core/operator_priorities.py 1.0.0 + state/operator_priorities.json, router 1.9.108. Backups are beside each changed file (.bak-claude-memfix-20261010). - Working copy with the pre-install snapshot: ~/piper-memfix on the bench machine.

Update 2026-10-08 — installed: history-aware recall (+15.8 on fresh real questions, outdated answers never outrank current ones) and a repaired graph lookup (2.4 → 30.5); live configuration now 51.3

The operator asked for the +22 from the history-aware recall, and for her to get value out of the graph path. Both are now live, after preregistered tests on fresh questions.

1. History-aware recall, second version (memstore 1.22.1). One rule was added to the first version: an outdated memory can never rank above the memory that replaced it. If both are found, the old one moves to directly below its correction. That was the only reason the first version was not installed.

The test used 42 new sealed questions on her real memory (41 survived the blind rewording). The history and current-must-win questions are about corrected memories the first test never used where possible, and the rest re-ask about the first test's memories from new angles. The event questions use dates the first test never used.

• All six preregistered conditions passed. That includes a new independent check on the synthetic store, which has 125 corrected memories: wrong-version answers stayed at 0.
• On the first test's 40 questions, the gain holds (+21.1) and the outdated-above-current rate falls from 14.7% to 0%.
• It works only for her explicit memory commands. Internal callers that treat any hit as current truth (deduplication, auto-recall, prompt context) are unchanged.
• Scale: her store has only 25 corrected memories, so the history part helps a rare kind of question. The date reading helps every question that names a date.

2. The graph lookup, repaired (hippo v2). The earlier seed fix had been measured only on the synthetic store. On her real memory the walk still drained into junk hub entities (an entity called "i" with about 989 links), so unrelated questions returned the same few memories. It also skipped the memories of the very thing named in the question, and never ranked its results.

The new version: - counts the named entities' own memories; - ignores junk entities; - discounts hubs by how many memories they link to; - ranks memories by their total score.

On the 41 real held-out questions, the graph path went from 2.4 to 30.5 (+28.0 [+15.9, +41.5], 13 better, 0 worse), with fused +1.8 and her habit unchanged. On the synthetic store, the graph path went from 37.5 to 51.4.

What was not done. Mixing graph results into recall (her most-used call) was tested on the real-history set first. It gave no gain, because her plain recall already finds what the graph finds on real questions. So graph now improves the paths that already use it: fused, direct graph calls, and the router's graph trigger after its next restart. It does not compete inside recall.

Live. Both changes ship as one file, merged onto her current memory code so her own latest edit is kept. A backup sits beside it, and each change has its own off switch. Her memory tools load the file fresh on every call, so the change took effect at once. A live check confirmed that a corrected memory now appears, labelled, directly under its correction. On the v2 synthetic set the live configuration scores 51.3: that set has few history or dated questions, so the real-memory gains above are the ones that matter.

Update 2026-10-08 — history-aware recall: a large gain on her real history (+21.7 [+12.4, +31.7]), but an outdated memory sometimes outranks the current one, so it is not installed; the summary now shows her live configuration (50.8)

This follows the archaeology finding: once a memory is corrected, its earlier version disappears from every retrieval path except a dated as-of query, and dated questions are all read as "state at that date".

The change (memstore 1.22.0, built on the bench machine only, behind a flag that is off by default). - A superseded memory may take up to 2 of the top places in recall and search, ranked with a penalty and labelled "[SUPERSEDED date by #id]". - A dated question is read either as an event ("what happened on Sept 11": memories recorded that day get a boost) or as a state ("as of", "still", "set to": what was current at the end of that day). - It applies only to the retrieval commands her tools call. Internal callers that treat any hit as current truth (deduplication, auto-recall, prompt context) keep the old behaviour.

The test (preregistered before any run). 43 sealed questions written from her real memory, in four families: history, current-must-win, event-on-date and state-as-of. An independent model reworded each one. 3 were dropped because a name or date did not survive the rewording, the same rule as the earlier sets, which leaves 40. Each question ran through her real paths with the flag off and on, scored as her habit (her logged mix of recall, search and fused).

• One preregistered condition failed. On current-must-win questions, an outdated memory now ranks above the current one in 14.7% of cases (it was 0%; the limit was a 5-point rise). That is 2 of 9 questions. In both, the current memory is still in the top 5, but the labelled old version sits above it, because the penalty is too small when the two are worded almost the same.
• The other four conditions passed: the real-history archaeology set improves with nothing worse (+6.6 [+1.8, +12.5]); with the flag off the results are byte-identical to before; and latency is unchanged.
• Verdict: not installed. The likely fix is a rule that a superseded memory can never outrank the memory that replaced it. That needs a new preregistration and fresh questions.

The summary now shows her live configuration. Before this, "Latest" was r4route's 88.8, a layer that was never installed and that did not transfer to her real memory. The v2 held-out set was rerun on a copy of her live retrieval code (memstore 1.21.2: the hippo fix plus her own edits since): 50.8, against the 52.0 baseline. - Her habit's retrieval quality is unchanged or slightly up (Recall@5 57.4 → 57.5). - The graph path improved a lot (Recall@5 9.5 → 37.5) and the best possible route rose from 70.4 to 78.0. But her habit never calls the graph path, so the gap between what she does and the best route grew. That gap (route regret) is what lowers the composite. - In short: the hippo fix made a better path available, and she doesn't use it yet.

r4route and her own R4 v1 flags are kept on the page as reference rows, not as her score. The real-traffic shadow week still reads out on 10-15.

Update 2026-10-08 — the four next steps, done: hippo fixed live (top-5 hits 50 → 149 of 348), but on her real memory r4route does not beat her habit (+5.0 [−2.2, +12.8]); real history disappears from retrieval once corrected; a shadow week is running

The operator said go on all four. Everything except step 1 is read-only on her machine: it runs on the bench machine against copies of her memory and her seat transcripts.

1. The graph lookup is fixed, live. memstore._match_entity_ids now ignores junk names (her own is_junk_entity()). An entity name inside the query only counts as a match when it is a whole word of four or more characters. Her memstore is now 1.21.1, with a backup kept beside it. On the v2 bench, with the same cached entity splits, hippo alone went from 50 to 149 of 348 probes with the right memory in its top 5. No probe got worse. When it finds the right memory, it ranks it first in 143 of 149 cases. It now returns 229 different top-5 sets instead of 84. 72 questions now return nothing, because their only seeds were junk; none of them had been hits before. This also changes the seeds for causal v1 and counterfactual.py.

2. Archaeology on her real September history (plan §23): the bench gain does not transfer. The subsystem is her seat infrastructure: router, supervisor, succession, delegate path and ledger. There are 51 probes, with the answer key built by Claude from her own memories and sealed root-only. One independent model reworded every question, and the test was preregistered before any run.

• r4route − habit: +5.0 [−2.2, +12.8]. By the preregistered rule, that means it does not transfer. Her real store has 2,337 memories, and plain recall and search already find most things in it. Her habit scores 80 here, against 52 on the synthetic bench, so the headroom the bench measured doesn't exist on her real memory.
• By family (habit → r4route): version changes 91 → 100, state at a date 38 → 80, corrections 100 → 100, associated components 100 → 100, failures 88 → 80, original design 50 → 50. On questions with no answer in the record, r4route said "nothing found" 3 of 4 times; her paths always returned something.
• Real defect 1 — history disappears. Every retriever except as_of filters on valid_to IS NULL. Once a memory is corrected, its original version is only reachable through a dated as-of query. All four "original design" misses are such memories: no path finds them. 24 memories are affected today. The plan (§15) says a stale value should stop competing, not vanish.
• Real defect 2 — r4route reads every date as "as of". For "What took recall down on September 11?" it fetches the state as of that day instead of the event. Both of its failure-family losses are this. On the synthetic bench, a date always meant state-at-a-date. This is fixable, but it was not changed, per the preregistration.

3. Retrieval → answer tracing (plan §11), measured from her transcripts. This covers 88 seat sessions, 459 memory-tool calls and 3,148 surfaced items. Each item was labelled by whether its rare words reappear in her reply (used), in her thinking or next calls (consulted), or not at all (ignored). - Items: used or cited 11.6%, consulted but not used 18.7%, ignored 69.6%. - Calls: 35.5% led to a memory she visibly used, and 70.8% shaped her next step. - A hand check of 16 items found the labeller right on 6 of 8 "used" and 7 of 8 "ignored". search results show only about 100 characters per item, so use of them is under-counted. - This is the plan's "retrieved but ignored" number, measured for the first time. It belongs to Attention, not to the retrievers.

4. Shadow mode (plan §24). Each night at 03:40 the bench machine copies her memory and transcripts. It replays every new seat retrieval through r4route, using only memories that existed at that moment, and logs both result sets. A judge model then compares each pair blind, without knowing which set came from which system. The verdict rule was preregistered before any number. - Preview over 10-01 to 10-08 (68 real retrievals; it does not count toward the verdict): the judge preferred her results 39 times to r4route's 25. r4route's share is 39% [28, 51]. That is "no difference shown", but it leans hers. - r4route said "nothing found" on 9 of her real queries. - Her real queries are keyword-style, so r4route's "no names → paraphrase expansion" rule fired on 30 of 68. - Of the memories she actually used, r4route would also have surfaced 77%. - The preregistered week runs to 10-15.

Where that leaves #4. On synthetic memory, r4route fixed what the bench was built to test. On her real memory and real traffic, it doesn't beat what she already does. It should not be installed as it is. What the real data points at instead: - (a) keep corrected memories retrievable as history, for example by labelling them "superseded" instead of filtering them out; - (b) separate "what happened on a date" from "what was true as of a date" in routing; - (c) the 70% of retrieved memories she never uses, which is an Attention question.

The hippo fix is the one change that is live.

Update 2026-10-08 — Claude took the program over: an independent bench on her real retrievers (52.0 baseline), two broken lookups found, and a router layer that scores 88.8 on blind-worded held-out questions; nothing live yet

Why a second bench. The operator asked Claude to finish the component, since Piper can't fairly write and grade her own tests. R4-Bench v2 is built so that she couldn't have tuned on it: - the generator and the answer key are sealed root-only on the bench machine; her machine never saw the questions or the key; - it runs her real retrievers (keyword_search, recall, associate, hippo, causal, timeline, as_of, fused) on a copy of her code and a snapshot of her live memory (2,196 memories, 25k entities, 19k graph edges), so her own history is the noise; - 1,972 bench memories go in through her own remember(). Graph and causal edges are written exactly as stated, because writing memory is a different component; - there are 400 probes, 8 families × (20 dev + 30 held-out), in the plan's eight families, with the plan's hard negatives (near-miss ticket numbers, sibling components, near-name people, superseded settings, correlated-but-not-causal events); - dev and held-out use different question wording and different names. Fixes were built on dev only. The held-out test was preregistered and hashed before it ran.

Her machine's heavy-work guard (the operator's rule: benches run on the bench machine) stopped the first load twice, so the whole bench runs on the bench machine.

Baseline: what her retrieval actually does. - Her habit scores 52.0. That is the expected score of her logged live mix (recall 184 / search 158 / fused 35 calls). Picking the best existing path for every question (the oracle) would score 70.4, so path choice alone is worth about 18 points. That supports the plan's central hypothesis (§30). - hippo (graph) is broken: it returns the same memories for every question. Her graph holds 116 one- and two-letter entities ("i" alone has 989 edges). Her entity matcher accepts a match when either name contains the other, so "i", "e" and "r" seed every query and PageRank ends at the same hubs. Her own is_junk_entity() already flags these names, but the matcher never calls it. - causal drops the edge it is asked about when cause and effect mention the same subsystem ("memory leak in X" → "X OOM-killed"). It finds both nodes, but files them under one named thing and only reports links between things. - Paraphrase recall is weak: the embedding model (bge-small) misses reworded questions in a 4,000-memory store (median gold rank about 40). The model's query prefix doesn't help. - Every path returns something when no answer exists (no-answer false positives 100%). Results carry no validity state.

Her own upgrades on this bench. The three of her five R4 upgrades that can run on the real store (router + temporal validity + confidence gate) score 44.2, −7.8 [−10.5, −5.3] vs her habit. Exact lookups fall from 0.86 to 0.67, and fused questions go to 0. Her multi-hop and fused-v2 modules only run on her own bench files. Recommendation: keep all five flags off.

What Claude built: r4route 1.0.0, a layer over memstore that changes none of her code: - a deterministic router built from the plan's §12 table. Its cues are date phrases (→ as-of), "now/latest" (→ current state), "before it changed" (→ predecessor), "stop being valid" (→ end of validity), why/because (→ causal), relationship verbs (→ graph or association) and identifier shapes (→ exact). Questions with several parts go to an expansion step; - repaired graph seeds: junk names are dropped, and an entity name only counts inside a seed when it is a whole word of four or more characters; - causal edge lookup: stored edges are ranked by how well both ends cover the question; - time as a field: every stored statement about a setting is read in time order, so "now", "on Sept 15", "before it became 420" and "until when was it 600" each return the right version, including superseded ones; - multi-part expansion: anchor on the incident ID verbatim (her keyword index splits INC-113 into "inc" + "113"), hop through that memory's entities, then read the neighbour's setting at the incident's time; - a no-answer gate (plan §17): when two named things never co-occur and no edge joins them, nothing qualifies; - HyDE expansion (plan §16), only for questions with no names in them: one cached model call writes the note that would answer the question, and that note is searched too. On dev, paraphrase Recall@5 went from 0.20 to 0.50; - every result carries path labels, rank, score, timestamp and validity (provenance 1.0).

Results (R4 composite, plan §9 weights; paired bootstrap 95% CI vs habit)

Why H2 exists. Claude wrote the held-out questions and then wrote the router's cue words, and several cues ("would notice", "team lead", "these days") came from held-out templates. That is the same defect found in Piper's router. So every held-out question was reworded once by an independent model, with names, numbers and dates checked to survive verbatim, and run once under a second preregistration. As predicted, the score fell (94.2 → 88.8). Multi-hop lost most (0.82 → 0.62), so the router still leans on wording. It passed the rule anyway.

H2 by family (Recall@5; H = correct "nothing found")

Against the plan's targets (§27): composite ≥ 90 nearly met (88.8); no-answer false positives ≤ 5% met (0%); as-of ≥ 90% met (100%); provenance 100% met; route regret 0.05 met. Recall@5 ≥ 0.95 on exact/paraphrase not met (0.90 / 0.61); specialised paths ≥ 0.85 not met for multi-hop (0.62).

Limits, stated plainly. - Claude wrote both the corpus and the fixes. The time handling keys on how settings are phrased in this corpus, and the no-answer gate on how names look. Transfer to her real, messier memories is untested. That is the plan's archaeology test (§23), and it comes next. - The current-state stale-error metric is 0 for every arm, so it does not discriminate here. The bench needs harder stale cases. - Cost: the bench's model calls (entity splits, HyDE, rewording) went on the bench key and were not separately metered. They are small (a few hundred short calls).

Nothing is live. r4route is not installed, and her five flags stay off.

Next (the operator's call). 1. Fix _match_entity_ids in memstore itself (one guard), since every live hippo call is affected now. 2. Install r4route in shadow (plan §24): it logs what it would retrieve next to what she retrieved, for 7 days. 3. Run the archaeology test on her real September history, with a gold key built from her records. 4. Then retrieval-events v2 (plan §18), so it can be measured whether retrieved evidence was used.

Update 2026-10-07 — Piper's own R4 bench and five upgrades, reviewed: 62.2 → 90.2 on her bench, but the bench measures stand-in retrievers on templated questions she tuned against; the router falls back to semantic on reworded questions; nothing live

What she did (10-07, about 20:20 → 21:25, from the operator's Recall / Retrieval plan; everything in foundation/state/r4_* and memory/). She: - built R4-Bench v1: 240 probes, 30 in each of the plan's eight families (exact, semantic, association, graph multi-hop, causal, temporal, fused, no-answer), each with hard negatives of every class the plan lists; three corpora (small 2,130 / standard 2,500 / stress 10,000 records) with sha256s, a seed, and a manifest; - wrote a scoring harness (r4_harness.py 1.0.0) with the plan's nine metrics (Recall@5, nDCG@5, MRR, route regret, stale error, as-of accuracy, support completeness, negative false positives, provenance) and a self-check that proves each metric on a hand-built case; - froze a baseline with every path also run on its own, plus an oracle (best path per probe); - ran five upgrade rounds, each measured against the frozen baseline and each behind its own off-by-default flag with a byte-identical revert: a deterministic retrieval router (retrieval_router.py 1.0.0, wired into memstore.fused()), temporal validity (temporal_validity.py, memstore 1.20.0: a superseded value stops competing in current-state reads), a retriever-evidence confidence gate (returns nothing when no relation is evidenced), directed multi-hop path retrieval, and fused v2 (reciprocal-rank fusion that keeps each result's path labels); - measured ceilings: for the two families still short, she checked how much of the gold set any path reaches in its top 20 / top 100, so she could say what no fusion change can recover. Her report separates measured from inferred and names what was not measured (cross-corpus generalisation; "the query families are templated").

Her numbers (re-run by Claude: the baseline reproduces exactly from her files).

By round: router 62.2 → 80.0 (E causal 0 → 1.0, F temporal 0 → 1.0, but F's stale error went 0 → 100%); temporal validity → 80.9 (stale error back to 0); confidence gate → 85.9 (no-answer false positives 100% → 0); multi-hop → 89.2 (D graph 0.67 → 1.0); fused v2 → 90.2. Still short: B semantic (Recall@5 0.47; a third of its gold is outside every path's top 100) and G fused (0.67; the bridge record is reached by no path).

Review. The engineering discipline is good: frozen artefacts with hashes, an oracle, one change per round with a measured before/after, flags that default off, and an honest list of what she did not measure. But the bench cannot carry the claim the numbers suggest, for four reasons.

1. It does not measure her retrievers. The harness re-implements eight paths over a JSONL file: "semantic" is a character-trigram cosine (no embeddings), "graph" is a breadth-first search over the bench's own edges. Her real recall (fastembed vectors), BM25 index, hippo-kg and causal_v2 are never called. B semantic's 0.30 is a trigram score, not a measurement of her semantic recall.
2. The questions are templates. Seven of eight families are one sentence with a number changed ("What is config00 now?" … "config29"). Each family therefore behaves as one item repeated 30 times, which is why per-family scores land on exactly 0, 0.67 or 1.0.
3. The labels are not held out, and the rounds were tuned on the test. The gold IDs on the bench machine are byte-identical to the gold_memory_ids in her probe file on her own machine (240/240). All five rounds were fitted and judged on the same 240 probes; there is no second set. The router's own comments cite the family scores it was written to fix.
4. The router does not generalise past the templates. Claude reworded one question per family and ran her decide():

All seven reworded questions fell through to the semantic default, and so did the plan's own as-of examples. Only wording the templates copy (the plan's version and multi-hop examples) routed as intended. That is the "universal hammer" the plan warns about. Two rules also work around the bench instead of fixing a path: causal questions go to association because the bench's causal path scored 0, and current-value questions go to semantic, not temporal. As-of questions have no rule at all; the as-of score of 1.0 comes from the harness passing the correct timestamp, not from routing.

Not done from the plan: retrieval-events v2 and the evidence-use bridge (still 14 of 4,449 labelled), the ablation runs, Operation Palimpsest, the archaeology test on her real history, shadow mode, and a run on the live memory store. Live state: none of the five flags is set, so live retrieval is unchanged. memstore.py 1.20.0 carries the router and temporal hooks behind those flags (its changelog is dated 10-08, a day ahead). The four new modules are untracked in her memory repo.

What it means. She built the measurement scaffolding the plan asked for and showed real engineering care, but the 62.2 → 90.2 is a score of stand-in retrievers on questions the fixes were fitted to. It is not yet evidence about her recall. Temporal validity and the no-answer gate are the most likely to carry over, because they are rules about record state, not about wording.

Next. 1. Point the harness at her real retrievers (memstore recall/search/hippo/causal/as_of) over a bench namespace in a copy of memory.db. 2. Write a held-out set of freely worded questions (several phrasings per fact, real foundation_router-style names), keep its key off her machine, and score the router and the five flags on it once, preregistered. 3. Give as-of questions a real route (date phrases → as_of with the parsed time). 4. Then the plan's Phase 1: retrieval-events v2 with session and result IDs, so use can be traced.