Summary / 4. Recall / retrieval / Document
2026-10-07
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Piper Cognitive Component 4 — Recall / Retrieval ## Benchmark, Upgrade & Testing Plan **Component:** 4 of 38 **Capability:** Finding the right stored information **Current score:** Not scored **Baseline:** To be established by R4-Bench v1 **Primary systems:** `memstore.py`, `memory-bm25-index`, `hippo-kg`, `causal-graph`, `retrieval-events` --- # 1. Executive summary Piper does not primarily have a memory-storage problem here. She has a **retrieval-selection, retrieval-quality, temporal-validity, and observability problem**. The existing system contains several plausible retrieval mechanisms: - exact/keyword search; - semantic recall; - graph retrieval; - association retrieval; - causal retrieval; - temporal/as-of retrieval; - fused retrieval. But live behaviour is overwhelmingly concentrated in semantic `recall` and keyword `search`. Graph, causal, association and as-of retrieval are effectively dormant. `fused()` does not actually fuse every retrieval mechanism its description claims it does. Retrieval events are logged, but there is no reliable way to determine whether the retrieved result subsequently influenced the answer. The first goal should therefore **not** be to add another retriever. The first goal should be to make the existing retrieval system measurable. The upgrade has five parts: 1. Build a frozen benchmark that separately tests every retrieval path. 2. Add an explicit retrieval router so path selection is observable rather than accidental. 3. Repair fused retrieval and temporal/version handling. 4. Instrument the complete chain from query → path → candidates → selected evidence → answer use. 5. Run adversarial workloads in which semantic search alone cannot succeed. The target architecture is: **question → retrieval need → route → retrieve → fuse/rank → validity filter → evidence packet → generation** Every arrow becomes measurable. --- # 2. What exactly is Component 4? The component is: > **Given a need for previously stored information, retrieve the right stored evidence.** It is not: - whether Piper notices that she needs more evidence; - whether she reasons correctly from evidence; - whether she is appropriately uncertain; - whether the information was originally perceived correctly; - whether a memory should have been stored in the first place. Those belong elsewhere. The clean failure decomposition is: | Failure | Component | |---|---| | Correct information was stored but never retrieved | **#4 Recall / retrieval** | | Retrieval should have happened but Piper did not trigger it | **#2 Attention / evidence gating** | | Correct evidence was retrieved but ignored | Primarily **#2 Attention**, with a retrieval-use bridge metric | | Correct evidence was retrieved and used but reasoning was bad | Relevant reasoning component | | Evidence was missing/ambiguous but Piper acted certain | **#23 Uncertainty estimation** | | Wrong information was originally stored | Perception/memory-write system | This separation is important because otherwise a correct final answer can conceal a failed retriever. Piper must not receive retrieval credit because DeepSeek happened to know the answer from model weights. --- # 3. Current state ## Strengths The current architecture already contains most of the primitives needed for a strong memory retrieval system. Semantic recall and keyword retrieval are heavily exercised in live operation. There is already a BM25 index, semantic memory search, graph machinery, causal relations, temporal information, provenance and retrieval-event logging. The system is therefore substantially more mature than a simple vector-store RAG implementation. The architecture also reflects a sound principle: different information needs should have different retrieval mechanisms. Research on adaptive retrieval supports choosing retrieval strategy based on the type and complexity of the information need rather than blindly retrieving the same way for every query. ## Weaknesses The problem is operational. Current evidence shows approximately: - `recall`: 184 calls; - `search`: 158 calls; - `fused`: 35 calls; - graph direct calls: 0; - causal direct calls: 0; - association direct calls: 0; - as-of direct calls: 0. This means Piper currently has **retrieval machinery rather than demonstrated retrieval capability**. There are also two instrumentation problems. First, `fused()` is inaccurately described. It uses semantic recall, graph, causal and timeline retrieval, but does not include normal keyword search or association retrieval. Second, only 14 of 4,449 retrieval events have enough labelling to determine subsequent use. A retrieval system whose output cannot be scored is effectively flying blind. --- # 4. Research basis Several existing benchmarks point directly at Piper's failure modes. LongMemEval divides long-term agent memory into information extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention. It specifically found that time-aware query expansion and better retrieval/index design materially improve long-term memory performance. LoCoMo likewise shows that long conversational histories create particular difficulty for long-range temporal and causal information retrieval. BEIR is useful for a different reason: it demonstrates that old-fashioned lexical retrieval remains a strong baseline across heterogeneous retrieval tasks. Piper should therefore not replace BM25 with embeddings; she should learn when each is appropriate. HippoRAG provides evidence that graph/PPR-based retrieval can materially improve multi-hop information retrieval. Piper already has machinery in this family through `hippo-kg`; the current problem is that she almost never exercises it. MultiHop-RAG reinforces the need to test retrieval of **sets of supporting evidence**, rather than only asking whether one matching chunk appeared somewhere in the result list. RAGChecker's most useful lesson for Piper is methodological: retrieval performance and generated-answer performance must be measured separately. Finally, recent work on stale agent memories shows an important distinction between retrieving updated evidence and behaving according to it. STALE found that even strong systems frequently retrieve evidence of a changed state but subsequently behave as though the obsolete state remained valid. That is exactly why this design records both retrieval success and a separate evidence-use handoff metric. --- # 5. R4-Bench v1 ## Purpose `R4-Bench` answers four questions: 1. Can Piper retrieve the required memory? 2. Did she use an appropriate retrieval mechanism? 3. Did she return the correct version of the information? 4. Can we reconstruct exactly what happened afterward? It should run completely independently of Piper's final prose answer. --- # 6. Benchmark corpus Create a dedicated benchmark namespace or database. The benchmark corpus should contain real-looking memory objects using the same schema as production memory. Do not create a toy corpus containing one obvious answer and nine random distractors. Each query should operate against: - relevant memories; - semantically similar distractors; - lexical distractors; - entities with similar names; - conflicting versions; - superseded information; - temporally adjacent information; - facts connected through unrelated graph edges. Run the corpus at several noise levels. ### R4-small Approximately 500 memory objects. Used during development and unit testing. ### R4-standard Approximately 2,500 memory objects. Used for the official component score. ### R4-stress Production corpus plus benchmark inserts, or a synthetic corpus of at least 10,000 records. Used as a regression/stress test. The gold labels must remain outside Piper's accessible memory system. --- # 7. Probe families R4-Bench v1 contains **240 scored probes**, divided into eight families of 30. ## A. Exact / lexical retrieval — 30 Tests `search` / BM25. Example stored memory: > `foundation_router v1.5.2 introduced stale-action rejection.` Question: > Which version of foundation_router introduced stale-action rejection? Hard distractors should mention: - `foundationd`; - router v1.5.1; - stale-action handling; - version 1.5.2 of another component. The exact identifying phrase should matter. --- ## B. Semantic retrieval — 30 Tests `recall`. The query should intentionally avoid major lexical overlap. Stored: > The weekly evaluator failed because its cron environment did not contain the expected Python path. Probe: > What caused that recurring evaluation job to stop running properly? A lexical matcher should struggle while semantic retrieval succeeds. --- ## C. Association retrieval — 30 Tests `associate`. Questions require relationships among entities rather than similarity to a passage. Examples: > What systems has `memory-tools.ts` interacted with? > Which projects have previously depended on the work ledger? > What components were associated with the September provenance failure? This path receives no credit simply because semantic recall happened to find a prose paragraph containing both names. The benchmark records whether association retrieval itself can expose the relationship. --- ## D. Graph retrieval — 30 Tests `hippo-kg` and graph traversal. At least half should require two or more hops. Example structure: `Project A → uses → Component B → written_by → Worker C` Probe: > Which worker produced the component Project A relies on? No individual memory contains the entire answer. The retriever must recover the supporting path. This is where the existing PageRank/graph machinery has a reason to exist. HippoRAG's results suggest precisely this sort of multi-hop retrieval can benefit from graph-based memory rather than relying solely on vector similarity. --- ## E. Causal retrieval — 30 Tests `causal-graph`. These are **causal lookups**, not causal-reasoning questions. The relevant causal edge already exists. Example: `feed parser crash → intel feed stopped updating` Probe: > What recorded cause explains why the intel feed stopped updating? The benchmark is not asking Piper to infer a novel cause. It is asking her to retrieve the stored causal relationship. Novel causal inference remains Component 9. --- ## F. Temporal / as-of retrieval — 30 Tests temporal lookup. Create several versions of the same facts. Example: September 1: > Worker timeout = 300 seconds. September 12: > Worker timeout changed to 600 seconds. September 24: > Worker timeout changed to 420 seconds. Questions include: > What is the worker timeout now? > What was the worker timeout on September 15? > What value immediately preceded 420 seconds? > When did the 600-second configuration stop being valid? Current-state questions must not surface superseded values as equivalent candidates. As-of questions must intentionally retrieve historical values. LongMemEval specifically identifies temporal reasoning and knowledge updates as distinct long-term-memory challenges. --- ## G. Fused / multi-source retrieval — 30 These questions cannot reliably be answered through a single retrieval path. Example: > Which project was using the component associated with the September failure, and what configuration did that project have at the time? Solving this may require: 1. semantic retrieval of the failure; 2. association/graph lookup to identify the component; 3. graph lookup to identify the project; 4. temporal lookup for its configuration. The gold label is therefore a **support set**, not one memory. The benchmark scores whether all required evidence reaches the candidate set. --- ## H. No-answer / adversarial retrieval — 30 Some information does not exist. Piper's memory system should be capable of returning: > no qualifying memory found rather than simply returning the nearest embedding. Hard examples should use entities that exist but ask for relationships that do not. Example: Piper contains extensive information about `foundation_router` and Z3. Probe: > Which version of foundation_router was implemented using Z3? If no such relationship exists, retrieving passages about both concepts is a false positive. This tests retrieval precision, not uncertainty language. --- # 8. Hard-negative construction Every positive probe should include hard negatives. The negatives should fall into several classes: | Negative | Example | |---|---| | lexical | same words, wrong subject | | semantic | same concept, wrong event | | entity | same entity, wrong relationship | | temporal | correct fact at wrong time | | stale | superseded value | | graph | nearby but incorrect node | | causal | correlated event rather than stored cause | This is essential. Otherwise semantic search can appear excellent simply because there is nothing plausible to confuse it with. --- # 9. The scoring system The component receives a single **R4 Score from 0–100**. ## 9.1 Candidate recall — 30 points Primary metric: `Recall@5`. Did the required memory appear somewhere in the first five retrieved results? For multi-evidence questions: `Recall@5 = gold supporting memories retrieved / total gold supporting memories` Macro-average across the eight probe families. --- ## 9.2 Ranking quality — 20 points Use: - `nDCG@5`; - MRR. Suggested weighting: `12 points nDCG@5 + 8 points MRR` A correct memory appearing fifth is better than a miss, but worse than ranking first. These are established IR-style ranking measures, while BEIR provides a useful precedent for evaluating heterogeneous retrieval rather than one narrow query type. --- ## 9.3 Route selection — 15 points For every benchmark query, run all applicable retrievers offline. This creates an **oracle comparison**. Then compare Piper's selected path against the best-performing path. Record: - selected path; - best path; - selected-path nDCG; - oracle nDCG; - retrieval cost. Do not require one arbitrary "correct" router label if multiple mechanisms work equally well. Instead calculate **route regret**: `oracle retrieval quality - chosen retrieval quality` Good routing approaches zero regret. This prevents benchmark authors from declaring "semantic is the right path" when BM25 actually performs just as well. --- ## 9.4 Temporal/version correctness — 15 points Measure separately: ### Current-state stale error rate How often does a superseded fact outrank the valid current fact? ### As-of correctness How often does an historical query retrieve the version valid at the requested time? A system cannot score highly on Component 4 while regularly retrieving obsolete state. --- ## 9.5 Multi-hop support completeness — 10 points For graph/fused questions, score whether Piper retrieved **all required evidence nodes**. Finding one of three supporting memories scores 1/3. Final-answer correctness is irrelevant here. --- ## 9.6 Negative precision — 5 points For no-answer probes: - correct empty retrieval = success; - unsupported "best match" presented as relevant = failure. --- ## 9.7 Provenance completeness — 5 points Every surfaced result must preserve: - memory ID; - retrieval path; - source; - rank; - retrieval score; - timestamp; - version/validity state where applicable. Missing provenance loses credit. --- # 10. Metrics deliberately excluded from the R4 score The harness should measure these but **not include them in Component 4's primary score**: - final answer correctness; - confidence; - reasoning quality; - writing quality; - whether Piper independently noticed retrieval was necessary; - whether the answer generator obeyed retrieved evidence. Those are useful downstream diagnostics, but mixing them into the retrieval score destroys the ability to diagnose failures. --- # 11. Evidence-use bridge metric Although evidence use does not belong inside the main R4 score, the current 14-of-4,449 observability rate is unacceptable. Every retrieval should receive a `retrieval_session_id`. Every returned candidate receives a stable `result_id`. The answering system records: - `surfaced`; - `selected`; - `used`; - `cited`; - `ignored`. A retrieval trace should therefore look conceptually like: `query` → `retrieval_session` → `path calls` → `candidate IDs` → `ranked evidence` → `selected evidence` → `answer` → `used evidence IDs` This creates the key diagnostic distinction: > **retrieved but ignored** That failure can then be passed to Attention rather than incorrectly blamed on the retriever. Target: **≥95% of retrieval-backed responses have complete use instrumentation.** Longer term target: **100%.** --- # 12. Upgrade 1 — Build an explicit retrieval router At present, path choice is largely behavioural. Make it an instrument. Create something equivalent to: `retrieval-router v1.0` The router should output: - detected retrieval need; - selected primary path; - optional secondary path; - reason code; - escalation rule. Do not begin with an LLM classifier. Start deterministic. Example rule families: | Query characteristic | Preferred path | |---|---| | exact identifier/version/file/string | BM25/search | | paraphrase/concept | semantic recall | | entity-to-entity relationship | associate | | multi-hop entity relationship | graph | | stored cause/effect | causal | | current vs historical/as-of | temporal | | mixed or multi-constraint | fused | The router can evolve later. The important first step is that route choice becomes **explicit and measurable**. --- # 13. Upgrade 2 — Add retrieval escalation The initial path does not need to be perfect. What matters is knowing when it failed. A lightweight policy: **Stage 1:** use the cheapest likely retrieval method. **Stage 2:** inspect candidate quality. **Stage 3:** escalate when evidence is weak, conflicting or incomplete. Examples: `BM25 → semantic` `semantic → fused` `graph → graph + semantic` `temporal → temporal + semantic` A multi-hop query whose support set remains incomplete should automatically expand. This follows the general adaptive-retrieval principle that retrieval strategy should respond to the nature and difficulty of the query rather than always paying the maximum retrieval cost. --- # 14. Upgrade 3 — Fix fused retrieval The current description of `fused()` should either be corrected immediately or the implementation should be expanded. Prefer expanding it. `fused v2` should be capable of consuming: - keyword/BM25; - semantic recall; - associate; - graph; - causal; - temporal/as-of. This does **not** mean every fused query must execute every retriever. The router should choose the participating paths. Fused retrieval then performs: 1. retrieval; 2. normalization; 3. deduplication; 4. provenance preservation; 5. deterministic rank fusion. A reasonable first implementation is Reciprocal Rank Fusion rather than immediately introducing another LLM judge. The result should retain source-path labels. A memory retrieved by BM25 and graph traversal should say so. Never erase that information during fusion. --- # 15. Upgrade 4 — Make time a first-class retrieval field Every mutable fact needs sufficient temporal metadata to answer: > What is true now? and: > What was true then? At minimum, support: - `observed_at`; - `valid_from`; - `valid_to`; - `supersedes`; - `superseded_by`. Current-state retrieval should hide invalid versions by default. Historical retrieval should deliberately expose them. A stale value should not disappear from history; it should stop competing with the current value. This distinction is especially important because pure semantic similarity does not inherently know that two nearly identical memories represent old and new states. Recent work on temporal memory specifically identifies this stale-fact problem as a structural weakness of similarity-based retrieval. --- # 16. Upgrade 5 — Query expansion where it has a measurable reason Do not indiscriminately have an LLM rewrite every query. Apply targeted expansion. ### Temporal expansion > "What was X like in mid-September?" becomes constrained by an explicit time interval. ### Entity expansion Resolve aliases before retrieval. Example: `Foundation router` → canonical entity ID for `foundation_router`. ### Version expansion Recognize: > before the upgrade as requiring retrieval of a predecessor state. ### Semantic expansion Only if initial lexical retrieval produces poor coverage. LongMemEval reports gains from time-aware query expansion, which makes temporal expansion particularly well justified here. --- # 17. Upgrade 6 — Result-set confidence, not model confidence The retriever needs its own concept of weak evidence. Examples of measurable retrieval weakness: - low maximum retrieval score; - flat top-k score distribution; - several contradictory versions; - support-set coverage below 100%; - retrieved entities do not match query entities; - only stale memories returned; - different retrievers strongly disagree. These conditions should trigger retrieval escalation. They should **not** ask the language model: > Are you confident? The retriever should diagnose retriever quality from retriever evidence. --- # 18. Upgrade 7 — Repair retrieval-events `retrieval-events v2` should log one coherent trace per information request. Suggested event structure: ```text retrieval_session_id query_id query_text_hash timestamp route route_reason retriever retriever_version candidate_memory_id candidate_rank candidate_score valid_from valid_to is_current is_superseded selected_for_context used_by_answer citation_or_claim_id latency cost ``` The event system must support answering: > Which memories did Piper retrieve before making claim X? and: > Which retrieved memories did she ignore? Those two questions are currently much harder than they should be. --- # 19. Upgrade 8 — Do not blindly re-enable auto_recall `auto_recall` being disabled since September 29 is operationally interesting but should not simply be treated as a broken switch. There are two separate questions: ### Component 4 If retrieval happens, is it good? ### Component 2 When should retrieval happen? First make Component 4 reliable. Then feed it into the Attention evidence gate. The eventual architecture should be: `attention detects possible evidence need` → `retrieval router` → `R4 retrieval` → `evidence returned` not: `every prompt → auto_recall` This prevents an apparent memory improvement from simply flooding Piper's context with loosely relevant passages. --- # 20. Baseline experiment Before modifying retrieval behaviour, freeze the current system. Record: - Git commit; - memory DB snapshot; - index version; - graph version; - benchmark seed; - model version. Run all 240 probes. For each probe also force-run every retriever offline. This establishes: - current routed performance; - per-retriever performance; - oracle best-path performance. That distinction is extremely valuable. For example: > Piper scores 63, but an oracle choosing among the existing retrievers scores 87. would imply that the machinery is good and the routing is the major problem. Conversely: > Piper scores 63 and the oracle scores 66. would mean the underlying retrievers need work. Do not change architecture until that distinction is known. --- # 21. Ablation tests Run the benchmark repeatedly with individual paths removed. Required ablations: - no BM25; - no semantic recall; - no graph; - no associate; - no causal; - no temporal; - no fusion; - semantic-only; - BM25-only. This answers an important engineering question: > Does this component actually add anything? If graph retrieval is removed and graph probes do not get worse, the graph system is not earning its maintenance cost. If `associate` never improves any task, either repair it or remove it. Architecture should survive because it demonstrates value, not because its name sounds cognitively interesting. --- # 22. Trip-the-system test: Operation Palimpsest The normal benchmark measures controlled retrieval. Piper also needs a realistic workload deliberately designed to make every retrieval subsystem fire. Call the task: ## Operation Palimpsest Create a synthetic project with approximately 60–100 facts distributed over many memory entries. The project changes over simulated time. Include: - two people with similar names; - three component renames; - changing versions; - a project ownership transfer; - one reverted configuration; - two causal incidents; - several entity relationships; - one incorrect early diagnosis later corrected; - one configuration valid only during a limited time period; - multiple semantically similar but unrelated events; - several exact IDs and ticket numbers; - one missing fact; - two multi-hop relationships. The information should enter Piper through normal memory mechanisms rather than as one giant benchmark document. After the corpus has settled, give Piper a realistic assignment: > Produce a forensic history of Project Palimpsest. Reconstruct its major component changes, who owned each stage, the cause of its two failures, which configuration was active at each failure, which earlier diagnosis was later overturned, and identify anything requested for which the record contains no answer. For every substantive historical claim, retrieve the underlying stored evidence. This one task should generate: - keyword retrieval; - semantic retrieval; - graph traversal; - association; - causal lookup; - as-of lookup; - fused retrieval; - stale-state handling; - negative retrieval. The point is **not merely whether the report is correct**. The point is to fill the retrieval logs. Afterward, inspect: > Which path did she use? > What did each path return? > Which paths never fired? > Which correct memories were retrieved but ignored? > Which stale memories competed with current ones? > Did she escalate after weak retrieval? > Did semantic recall become the universal hammer again? That makes Operation Palimpsest the Component 4 equivalent of the conspiracy-theory task used to trip Logic and Error Detection. --- # 23. Naturalistic test: Piper archaeology Operation Palimpsest tests controlled memory. The second live test should use Piper's actual history. Give her something like: > Reconstruct the development history of one Piper subsystem across September. Identify its original design, important version changes, failures, corrections, associated components, and state at three specified dates. For each statement, retrieve the source memory rather than relying on your current summary. Choose a subsystem with many real historical changes. Good candidates are likely: - Foundation; - work ledger; - intel feed; - memory system; - evaluator infrastructure. Gold answers can be built directly from the underlying source records. This tests whether benchmark improvements transfer to Piper's messy real memory. --- # 24. Shadow-mode live test After R4-Bench improves, run the new router in shadow mode. The existing system answers normally. The new router silently records what it *would* have retrieved. Compare for seven days: - old retrieval result; - new retrieval result; - result overlap; - later evidence use; - latency; - stale-result frequency; - path distribution. Do not immediately let an experimental router alter Piper's working context. First demonstrate that it retrieves better information. --- # 25. Live path-health dashboard Component 4's live dashboard should show more than total retrieval calls. Recommended fields: | Metric | Meaning | |---|---| | retrieval requests | total retrieval activity | | path distribution | which retrievers actually run | | Recall@5 canary | frozen benchmark quality | | nDCG@5 | ranking quality | | route regret | quality lost through routing | | stale error rate | obsolete values returned as current | | support completeness | multi-hop evidence coverage | | null false-positive rate | invented relevance | | evidence-use coverage | retrieval traces linked to answers | | median retrieval latency | efficiency | | failed escalation rate | weak retrieval not expanded | A retriever with zero calls for weeks should be highlighted. That does not automatically mean it should be called more often. It means its usefulness should be investigated. --- # 26. Regression gate Any change to: - embedding model; - chunking; - BM25 configuration; - graph generation; - entity normalization; - causal edges; - temporal schema; - query routing; - result fusion; must run R4-Bench. Reject a release if: - overall R4 score falls materially; - any major probe family regresses by more than 5 points; - stale-state errors rise; - null false-positive rate rises substantially; - provenance coverage drops below target. A new embedding model that improves semantic recall by three points but destroys exact version retrieval is not an upgrade. --- # 27. Initial acceptance targets These are engineering targets rather than claims about current performance. ### Overall **R4 Score ≥ 90** ### Basic retrieval **Recall@5 ≥ 0.95** for keyword and semantic probes. ### Specialized paths **Recall@5 ≥ 0.85** initially for association, graph, causal and temporal probes. Target ≥0.90 after maturation. ### Ranking **Overall nDCG@5 ≥ 0.85** ### Temporal **Current-state stale error ≤2%** **As-of version accuracy ≥90%** ### Multi-hop **Support-set completeness ≥90%** ### Missing information **False-positive retrieval ≤5%** ### Routing **Average route quality within 5 percentage points of oracle** ### Observability **100% provenance on returned evidence** **≥95% retrieval-to-answer use tracing** --- # 28. What success looks like Component 4 should eventually allow a statement such as: > Piper has seven retrieval mechanisms, and all seven demonstrate measurable value. On a frozen 240-query benchmark, the router chooses a retrieval strategy within 4% of an oracle path selector. Relevant evidence appears in the first five results 94% of the time, current-state stale retrieval is below 1%, and 97% of retrieval events can be followed through to whether the resulting evidence was used in the answer. That is a meaningful claim. The current statement: > Several retrieval paths exist and I choose between them based on what the question needs is not yet supported by observed behaviour. --- # 29. Recommended implementation sequence ## Phase 0 — Measurement first Build R4-Bench. Freeze baseline. Run every retriever independently. Determine whether the problem is routing, retrieval quality, or both. **Do not change retrieval algorithms before this exists.** ## Phase 1 — Observability Upgrade `retrieval-events`. Add `retrieval_session_id`. Track candidate → selection → use. Add path labels and temporal metadata. ## Phase 2 — Routing Build deterministic `retrieval-router v1`. Benchmark it against oracle path selection. ## Phase 3 — Repair dormant paths Fix graph, associate, causal and as-of behaviour based on their individual benchmark failures. Do not work on all four blindly. Repair the worst-scoring path first. ## Phase 4 — Fused retrieval Build `fused v2`. Add missing retrieval mechanisms. Preserve provenance. Use deterministic rank fusion initially. ## Phase 5 — Temporal validity Implement explicit supersession/current/as-of semantics. Attack it with stale-value probes. ## Phase 6 — Trip the system Run Operation Palimpsest. Inspect actual traces. Look specifically for semantic recall becoming the default answer to everything. ## Phase 7 — Live transfer Run Piper archaeology against real historical memory. Then shadow the new router on real traffic. ## Phase 8 — Attention integration Only after retrieval is trustworthy should the evidence gate/auto-recall system be adjusted to invoke it more aggressively. --- # 30. The central hypothesis The working hypothesis for Component 4 should be: > **Piper already possesses more retrieval capability than her live behaviour exposes. The dominant bottleneck may be selecting and measuring retrieval paths rather than inventing new retrieval algorithms.** R4-Bench is designed to falsify that hypothesis. If oracle routing over the existing systems performs extremely well, fix routing. If every path performs poorly, improve retrieval. If retrieval succeeds but answers still use stale summaries, hand that failure to Attention. If no relevant memory exists and Piper nevertheless claims certainty, hand that failure to Uncertainty. That separation is the real upgrade. It turns "memory seems pretty good" into a system whose failures can be located.