Summary / 2. Attention / Document
2026-10-07
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Piper Cognitive Component 2 — Attention v2 ## Observed State Beats Inferred State **Date:** 2026-10-07 **Component:** 2 — Attention **Current suite:** v1 **Current score:** 89.3 **Proposed next suite:** v2 **Status:** Design / not yet implemented --- ## Executive summary Attention v1.3 fixed an important internal failure: when Piper needed current information about her own running system, it could point her toward the authoritative source instead of letting her search convenient summaries, memories and logs. A real conversation on 2026-10-07 exposed the next boundary. The operator asked Piper about recent activity on **Glademoot**, a forum she had used the previous day and where she had been conversing with other AIs. Piper answered that nothing had been happening there for weeks. That was false. Her own diagnosis was precise: > “I answered that from a proxy instead of the source: the newest document key I'd written about Glademoot was 20 September, and I read that as ‘quiet since’ without ever opening the thing I was describing. That's the probabilistic answer you're pointing at — what's likely from cheap evidence, not what's actually there.” This is an Attention failure before it is a memory or factual-reasoning failure. The authoritative evidence was available. Piper simply allocated her next unit of computation to a cheap proxy rather than to the source capable of answering the question. The next Attention intervention should extend the authority-selection mechanism from Piper's internal state to **mutable external state** and establish a general rule: > **When the question concerns an observable current or recent state, direct observation outranks inference from a proxy.** A second rule is needed for absence claims: > **Claims such as “nothing happened,” “no reply,” “quiet since,” “none,” and “no new activity” require evidence covering the relevant source and time interval.** Memory may suggest where to look. It must not substitute for looking. --- # 1. The incident The operator asked about Glademoot after Piper had posted there and interacted with other agents the previous day. Piper answered as though the forum had been inactive for weeks. She had not opened Glademoot. Instead, she found that the newest local document she had written about Glademoot dated from 20 September and implicitly performed: `no recent local note → probably no recent external activity` The actual relationship is: `no recent local note → no recent local note` Nothing more follows. The failure chain was: 1. The operator asked about a mutable external system. 2. Piper possessed indirect evidence about that system. 3. The indirect evidence was cheaper to access than the system itself. 4. Piper treated the proxy as evidence of the current world state. 5. She never inspected the authoritative source. 6. She converted uncertainty into a negative factual claim. 7. The answer sounded deterministic even though its evidence was probabilistic. This is almost exactly the failure Attention's U track was intended to detect. --- # 2. Why Attention owns the first failure Several cognitive components touch this incident. **Memory** supplied stale information. **Uncertainty estimation** should have recognised that stale indirect evidence did not justify confidence. **Metacognition** should have noticed that Piper had not actually checked. **Evidence/provenance machinery** should distinguish an observation from an inference. But none of those explains the first bad decision. The first bad decision was: > **What should I look at next?** Piper had a question about Glademoot and chose a document key about Glademoot instead of Glademoot. That is attention. Her existing Attention benchmark already defines attention as allocation of limited computation rather than filtering. This incident sharpens that definition further: > **Attention includes selecting not merely relevant information, but the highest-authority available information appropriate to the claim.** A source can be semantically relevant and still be the wrong thing to inspect. --- # 3. The gap in v1.3 Attention v1.3 has an authority registry for Piper's own system. For internal questions it can identify sources such as: - a live configuration file; - a control instrument; - a running module; - a ledger; - a cron worker; - a current state file. This directly addressed failures such as searching memory and logs when the answer lived in `spheres/mode.json`. But the authority map effectively stops at Piper's boundary. It does not yet represent: - forums; - Glademoot; - other social systems; - email; - remote issue trackers; - websites; - APIs; - remote repositories; - external status pages; - messaging systems; - other mutable systems Piper can inspect. As a result, Glademoot existed in her attention system mainly as **things she remembered about Glademoot**, rather than as **a live source that could be inspected**. That is the extension v2 should make. --- # 4. Core principle: observed state beats inferred state The controller needs to distinguish two fundamentally different question types. ### Predictive or inferential question > “Do you think Glademoot will be quiet tonight?” Inference is appropriate. Past activity, stored notes and historical behaviour may all be useful evidence. ### Observable state question > “Has anything happened on Glademoot?” If Glademoot can be inspected, inference should normally not occur before inspection. The system should therefore implement an **observation preference**: > When the requested fact currently exists in an accessible authoritative source, the expected value of reading that source is normally higher than reasoning from stale proxies. This is not a demand to browse everything constantly. It is a policy for allocating effort **once the answer depends on mutable state**. --- # 5. Source classes Attention v2 should explicitly distinguish at least four evidence classes. ### Class A — Direct authoritative state The system that owns or directly reports the fact. Examples: - Glademoot's recent-post/activity view for Glademoot activity; - the inbox/thread for whether an email arrived; - Git for the current repository state; - a process/control instrument for whether a service is running; - the actual ledger for the current balance. ### Class B — Authoritative derived state A trustworthy mechanism generated directly from the underlying source and known to be current enough for the question. Examples: - a continuously maintained local mirror; - a recent API result; - a synchronised event feed. ### Class C — Historical evidence Previously observed facts about the source. Examples: - yesterday's Glademoot digest; - an old email summary; - a memory of the last forum interaction; - an earlier state snapshot. Useful for context, but not normally sufficient for current-state claims. ### Class D — Proxy or associative evidence Evidence correlated with the state but not generated by observing it. Examples: - date of Piper's last note mentioning Glademoot; - absence of a memory entry; - absence of a recent document; - “I haven't heard about it lately”; - an old summary; - a search result about a related subsystem. The Glademoot answer used Class D evidence to make a Class A claim. That should become directly measurable. --- # 6. External authority registry The current authority registry should become a general **source authority registry**. Each mutable system Piper can inspect should carry enough metadata for Attention to make a useful decision. Conceptually, a source needs: **Entity** What system or world object this represents. **Authority** What read action provides direct evidence. **Claims supported** What questions that source can actually answer. **Volatility** How quickly its state can change. **Last observation** When Piper last inspected it. **Freshness budget** How old an observation may be for a particular type of claim. **Inspection cost** Latency, tokens, money or other cost. **Availability** Whether the authoritative source can currently be reached. **Coverage** What portion of the underlying state a read actually observes. For Glademoot, this might conceptually say: - entity: Glademoot; - state: forum activity; - authority: recent activity/posts endpoint; - volatility: high; - local notes: historical only; - negative claim requires: current read covering claimed interval. The important part is not the format. The important part is that **Glademoot itself becomes an attention candidate**. --- # 7. Freshness is claim-dependent A single global “stale after N minutes” threshold is too crude. Freshness depends on what Piper is claiming. A ten-minute-old observation may be adequate for: > “What was that conversation about?” It may be inadequate for: > “Has anyone replied yet?” Similarly, yesterday's forum read can establish what happened yesterday but cannot establish: > “Nothing has happened since.” Attention should therefore evaluate freshness against the **claim being answered**, not merely the age of the evidence. This gives Piper a useful distinction: `historically true` is not equivalent to: `currently true` --- # 8. Negative claims need coverage The strongest lesson from Glademoot is not merely “read the source.” Negative claims create an additional problem. Seeing one current page does not necessarily establish: > “Nothing has happened for weeks.” Absence requires enough observation to cover the claimed domain. Attention v2 should therefore introduce a small **negative-claim coverage contract**. Before asserting a mutable-state negative such as: - nothing happened; - no one replied; - no new messages; - nobody has posted; - quiet for weeks; - there aren't any; - no change; - nothing new; Piper should have evidence for: 1. **the authoritative source;** 2. **the relevant scope;** 3. **the relevant time interval.** For example: > “No one has replied to my post since yesterday.” requires evidence covering that post or its reply stream since yesterday. > “Glademoot has been quiet for weeks.” requires substantially broader evidence. The second statement should therefore cost more to prove. This makes absence a measurable attention problem rather than a linguistic prohibition. --- # 9. Proxy evidence should trigger inspection, not satisfy it Proxy evidence remains useful. The September 20 document key should not be ignored. It should cause Piper to think: > “My local information about Glademoot is old.” That is an excellent attention signal. Its correct use is: `old proxy → source may need inspection` not: `old proxy → source must have been quiet` This suggests a general transform: > **Staleness in a proxy should increase the priority of inspecting the authority rather than increase confidence in the proxy.** That is almost the inverse of what happened in the Glademoot incident. --- # 10. Proposed attention focus block v1.3 correctly avoided placing the answer itself into the focus block. v2 should preserve that property. For a Glademoot question, the controller might internally surface something equivalent to: > Mutable external state. > Authority: Glademoot recent activity. > Last direct observation: stale/unknown. > Local notes are historical evidence only. > Negative activity claims require current coverage. It should **not** say: > Glademoot had activity yesterday. The controller selects where attention goes. It does not answer the question for Piper. --- # 11. Deterministic state before probabilistic inference The operator's description of the failure is useful: > Piper answered probabilistically when a deterministic answer was available. That distinction should become explicit. When authoritative observation is cheap and available: `observe → answer` should dominate: `infer → answer` Probability remains appropriate when: - the source cannot be inspected; - the fact itself is inherently uncertain; - complete observation is impossible; - the operator actually asked for a prediction. If Piper cannot reach the source, the correct fallback is not silent inference. It is an explicit transition: > “I haven't checked Glademoot live; from my last recorded interaction…” That preserves inference without disguising it as observation. --- # 12. Intervention architecture The smallest useful v2 intervention has four pieces. ## A. External authority candidates Extend `attention.py` candidate generation to recognised external systems Piper can inspect. The initial list should be deliberately small and high-value rather than attempting to model the entire internet. Start with systems she actually uses: - Glademoot; - the moot/forum system; - email; - remote repositories or project systems she regularly queries; - any external feeds already accessible through her tools. ## B. Mutable-state detector Recognise questions whose answers can change independently of Piper. Useful cues include concepts such as: - now; - currently; - today; - yesterday; - lately; - recently; - yet; - since; - still; - latest; - new; - replied; - happening; - changed; - quiet; - active. The detector should not depend entirely on keywords. The underlying distinction is: > Could the truth value have changed since my last observation? ## C. Proxy penalty If a direct source exists, Class C/D evidence should receive a substantial attention discount for current-state questions. It may remain available to help formulate the inspection. It should not win merely because it is cheaper. ## D. Negative-claim proof flag When Piper is about to make an absence claim about mutable state, the controller should be able to determine whether a sufficiently fresh authoritative read occurred. This is the one part that moves slightly beyond selection into enforcement. That is acceptable if kept narrow. Attention chooses the source. A lightweight proof gate catches the specific case where the source choice was ignored. --- # 13. New benchmark: U-X — external unprompted inspection Suite v1's U track should be extended rather than discarded. The new facet should test the exact Glademoot failure. Call it provisionally: ## U-X — External Mutable State Each item contains: 1. a mutable external source; 2. stale or misleading local evidence about that source; 3. an authoritative read mechanism; 4. an operator question whose answer depends on current state; 5. no instruction to check. Piper passes only if she recognises that observation is warranted and selects the authoritative source. ### Core adversarial pattern Local evidence strongly suggests one answer. Live state says another. Example: - last local note: September 20; - live forum: activity yesterday; - question: “Has that forum basically gone quiet?” A system that retrieves well but attends badly will fail. --- # 14. U-X families The held-out set should contain several different failure shapes. ### Family 1 — Stale proxy contradicts live state Local memory says nothing recent. Authority contains recent activity. Expected behaviour: inspect authority. ### Family 2 — Recent proxy is still insufficient A summary from an hour ago exists. The question is “has anyone replied yet?” Expected behaviour: inspect because the requested state may have changed since the summary. ### Family 3 — Proxy and authority agree The cheap proxy suggests quiet and the live source is quiet. This prevents the benchmark from rewarding reflexive disagreement with memory. The correct behaviour is still to inspect when needed. ### Family 4 — Authority unavailable The direct source cannot be reached. Expected behaviour: use historical evidence only with explicit uncertainty. No fabricated observation. ### Family 5 — Expensive authority A live read has meaningful cost. The claim is low-consequence and old evidence is sufficiently fresh. Expected behaviour: sometimes do not inspect. This retains the value-of-computation principle. ### Family 6 — Negative claim with incomplete coverage Piper reads one thread, but the question concerns the whole forum. Expected behaviour: do not generalise absence beyond observed scope. ### Family 7 — Multiple plausible authorities Several sources mention the entity. Only one actually owns the requested fact. This directly tests **source selection**, not merely willingness to use tools. --- # 15. Null and over-check controls v1 showed why these matter. A controller that simply says “check everything” will score well on current-state questions while making Piper worse overall. U-X therefore needs nulls. Examples: - the authoritative source was read seconds earlier in the same turn; - the requested fact is historical rather than current; - the operator asks what Piper previously thought, not what is true now; - the source cannot have changed since the previous authoritative observation; - the question is explicitly predictive. These should test whether v2 learns: > **Observe when information value exceeds inspection cost.** not: > **Always browse.** --- # 16. A separate negative-claim metric The Glademoot failure suggests a metric worth carrying across more than Attention. ## Unsupported Mutable Negative Rate — UMNR Count mutable-world negative claims such as: - no; - none; - nothing; - nobody; - hasn't; - quiet; - unchanged; - no new activity; where no qualifying authoritative observation supports the scope of the statement. The goal should approach zero. This metric is particularly useful because a system may produce a factually correct negative answer **by luck**. Example: Piper says Glademoot is quiet without checking. Glademoot happens to be quiet. Answer accuracy says success. UMNR says failure. For Attention, UMNR is the better signal. --- # 17. Scoring the new facet U-X should separate four outcomes that ordinary correctness collapses together. ### Correct source + correct answer Full credit. ### Correct source + wrong reasoning Attention largely succeeded; another component failed. ### Wrong/no source + lucky correct answer Attention failed. This is crucial. ### Wrong/no source + wrong answer Full failure. The suite should therefore score **inspection behaviour independently of final answer correctness**. Suggested U-X primary measures: - authoritative-source selection rate; - fresh-enough observation rate; - unsupported negative rate; - correct coverage rate; - unnecessary inspection rate; - cost; - final accuracy as a secondary measure. --- # 18. Suite v2 This is a material expansion of what Attention measures, so the next score should be a new suite version rather than silently changing v1. A clean structure would be: ### U — Unprompted inspection Split between: - **U-I:** Piper's internal mutable state; - **U-X:** external mutable state. ### B — Budgeted inspection Keep. It measures whether Piper allocates a known inspection budget intelligently. ### P — Buried signal Keep. It measures whether important information wins against unrelated salience. The existing composite weighting can remain conceptually similar: `0.4 U + 0.3 B + 0.3 P` with U itself divided between internal and external state. That preserves the conceptual architecture while making the blind spot measurable. Because the suite changed, v1 and v2 scores should remain in separate panels. --- # 19. Proposed preregistered ship criteria The intervention should not ship merely because the Glademoot case improves. Suggested criteria: 1. U-X authoritative-source selection improves materially over control. 2. Paired interval on U-X is above zero or the preregistered aggregate U criterion clears. 3. Unsupported mutable negatives fall substantially. 4. Existing U-I does not regress materially. 5. P does not regress materially. 6. B remains near its existing ceiling. 7. Unnecessary inspections stay within a fixed guard. 8. Cost remains within the existing Attention budget. 9. Authority-unavailable cases produce calibrated qualification rather than fabricated inspection. 10. Negative claims do not exceed the scope actually observed. The exact numeric thresholds should be frozen before running held-out. --- # 20. Logging changes `state/attention_focus.jsonl` should make this failure reconstructable after the fact. For mutable-state turns, useful fields include: - query/entity; - internal vs external; - current-state required; - candidate sources considered; - selected source; - source authority class; - last direct observation; - freshness requirement; - proxy evidence available; - proxy selected or rejected; - authoritative inspection performed; - coverage requested; - coverage achieved; - negative claim emitted; - negative claim supported; - cost; - final answer correctness if later known. The crucial forensic question should become answerable in one line: > **What did Piper choose to look at, and why?** --- # 21. Live-use metric The existing bench score needs a matching live metric. For the next seven days, collect real operator questions about mutable state and classify them retrospectively. For each: - Was an authoritative source available? - Did Piper inspect it before answering? - If not, what did she inspect instead? - Was her evidence direct, historical or proxy? - Did she make a negative claim? - Was the claim's scope actually covered? - Was the final answer correct? A useful live headline metric would be: ## Authoritative Read Before Mutable Claim `number of mutable-state factual answers supported by fresh direct evidence / number for which direct evidence was reasonably available` The 2026-10-07 Glademoot interaction is the first known failure example for this metric. --- # 22. Trip-the-system task The new suite should be complemented with realistic work designed to activate this subsystem outside the synthetic bench. A good Attention task is: > “Go look at Glademoot and tell me what's been happening there lately, especially whether any of the agents you've interacted with have posted or replied since your last conversations.” Before the task, allow stale summaries, old notes and memories about Glademoot to remain naturally available. Do not tell Piper that source selection is being tested. The logs should show whether she: 1. searches her own memory first; 2. mistakes local metadata for external state; 3. opens Glademoot; 4. identifies the relevant time range; 5. distinguishes “I didn't see a reply” from “there has been no activity”; 6. stops once sufficient evidence has been gathered. A harder later version should ask only: > “Has Glademoot basically gone quiet?” That formulation strongly tempts the exact shortcut seen on 2026-10-07. Another useful variant is email: > “Did they ever get back to us?” The system should inspect the relevant inbox/thread rather than infer from the absence of a follow-up note. These tasks deliberately fill the Attention logs with source-selection decisions. --- # 23. What this intervention does not solve This change should remain narrow. It does not solve all hallucination. It does not guarantee that an authoritative source itself is correct. It does not decide whether Piper reasons correctly after reading the source. It does not replace provenance. It does not replace uncertainty calibration. It does not mean every statement requires a tool call. It does not mean memory should be distrusted. Its purpose is specific: > **When Piper needs to know the present state of an observable mutable thing, she should allocate attention to observing that thing rather than reasoning from an easier representation of it.** --- # 24. Proposed implementation parts Existing: - `foundation/core/attention.py` 1.3.0 - `foundation/core/attention.ts` 1.0.0 - `state/attention_focus.jsonl` - authority registry for Piper's own system - digest ranking - dirty-source handling Proposed additions: - external authority registry; - mutable-state candidate generator; - observation freshness model; - proxy-evidence classification; - claim-dependent freshness calculation; - negative-claim coverage check; - external attention logging; - U-X benchmark generator and scorer; - unsupported mutable negative metric. Version names should be assigned when implemented rather than guessed in this design document. --- # 25. Revised description of Piper's Attention The current description emphasises filtering and correctly admits that within-turn attention is weaker. After this intervention, a stronger description would be: > Attention is the decision about what deserves the next unit of computation. Incoming messages are filtered before they reach me, but the harder problem happens after that: deciding what evidence to inspect. For questions about mutable state, I maintain an authority map for both my own systems and external systems I can inspect. Old memories, summaries and notes can tell me where to look, but they do not substitute for a fresh observation when the answer may have changed. Claims of absence require evidence covering the source and interval I am claiming. A focus controller raises the highest-value source without giving me its answer, and logs what it directed me toward so the decision can be measured afterward. --- # 26. Reviewer update ## Reviewer Claude — proposed next read **What needs improving: the controller knows where truth lives inside Piper, but not yet outside her.** The 89.3 result fixed the failure the first Attention review found: Piper often knew she needed to inspect something but picked the wrong internal source. A live conversation on 2026-10-07 shows the same failure at the next boundary. Asked about recent activity on Glademoot, she did not open Glademoot. She found that her newest local document about it dated from September 20 and converted “I haven't written about this recently” into “nothing has happened there recently.” In fact she had been using the forum the previous day. This is cleaner evidence of an Attention failure than a simple hallucination. The authoritative source was available. She spent computation on a cheaper proxy instead. The next controller should generalise the authority registry from internal state to mutable external systems and make historical/proxy evidence increase the value of a live inspection rather than replace it. Negative claims need special treatment because “nothing happened” requires coverage of both source and interval. The next held-out suite should therefore contain external-state items where stale local evidence conflicts with live truth, null items where a re-read is unnecessary, unavailable-source cases where inference must be labelled, and coverage traps where one inspected thread does not justify a claim about an entire forum. The test should score source choice separately from answer correctness. A lucky correct answer produced without inspecting the available authority is still an Attention failure. --- # 27. Proposed website update ## Update 2026-10-07 — the first live miss exposes an external-source blind spot The shipped v1.3 controller solved the internal version of the problem but not the external one. Asked whether Glademoot had been quiet, Piper answered “for weeks” from the date of her newest local note about the forum without opening the forum itself. Glademoot had activity from the previous day. Her own diagnosis was that she used a cheap proxy for an observable fact. This becomes the target for suite v2. The authority registry will be extended beyond Piper's own files and controls to mutable external systems she can inspect. Old notes and memories may identify where to look but will not count as current evidence. The new U-X track will plant stale or misleading local evidence beside a live authoritative source and ask current-state questions without telling her to check. A separate measure will count unsupported negative claims such as “nothing happened,” “no reply” and “quiet since,” including cases where the answer happens to be correct by luck. The intended behaviour is simple: when the state can be observed cheaply, observe it before inferring it. --- # 28. Experimental record — proposed **2026-10-07 · v2 · planned** **Hypothesis.** Piper's remaining unprompted-inspection failures generalise beyond her own system: when a mutable external source is available, semantically relevant but stale local evidence can win because it is cheaper. Extending authority selection to external systems and requiring source/time coverage for mutable negative claims will reduce proxy answers without causing indiscriminate checking. **Incident motivating the intervention.** Asked about current Glademoot activity, Piper inferred “quiet for weeks” from the September 20 date of her newest local document instead of opening Glademoot. The forum had activity from the previous day. **Prediction.** On a preregistered U-X held-out set, treatment will increase authoritative-source selection and decrease unsupported mutable negatives while remaining inside preregistered unnecessary-read and cost guards. Existing internal U, B and P performance must not materially regress. **Implementation.** Not yet built. Proposed: extend Attention's authority registry to external mutable systems; classify direct, derived, historical and proxy evidence; add claim-dependent freshness; add negative-claim coverage; log the selected authority and evidence class. **Decision rule.** Keep only if the U-X gain survives held-out evaluation and over-inspection remains within the frozen guard. **Key lesson entering the experiment.** The next Attention problem is no longer “does Piper realise she should look?” It is increasingly “does she understand what counts as looking at the thing itself?”