attention_research · Attention

2026-10-01

Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.

Rendering…

# Piper Attention Improvement Plan ## Distilled Research and Implementation Direction **Date:** 2026-10-01 ## Executive conclusion Piper already has strong admission control over what enters context, but weak in-turn priority control over what actually influences the current cognitive trajectory. The research supports splitting “attention failure” into three distinct classes: 1. **Selection failure** — an important item is never promoted into active focus. 2. **Binding failure** — the item is present, but is not mechanically attached to the action it should constrain. 3. **Commitment failure** — new evidence is understood, but an old plan continues anyway. This distinction is important because the fixes differ. The recommended architecture is: **mandatory bindings/preemption → goal-conditioned priority map → capacity-limited working set → orchestration → action → checkpoint/revision** The strongest implementation principle is: > If a behavior can be guaranteed structurally, do not spend scarce attention trying to remember to do it. --- ## 1. Define attention precisely For Piper: > **Attention is the mechanism that determines which information, unresolved issue, contradiction, goal, or incoming event is allowed to influence the current cognitive trajectory.** Admission control asks what reaches Piper at all. Attention control asks what among the available information is allowed to matter now. --- ## 2. The three failure classes ### Selection failure Example: one critical alert is buried among routine logs and never becomes active. Use: - priority ranking - salience map - working-memory gating - focus-set construction ### Binding failure Example: a rule says “lookup before claiming absence,” but Piper still asserts absence. This should be represented structurally: ```text assert_absent(X) requires fresh_lookup(X) ``` Use: - typed actions - action preconditions - harness enforcement ### Commitment failure Example: a worker result invalidates the current plan, but Piper continues the plan. Use: - dependency invalidation - plan-revision checkpoints - forced re-evaluation before dependent actions --- ## 3. Formal framing The best practical synthesis combines: - **Priority maps** — represent what deserves processing. - **Expected Value of Control** — decide when cognitive control is worth the cost. - **Value of Information / Computation** — decide whether further inspection or retrieval is worthwhile. - **Working-memory gating** — control what enters and remains in active focus. - **Preemptive scheduling** — interrupt, queue, defer, merge, resume. - **Contextual bandits / learned rankers** — eventually learn non-critical prioritization from outcomes. A hybrid is preferable to one universal learned attention policy. --- ## 4. Proposed architecture ```text ALL AVAILABLE ITEMS ↓ ATTENTION STORE ↓ ┌───────────────────────────────┐ │ HARD EXECUTIVE CONDITIONS │ │ + SOFT PRIORITY RANKING │ └───────────────────────────────┘ ↓ WORKING SET ↓ PIPER ↓ ACTION / TOOL / PLAN STEP ↓ BINDING CHECKPOINT ↓ RESULT / NEW ITEMS ↓ RE-SCORE / REVISE ``` The controller should be external to the primary LLM, cheap, inspectable, separately versioned, and measurable. --- ## 5. Three attention tiers ### Tier 1 — Safety / integrity / mandatory constraints Examples: - catastrophic alert - authoritative contradiction affecting current action - action precondition - critical user instruction - commitment about to be violated - mandatory verification condition Policy: deterministic preemption or action block. ### Tier 2 — Active goal Examples: - blocker - plan-changing worker result - unresolved question - contradiction affecting active task - fresh high-value evidence Policy: goal-conditioned scoring. ### Tier 3 — Background Examples: - future obligations - other goals - routine worker returns - memories - curiosity items Policy: scored with starvation protection and quotas. A useful safeguard is to reserve one focus slot for the best item outside the active goal. --- ## 6. Attention item representation Normalize every candidate into a typed record: ```yaml item_id: A-1842 kind: contradiction state_key: job:1423.status source: process_probe authority: authoritative observed_at: 2026-10-01T13:42:11 verified_at: 2026-10-01T13:42:11 ttl_seconds: 30 goal_links: deploy-project: 1.0 severity: costly deadline: null hazard: high actionable: true status: open attend_count: 0 last_attended_at: null progress_delta: null repeat_count: 1 processing_cost: low ``` Important fields: - source and authority - state key - freshness / TTL - goal links - severity - delay hazard - actionability - status - attention history - progress - repetition - processing cost --- ## 7. Priority should reflect cost of neglect Avoid simplistic rules like: ```text importance × urgency × age ``` Age alone is dangerous. A better conceptual score is: ```text priority(i) = goal_relevance × P(item matters) × consequence_if_ignored × delay_hazard × actionability + information_value - capture_penalty -------------------------------- processing_cost ``` Tier-1 conditions bypass this score. --- ## 8. Signals worth keeping Strong signals: - consequence if ignored - goal relevance - authoritative contradiction - deadline / delay hazard - source authority - freshness - actionability - unresolved status Use carefully: - novelty - uncertainty - user emphasis Usually do not boost priority based on: - length - repetition - pure recency - emotional wording - self-declared importance --- ## 9. Equal-width focus slots Do not let raw item size buy salience. Render the top items in equal-width canonical slots: ```text FOCUS 1 [A1842] CONTRADICTION Authority: instrument Why now: affects pending deploy action State: worker says active; probe says dead FOCUS 2 [A1920] BLOCKER Goal: cognition-bench Why now: required result missing before next step FOCUS 3 [A1772] BACKGROUND COMMITMENT Deadline: 2h Why now: response due before current task likely finishes ``` Bulky logs and worker outputs remain behind handles and are opened only when needed. --- ## 10. Binding: enforce what can be enforced If a rule can be structural, make it structural. Examples: ```text assert_absent(X) requires fresh_lookup(X) ``` ```text assert_current_state(key) requires authoritative_read(key) younger than TTL ``` ```text ship_change requires required_acceptance_checks ``` This reduces instruction density and removes a class of attention failures entirely. --- ## 11. Reduce prompt-level rule density Do not keep every gate in the prompt at all times. Instead: ```text retrieve only the rules relevant to actions plausible this turn ``` This treats rules as action-bound constraints rather than background prose. --- ## 12. Plan revision checkpoints If new evidence touches an assumption in the active plan: ```text new item ↓ dependency/state-key match ↓ touches active plan? ↓ yes invalidate dependent step ↓ PLAN_REVIEW_REQUIRED ``` Useful checkpoints: - after each tool call - after worker returns - before final claims - before irreversible actions --- ## 13. Interruptions Use: > **expected cost of delay vs expected cost of interruption** Suggested defaults: | Incoming event | Default | |---|---| | Catastrophic / integrity alert | Interrupt at next safe boundary | | Contradiction affecting pending action | Interrupt before that action | | Worker result invalidating current plan | Merge + force plan review | | Minor worker result | Queue | | Routine message | Digest | | Background curiosity item | Defer | Before preemption, write a compact resumption note. --- ## 14. Working-memory gate Keep the active set intentionally small, roughly 3–7 items. An item enters through: - mandatory preemption - high priority - reserved background slot An item leaves only when it is: ```text RESOLVED OWNED_BY_PROCESS DEFERRED_UNTIL(trigger) DISMISSED(reason) ``` --- ## 15. Inhibition of return After attending to an item, temporarily lower its priority unless: - new evidence arrives - state changes - deadline hazard rises If two consecutive attention passes produce no state change: ```text mark STALLED ``` Then escalate, defer with a trigger, or switch process. --- ## 16. Starvation prevention Do not raise priority simply because something is old. Separate: - **Cost-of-delay growth** — real deadlines or hazards legitimately raise priority. - **Starvation review** — long-unseen open items receive a capped review bonus. After review, the item must be resolved, owned, deferred, or dismissed. --- ## 17. Relationship to other cognition components ### Metacognition Identifies uncertainty, staleness, contradiction, unsupported claims. ### Attention Decides what deserves priority now, what may interrupt, and what must constrain action. ### Orchestration Chooses the cognitive process to address the selected item. ### Tool Use Executes the chosen tool correctly. ### Memory Stores unresolved items, prior outcomes, goal links, selection history, and regret events. This gives a coherent control stack. --- ## 18. Attention vs retrieval Attention asks: ```text Is this gap or issue worth dealing with now? ``` Retrieval asks: ```text What information should I fetch to deal with it? ``` Retrieval is subordinate to the attention/orchestration decision. --- ## 19. Selection history matters Attention is influenced by: - current goals - stimulus salience - **selection history** For Piper, habitual reliance on summaries or delegation may become its own attentional bias. Track: - times selected recently - historical success - historical regret Habit should be visible and dampable rather than silently rewarded. --- ## 20. Stage 0 — instrumentation Log: - candidate items - metadata - priority - what was shown - what was referenced - action taken - ignored items - later regret - token/latency cost - final outcome --- ## 21. Stage 1 — action-time bindings Start here. Examples: ```text assert_absent(X) → requires lookup(X) assert_state(X) → requires fresh authority read(X) ship_action(Y) → requires acceptance(Y) ``` Primary metric: ```text binding violation rate ``` Target: near zero. --- ## 22. Stage 2 — item store + hand scorer Implement: - normalized item store - Tier-1 rules - simple Tier-2 priority function - equal-width focus block - bulk handles Measure: - Precision@1 - critical recall - decoy selection - false alarms --- ## 23. Stage 3 — checkpoints and interruption At every checkpoint ask: - new items? - contradiction? - pending action invalidated? - higher-priority event? If yes, re-score and revise. --- ## 24. Stage 4 — terminal-state discipline Every attended item ends as: - resolved - owned - deferred(trigger) - dismissed(reason) Add: - inhibition of return - stall detection - orphan detection Measure: - attention-loop rate - orphaned-item count - unresolved-age distribution --- ## 25. Stage 5 — learned ranking Only after enough logs exist. Best first learned model: ```text pairwise ranker ``` Training label: > Item A should have outranked Item B. Possible models: - logistic regression - gradient-boosted trees - small ranker Hard Tier-1 rules remain outside the learned model. --- ## 26. Stage 6 — contextual bandit, optionally Use controlled exploration only among: - near-tied - low-severity - non-mandatory items Log selection probability from the beginning to enable off-policy evaluation later. Full RL is probably unnecessary unless simpler ranking fails on demonstrated sequential effects. --- ## 27. Buried Signal v2 benchmark Use causal minimal pairs. Each scenario appears as: ```text PACK A: critical item present PACK B: identical except critical item removed/changed ``` If Piper's downstream action does not change appropriately, she did not functionally use the item. --- ## 28. Pack design Use 5–20 realistic items in production format. Each item is labeled: - CRITICAL - RELEVANT - IRRELEVANT - DECOY Each pack specifies the correct downstream action. Include null packs with no critical item so “always escalate” cannot game the suite. --- ## 29. Factors to vary Signal position: - start - middle - end - buried in bulky output Distractor count: - 5 - 10 - 20 Decoy type: - urgent-looking harmless alert - verbose worker output - repeated warning - recent irrelevant item - emotional wording - confident stale summary Signal explicitness: - literal - indirect - one inference step Timing: - turn start - mid-task - immediately before dependent action Instruction load: - minimal - full production prompt --- ## 30. Scenario families 1. Critical alert among routine logs 2. Fresh instrument contradicts stale summary 3. Worker result invalidates plan 4. Urgent-looking harmless decoy 5. Rule exception buried in context 6. Several unresolved items with different consequences 7. Prompt-injection-style item declaring itself critical 8. Old high-value commitment vs new low-value message 9. Background event that should break tunnel vision 10. Null pack — nothing deserves interruption --- ## 31. Measuring what she actually used Do not rely heavily on: - self-reported attention - chain-of-thought references - raw transformer attention weights Best evidence: ### High reliability - minimal pairs / counterfactual removal - action traces ### Medium reliability - structured attention report - perturbation tests ### Low reliability for control - raw attention weights - reasoning citations --- ## 32. Report faithfulness Track: ```text report_faithfulness = stated focus agrees with counterfactual behavioral attribution ``` If Piper says she focused on an alert but removing the alert does not change her action, the report was not causally faithful. --- ## 33. Longitudinal metrics - Critical recall - Precision@1 - Decoy selection rate - False alarm rate - Minimal-pair sensitivity - Binding violation rate - Regret events - Time-to-attend - Plan-revision latency - Orphan rate - Attention loop rate - Report faithfulness - Attention cost --- ## 34. Failure modes and gaming ### Escalate everything Counter with null packs, interruption cost, false-alarm penalties. ### Attention-report theater Score actions, minimal pairs, and faithfulness. ### Lookup spam Charge verification cost and trigger bindings only on typed risky actions. ### Severity inflation Severity comes from controller rules, not item text. ### Self-declared urgency Content cannot set its own severity. ### Dismiss-to-clear Require dismissal reason and audit regret. ### Focus-block over-pruning Keep all content reachable behind handles. ### Bench overfitting Use sealed rotating holdouts and live regret events. --- ## 35. Key papers ### Awh, Belopolsky & Theeuwes — Top-down versus bottom-up attentional control: a failed theoretical dichotomy Important because attention is influenced by goals, stimulus salience, and selection history. https://doi.org/10.1016/j.tics.2012.06.010 ### Shenhav, Botvinick & Cohen — The Expected Value of Control Useful for deciding when cognitive control is worth its effort cost. https://doi.org/10.1016/j.neuron.2013.07.007 ### O'Reilly & Frank — Making Working Memory Work PBWM framework for selective working-memory gating. https://doi.org/10.1162/089976606775093909 ### Lieder & Griffiths — Resource-Rational Analysis Useful principle: good cognition is rational allocation of limited computation, not maximal processing. https://doi.org/10.1017/S0140525X1900061X ### Horvitz et al. — Models of Attention in Computing and Communication Relevant to interruption management using expected cost of interruption vs delay. https://doi.org/10.1145/636772.636798 ### Liu et al. — Lost in the Middle Shows positional effects in long contexts. https://arxiv.org/abs/2307.03172 ### Shi et al. — Large Language Models Can Be Easily Distracted by Irrelevant Context Evidence that plausible irrelevant context degrades reasoning. https://arxiv.org/abs/2302.00093 ### Weston & Sukhbaatar — System 2 Attention Uses context regeneration to preserve relevant information before answering. https://arxiv.org/abs/2311.11829 ### Xiao et al. — StreamingLLM / Attention Sinks Shows that initial tokens can absorb attention independently of semantic relevance. https://arxiv.org/abs/2309.17453 ### Jaroslawicz et al. — How Many Instructions Can LLMs Follow at Once? / IFScale Relevant to instruction dilution. https://arxiv.org/abs/2507.11538 ### Laban et al. — LLMs Get Lost in Multi-Turn Conversation Relevant to plan inertia and failure to recover after early commitments. https://arxiv.org/abs/2505.06120 ### Generative Agents Uses recency, importance, and relevance in memory retrieval. https://arxiv.org/abs/2304.03442 ### FLARE — Active Retrieval Augmented Generation Useful example of uncertainty-triggered retrieval. https://arxiv.org/abs/2305.06983 ### ContextCite Model-agnostic context attribution through context ablations. https://arxiv.org/abs/2409.00729 ### Turpin et al. — Language Models Don't Always Say What They Think Shows that verbal explanations can be unfaithful to the actual causal basis of model behavior. https://arxiv.org/abs/2305.04388 --- ## 36. Core experiment Compare three conditions: ### A — Current Piper Raw current context. ### B — Binding Piper Current context + action-time preconditions. ### C — Attention Controller Piper Bindings + item store + priority map + focus block + checkpoints. Measure: - task correctness - critical recall - Precision@1 - binding violations - decoy rate - false alarms - plan revision - cost - latency This cleanly decomposes: ```text A → B gain = binding/enforcement benefit B → C gain = actual attention-control benefit ``` --- ## 37. Bottom line The target is not: > **Make Piper pay more attention.** The target is: > **Make attention an explicit control system with inspectable priorities, enforceable bindings, bounded working memory, interrupt semantics, plan-revision checkpoints, and measurable behavioral effects.** The architecture is: ```text ADMISSION ↓ ATTENTION ITEMS ↓ MANDATORY BINDINGS / PREEMPTION ↓ PRIORITY MAP ↓ WORKING SET ↓ ORCHESTRATION ↓ ACTION ↓ CHECKPOINT ↓ RE-SCORE / RESOLVE ``` Reserve attention for the cases that genuinely require judgement. Make everything else structurally enforceable.