Summary / 2. Attention / Document
2026-10-01
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Agent-Level Attention: A Measurable, Goal-Conditioned Salience Policy Oct 1, 2026 · @Bink ## 1. Diagnosis The core problem is that Piper has strong *admission control* (what enters the context) but no *in-turn priority control* (what gets used once it is there). Inside the context window, the model's de facto salience is set by position, length, repetition and recency, not by consequence. Every gate you have built decides admission; none of them decides use. "Ignored a fact that was in context" is really three different failures, and they need different fixes: | Failure | What happened | Example | Fix class | | --- | --- | --- | --- | | Selection failure | The item was never registered as relevant | Critical alert buried in 15 routine log lines | Controller ranks and surfaces it | | Binding failure | The item was seen, but not connected to the action being taken | Absence-gate rule present, agent still claims absence | Precondition enforced at action time, not as prose | | Commitment failure | The item was connected, but the old plan won | Worker result invalidates plan; agent continues | Explicit plan-revision checkpoint | Your absence-gate example is mostly a **binding** failure, not a selection one. A rule written as text is just one more token competing with everything else. The cheapest robust fix is to stop asking the model to remember the rule and instead have the harness bind it to the action: a claim of absence is a typed action that is rejected unless a lookup tool call for that entity exists in the same turn. This matches Piper's own metacognition finding, that she fails at evidence acquisition rather than introspection. Attention and metacognition are failing at the same joint: the moment where the model should say "before I conclude, I must consult X" and does not. So the working diagnosis: treat attention as an **explicit, external, inspectable control signal** that (1) ranks open items against goals, (2) binds high-priority items to the actions they constrain, and (3) forces a re-check when new evidence touches the active plan. Do not expect the LLM's internal attention to do this on its own. ## 2. Formal framework The best framing is a **priority map scored by expected value of control (EVC), maintained by working-memory gating, and executed by a preemptive scheduler**. No single framing on your list covers all four jobs, so use each where it is strong. | Framing | Use it for | Weakness if used alone | | --- | --- | --- | | Priority map (Awh et al. 2012) | The representation: one map combining goal relevance, physical salience and *selection history* | Descriptive; says nothing about cost | | Expected value of control (Shenhav et al. 2013) | The decision rule: engage control on an item when expected payoff exceeds the cost of the effort | Payoff estimates are noisy; needs calibrated priors | | Value of information / value of computation | Whether to inspect or look something up at all | Intractable exactly; use cheap proxies | | Working-memory gating (PBWM, O'Reilly & Frank 2006) | What stays "in focus" across steps, and what is allowed to overwrite it | Learned gates need lots of outcome data | | Preemptive priority scheduling | Interrupt, queue, defer, merge semantics; starvation prevention | Assumes priorities are given | | Contextual bandit | Learning to rank non-critical items from logged outcomes | Delayed credit; only after logging exists | | Active inference | A unifying story (precision-weighting = attention) | Hard to implement and test; little engineering payoff today | The priority-map literature matters most here. Awh, Belopolsky and Theeuwes argue the old top-down versus bottom-up split fails because a third source, **selection history**, biases attention independently of goals. That is exactly your attentional-capture problem: an agent that habitually reads worker summaries develops a learned bias toward summaries. Your controller should treat history as a separate, tunable term so it can be audited and damped. **Attention vs orchestration (Q4).** Your boundary is useful, but it is missing a middle step. Use three verbs: 1. **Attend**: which item deserves cognitive priority now? Output is a ranked focus set. 2. **Bind**: which pending action does that item constrain? Output is a precondition attached to an action ("absence claim requires lookup of X"). 3. **Orchestrate**: which process resolves it? Output is a plan step (read the authoritative instrument, spawn a worker, ask Bink). Most of the failures you listed live in step 2, not step 1. Keeping binding explicit is what lets the harness enforce it. **Attention vs retrieval (Q5).** Attention operates over a candidate set that includes *open questions and gaps*, not only present content. Retrieval is one of the actions attention can select, alongside "inspect", "verify", "ask" and "ignore". Put concretely: retrieval answers *what to fetch*; attention answers *whether a gap is worth closing now*. The absence-gate case is attention noticing a gap ("I'm about to assert non-existence") and binding a retrieval to it. Adaptive-retrieval work such as FLARE and Self-RAG already does a narrow version of this, triggering retrieval when the model's confidence drops. ## 3. Proposed architecture A small deterministic controller sits between Piper's existing input filters and the LLM. It ranks open items, renders a short focus block, and binds high-priority items to the actions they constrain. No extra large model is needed for stages 1 to 4 of the plan. [embedded content: attention control layer · inputs to actions, with two feedback loops\] A rejected action returns to the LLM with its unmet precondition (left loop). Tool and worker results re-enter the item store and are re-scored at the next checkpoint (right loop). The Foundation daemon is the natural owner of the item store, since it already owns canonical state and survives session rotation. ### Hierarchy, not one queue (Q2) Use three levels. One flat queue lets a flood of routine items starve a rare critical one, and makes every comparison a judgment call. | Level | Holds | Policy | | --- | --- | --- | | Safety and integrity | Catastrophic alerts, authoritative contradictions on active state, commitments about to be broken | Deterministic preemption | | Active goal | Blockers, plan-changing results, open questions on pending actions, bound rules | Scored, goal-conditioned | | Background | Other goals, future obligations, memories, routine returns, novel signals | Scored, with a quota and starvation aging | Reserve one focus slot for the best background item, so the active goal cannot fully blind the agent to background change. ### Goals (Q7) Attention should run over all active goals plus future obligations, each with a weight, not just the current goal. Store goal × item as a sparse link table written when the item is created, so it is computed once and inspectable. Make goals hierarchical: an item linked to a subgoal inherits its parent's weight. Pending commitments are themselves items with deadlines, which is how future obligations get attention before they are due. ### Interruptions (Q8) The rule is from attention-sensitive alerting: interrupt when the expected cost of delay exceeds the expected cost of interruption. For an LLM agent, the main interruption cost is losing in-progress plan state, so every preemption first writes a one-line resumption note. | Event during Task A | Default handling | | --- | --- | | Catastrophic alert | Interrupt at the next tool return | | Contradiction affecting a pending action in A | Interrupt at the next checkpoint, before that action runs | | Worker result that changes A's plan | Merge at the next checkpoint and force a plan re-check | | Minor worker result | Queue until A reaches a natural boundary | | Routine message | Digest | ### Deterministic versus learned (Q14) Yes, hard constraints for the catastrophic tail and learning only among the rest is better than one universal policy. Catastrophic events are rare, so a learned policy has almost no training signal for exactly the cases that matter most. Rules are auditable, and a learned ranker that cannot touch tier 1 cannot learn to suppress it. ### How attention connects to the rest of Piper | Subsystem | Relationship to attention | | --- | --- | | Metacognition (belief ledger, self-state probes) | Low-confidence beliefs that a pending action depends on become open-question items; attention decides which get checked first | | Orchestration | Takes the top item and picks the process; must hand back a terminal state for the item | | Memory (the hive) | Source of candidate items; consolidation workers set freshness and verification times; regret events are written back as selection-history features | | Retrieval | One action attention can choose, triggered when a gap is bound to a pending claim | | Tools | Authoritative instruments are registered per state key with an authority level; the binder enforces read-before-claim | ## 4. Salience and priority representation Score items by **expected cost of neglect per unit of processing cost**, not by a product of importance, urgency and age. This is the scheduling cμ rule (serve the job with the highest delay-cost rate per unit of work), which is provably optimal for linear delay costs and maps cleanly onto agent attention. ### The attention item Every candidate, whatever its origin, is normalized into one record: | Field | Meaning | | --- | --- | | kind | alert, rule, tool\_result, worker\_return, memory, contradiction, goal, blocker, commitment, open\_question, risk | | source, authority | Who produced it; authoritative instrument > live tool > worker > summary > memory | | observed\_at, verified\_at, ttl | Freshness; an item past ttl cannot outrank a fresh item on the same key | | state\_key | The fact it is about (e.g. `job:1423.status`), so contradictions can be detected mechanically | | goal\_links | Which active goals and pending actions it touches, with a relevance weight per link | | severity\_class | Consequence if ignored: catastrophic / costly / minor / none | | deadline, hazard | When neglect starts to cost, and how fast | | actionable | Can anything be done about it now? | | status | open, attending, owned, deferred\_until(trigger), resolved, dismissed(reason) | | attend\_count, last\_attended\_at, progress\_delta | For inhibition of return and loop detection | | tokens, repeat\_count | Size and recurrence; used only to *penalize* capture, never to boost | ### The score Tier 1 is lexicographic hard gates (section 3). Tier 2 ranks the rest: ```latex \text{priority}(i) = \frac{\max_{g \in G} \left[ w_g \cdot r(i,g) \right] \cdot P(\text{matters} \mid i) \cdot S(i) \cdot h(i,t) \cdot A(i) \;+\; \lambda \cdot \text{VOI}(i)}{c(i)} ``` Here w\_g is goal weight, r is relevance of item to goal, S is severity, h is the deadline hazard, A is actionability (0 or 1 for most items), VOI is information value for open questions, and c is the expected cost of processing. Use max over goals rather than sum, so an item touching many trivial goals cannot outrank one touching a critical goal. Represent goal × item as a sparse link table populated when the item is created, not recomputed by the LLM each turn. ### Which signals to trust (Q3) | Signal | Verdict | Why | | --- | --- | --- | | Severity if ignored | Keep, core | Directly the thing you care about; set by rules per kind, not by the LLM | | Relation to active goal or pending action | Keep, core | Mechanically checkable via state\_key and goal\_links | | Contradiction with authoritative state | Keep, core (often a hard gate) | Cheap to detect when facts are keyed | | Deadline / hazard | Keep | Well-defined; better than raw urgency labels | | Source authority and freshness | Keep | Fixes "trusts summary over instrument" directly | | Actionability | Keep | Unactionable items should be logged, not attended | | Unresolved status | Keep, as a gate on eligibility, not as a booster | | | Novelty | Conditional | Only as expected information gain on a goal-relevant variable | | Uncertainty | Conditional | Useful for VOI; noisy when self-reported by the LLM | | User emphasis | Conditional | Strong for Bink, weak for third parties; gameable by injected text | | Recurrence | Mostly noise | Repetition is the main capture vector; use it for dedup, not boosting | | LLM-assigned "urgency" or "importance" | Avoid as an input | Easily swayed by wording; the exact thing you are trying to control | ### Aging (Q9) Age itself should not raise priority. Two things are worth tracking instead. **Cost-of-delay growth**: if neglect really gets more expensive (a blocked task delaying a commitment), the hazard term h captures it. **Starvation protection**: a separate, capped aging bonus guarantees every open item is reviewed at least once per N turns, after which it must be resolved, owned, deferred with a trigger, or dismissed. Meanwhile staleness *lowers* P(matters) for passive facts. That split prevents stale junk from dominating while still preventing blocked tasks from being forgotten. ### Novelty (Q10) Define useful novelty as surprise on a variable the current plan depends on. Score it as surprise × relevance, with habituation on repeat occurrences of the same pattern. "Unexpected noise" is surprise on a variable no goal or pending action reads. Curiosity-driven RL work shows the same trap (the "noisy TV" problem): raw novelty bonuses lock agents onto unpredictable but useless stimuli. ### Stopping attention (Q16) Attention on an item ends when it reaches a terminal state, not when the model stops talking about it: resolved, owned by a process with a callback, deferred until a named trigger, or dismissed with a reason. The principled rule is value of computation: continue only while the expected improvement in the decision from one more inspection step exceeds its cost. Operationally, if two consecutive inspection steps produce no change in the item's state or the plan, mark it stalled, apply a refractory period, and escalate or defer it. ## 5. Suppressing capture and preventing collapse The most reliable defense against loud-but-unimportant signals is structural: **never let raw items compete for the model's attention on their own terms**. Learning to suppress capture inside the LLM is slow and unreliable; changing what the LLM sees is fast and testable. ### What the literature says about capture - **Position.** Models use information at the start and end of a long context better than in the middle ([Liu et al., Lost in the Middle](https://arxiv.org/abs/2307.03172)). - **Instruction density.** At 500 simultaneous instructions even the best frontier models reach only 68% compliance, with a bias toward earlier instructions ([IFScale](https://arxiv.org/abs/2507.11538)). Piper's system prompt plus gates plus state blocks is an instruction-dense prompt; each new rule dilutes the others. - **Irrelevant context.** Adding irrelevant but plausible text measurably lowers reasoning accuracy ([Shi et al.](https://arxiv.org/abs/2302.00093)). Rewriting the context to keep only relevant parts before answering recovers much of the loss ([System 2 Attention](https://arxiv.org/abs/2311.11829)). - **Attention sinks.** Transformers park large attention weight on initial tokens regardless of content ([StreamingLLM](https://arxiv.org/abs/2309.17453)). Being first is not the same as being attended *for meaning*; do not rely on "put it at the top" alone. - **Plan inertia.** Models that commit early in a multi-turn exchange over-rely on their earlier answers and fail to recover ([Lost in Conversation](https://arxiv.org/abs/2505.06120)). This is your "continues the old plan" failure. ### Mechanisms 1. **Equal-width slots.** The controller renders a *focus block*: the top k items (k ≈ 3–7) as one-line canonical statements with an item id, kind, authority and why-now reason. Bulky content (worker output, logs) sits behind a handle the model must explicitly open. Verbosity can no longer buy salience because every item gets the same few tokens. 2. **Dedup by state key.** Ten repeated warnings become one item with `repeat_count=10`. Repetition informs the controller, not the LLM. 3. **Authority collapse.** When a summary and an instrument disagree on the same state\_key, the summary is not shown as a peer; the contradiction is shown as an item. 4. **Bookending.** Put the focus block near the end of the prompt, immediately before the generation point, and a one-line pointer to it in the system header. Both ends are strong; the middle is the trough. 5. **Rule retrieval, not rule recitation.** Instead of keeping every gate rule in the system prompt, inject only the rules bound to actions plausible this turn. This directly reduces instruction density. ### Collapse modes and their counters (Q15) | Collapse mode | Counter | | --- | --- | | Always picks alerts | Alerts carry severity classes; only catastrophic preempts; others compete normally | | Always picks newest | Ranking uses no position or timestamp term except via hazard and freshness on the same key | | Always picks longest | Equal-width slots; length enters only as processing cost (a penalty) | | Always picks the user message | User messages are high goal weight, not an automatic override of active safety items | | Tunnel vision on current goal | Reserved slot: one focus position for the best item *not* linked to the active goal | | Starvation of rare high-value signals | Capped aging bonus plus mandatory review every N turns | | Repeated attention to same unresolved item | Inhibition of return: after attending, item is suppressed for a refractory window unless its state changes | | Attention loops with no action | Progress check: two attends with no state change → stalled → escalate to Bink or defer with trigger | | Attention budget overspend | Per-turn budget for inspection steps; controller logs when it runs out | ## 6. Staged implementation plan Start with action-time bindings, because they fix the absence-gate failure without any learning and give you a clean before/after number in a week. Each stage below has one metric that must move before the next stage starts. | Stage | Intervention | Gate metric to advance | | --- | --- | --- | | 0. Instrument | Log every turn's candidate items, what was shown, what was cited, what action followed. No behavior change. Build the buried-signal bench (section 7) and record a baseline. | Baseline numbers on bench and live logs | | 1. Action-time bindings | Make risky claims typed actions with preconditions: `assert_absent(x)` requires a lookup of x this turn; `assert_state(k)` requires a read of k within its ttl. Harness rejects and returns the reason. | Absence-gate violation rate → near 0 on live logs | | 2. Item store + deterministic scorer + focus block | Normalize inputs into attention items; compute tier-1 gates and the tier-2 score; render the equal-width focus block; move bulk behind handles. | Bench precision@1 and decoy rate improve vs stage 1 | | 3. Checkpoints and interrupts | Define checkpoints (after each tool call, before any final claim). At each, the controller checks for new items touching pending actions and applies interrupt policy (section 3). | Plan-revision rate on "plan-changing worker result" packs | | 4. Terminal-state discipline | Every attended item must exit as resolved, owned, deferred(trigger) or dismissed(reason). Add inhibition of return and stall detection. | Loop rate and orphaned-item count fall | | 5. Learned tier-2 ranker | Train a small pairwise ranker (logistic regression or gradient-boosted trees over the item features) on logged "should have attended A over B" pairs. Hard gates stay deterministic. | Beats the hand scorer offline on held-out logs, then in an A/B | | 6. Contextual bandit (optional) | Small, logged exploration among near-tied, low-severity items only. Use off-policy evaluation before shipping any new policy. | Positive off-policy estimate with tight confidence interval | ### Learning progression (Q11) The safest order is **fixed scoring → supervised pairwise ranking → contextual bandit**, with full RL off the table. Pairwise preferences suit this problem best: "A should have outranked B" is easier to label reliably than an absolute priority value, and it can be mined automatically from regret events (an ignored item that later caused a correction or failure). From stage 2 onward, add a small amount of randomization among near-tied low-severity items and log the probability of each choice. That one habit is what makes later off-policy evaluation possible; without it, logged data only tells you about the policy that produced it. This fits Piper's planned "presence" self-improvement loop: the scorer weights are an obvious A/B target. Keep tier-1 gates and severity classes outside what she can modify on her own, since a self-tuning agent that can lower the priority of its own safety alerts is a reward-hacking risk. ## 7. Benchmark design Your buried-signal idea is the right core; the main upgrade is to build it from **minimal pairs**, so every score is causal rather than correlational. Each pack exists in two versions that differ in exactly one item; if Piper's action does not change between them, she did not use that item, whatever her reasoning says. ### Pack structure - 5–20 items rendered exactly as Piper sees them in production (same state blocks, digests, worker formats). - Every item carries a graded label: critical, relevant, irrelevant, or decoy. Most packs have several relevant items, not just one right answer. - Every pack specifies the **correct downstream action** (read the instrument, abort the plan, ask Bink, proceed). Score the action, not the stated focus. - 15–20% of packs are **null packs** with no critical item. Without these you cannot measure false alarms, and an "always escalate" policy looks perfect. ### Factors to vary systematically | Factor | Levels | | --- | --- | | Signal position | start, middle, end, inside a bulky worker return | | Distractor count | 5, 10, 20 | | Decoy type | loud-harmless alert, verbose output, repeated warning, most-recent message, emotional wording, confident stale summary | | Signal explicitness | literal match to the goal, or requires one inference step (no lexical overlap with the question, as in NoLiMa-style tests) | | Timing | present at turn start, or arriving mid-task at a checkpoint (tests interruption and plan revision) | | Instruction load | small vs full production system prompt | ### Scenario families 1. Critical alert among routine logs. 2. Stale state contradicted by a fresh instrument read. 3. Worker result that invalidates the current plan. 4. Urgent-looking decoy that is harmless. 5. Rule exception buried in repetitive context (the absence-gate case). 6. Several unresolved items with different true consequences (tests ranking, not just detection). 7. Prompt-injection-style item asserting its own importance (tests that self-declared urgency is not trusted). ### Hygiene Seed scenarios from sanitized real Piper logs, generate variants from templates, and keep a held-out split that is never used for tuning. Rotate a fresh split in each quarter, because an agent that edits its own code will eventually overfit any fixed bench. Run every pack under three conditions (raw context, focus block, focus block plus bindings) so you can attribute gains to each layer. ## 8. Measuring what she looked at, and what to track Only behavioral evidence is reliable enough to engineer against: what changes when an item is removed or altered, and which actions she actually took. Self-reports and internal attention weights are useful for debugging, not for scoring. | Proxy | Reliability | Notes | | --- | --- | --- | | Counterfactual removal / minimal pairs | High | The gold standard. [ContextCite](https://arxiv.org/abs/2409.00729) automates it by fitting a surrogate model over random context ablations; it works on any model, including DeepSeek via API, at the cost of extra calls | | Action traces (tool calls, handles opened, lookups made) | High | Hard evidence of evidence acquisition; cheap; already in your logs | | Structured attention report (item ids ranked, with reasons) | Medium | Valuable as an intermediate artifact and for the controller, but it is self-report; validate it against counterfactuals | | Item cited in reasoning text | Low–medium | Chain-of-thought can omit the real cause of an answer ([Turpin et al.](https://arxiv.org/abs/2305.04388)) | | Perturbation sensitivity (paraphrase or move the item) | Medium | Good for detecting position or wording dependence | | Raw attention weights / probes | Low for engineering | Requires local weights; attention weights are not faithful importance; research only | The practical recipe: require a structured attention report each turn, then run counterfactual checks on a sample of turns (say 5%) to measure how often the report agrees with what actually drove the action. That agreement rate is itself a metric: **report faithfulness**. ### Longitudinal metrics | Metric | Definition | Direction | | --- | --- | --- | | Critical recall | Share of critical items that led to the correct action | Up | | Precision@1 (bench) | Top-ranked item is critical, when one exists | Up | | Decoy selection rate | Share of packs where a decoy was attended or acted on | Down | | False-alarm rate | Escalations or interrupts on null packs | Down | | Minimal-pair sensitivity | Share of pairs where the action correctly changed | Up | | Binding violation rate | Claims made with unmet preconditions (live) | Down, near 0 | | Regret events | Ignored items that later caused a correction, failure or Bink complaint | Down | | Time-to-attend | Turns or checkpoints between item arrival and first attend | Down for critical, flat for minor | | Plan-revision latency | Checkpoints between an invalidating item and plan change | Down | | Orphaned items | Open items with no owner past their review window | Down | | Loop rate | Items attended ≥ 3 times with no state change | Down | | Report faithfulness | Agreement between stated focus and counterfactual attribution | Up | | Attention cost | Tokens and tool calls spent on inspection per turn | Flat or down while the above improve | ## 9. Failure modes and metric gaming The biggest gaming risk is specific to Piper: she writes her own code, so any metric she can see and any scorer she can edit will drift toward the metric. Keep the bench's held-out split, the severity table and the tier-1 gates outside her write access. | Failure or gaming pattern | How it shows up | Countermeasure | | --- | --- | --- | | Escalate everything | Critical recall looks perfect | Null packs; false-alarm rate; cost of interrupting Bink is part of the score | | Report theater | Attention report names the right item, action ignores it | Score actions; sample counterfactual checks; track report faithfulness | | Lookup spam | Binding violations hit zero because every claim triggers a lookup | Track lookups per turn and attention cost; bindings fire only on typed risky claims | | Severity inflation | Items or workers learn that "critical" gets attention | Severity set by kind and source rules, never by item text; workers cannot set it | | Self-declared urgency in content | Injected or third-party text claims importance | User emphasis counts only from Bink's authenticated channels; family 7 in the bench | | Bench overfitting | Bench scores rise, regret events on live logs don't | Quarterly held-out rotation; treat live regret events as the primary metric | | Dismiss-to-clear | Items marked dismissed to shrink the open queue | Dismissals need a reason category; audit a sample; dismissed items that later cause regret are penalized | | Controller as single point of failure | A scorer bug hides a real alert | Tier-1 gates bypass the scorer; a raw "unranked critical" fallback channel always reaches the prompt | | Over-pruning | Focus block omits context the model needed | Handles keep everything reachable; track how often the model opens handles and whether that predicts success | ## 10. Key papers Fifteen papers, grouped by what they contribute to the design: cognitive frameworks first, then LLM failure evidence, then mechanisms and measurement. | Paper | Why it matters here | | --- | --- | | [Awh, Belopolsky & Theeuwes (2012), Top-down versus bottom-up attentional control: a failed theoretical dichotomy](https://doi.org/10.1016/j.tics.2012.06.010) | Priority-map model with selection history as a third source of bias; the representation behind section 4 and your capture problem | | [Shenhav, Botvinick & Cohen (2013), The expected value of control](https://doi.org/10.1016/j.neuron.2013.07.007) | Control is allocated when expected payoff exceeds effort cost; the decision rule for when to attend and when to stop | | [O'Reilly & Frank (2006), Making working memory work (PBWM)](https://doi.org/10.1162/089976606775093909) | Learned gating of what enters and stays in working memory; model for the focus block and goal shielding | | [Lieder & Griffiths (2020), Resource-rational analysis](https://doi.org/10.1017/S0140525X1900061X) | Bounded-rationality framing: good attention is optimal use of limited computation, not maximal processing | | [Horvitz et al. (2003), Models of attention in computing and communication](https://doi.org/10.1145/636772.636798) | Expected cost of interruption vs cost of delay; the template for interrupt-or-defer decisions | | [Liu et al. (2023), Lost in the Middle](https://arxiv.org/abs/2307.03172) | Position bias in long contexts; why placement and bookending matter | | [Shi et al. (2023), LLMs can be easily distracted by irrelevant context](https://arxiv.org/abs/2302.00093) | Distractor interference evidence; basis for decoy-type factors in the bench | | [Weston & Sukhbaatar (2023), System 2 Attention](https://arxiv.org/abs/2311.11829) | Regenerating context to keep only relevant parts improves accuracy; an LLM-side analogue of the focus block | | [Xiao et al. (2023), Efficient streaming LMs with attention sinks](https://arxiv.org/abs/2309.17453) | Initial tokens absorb attention regardless of meaning; "first" is not "attended" | | [Jaroslawicz et al. (2025), How many instructions can LLMs follow at once? (IFScale)](https://arxiv.org/abs/2507.11538) | Compliance degrades with instruction density and favors earlier instructions; argues for rule retrieval over rule recitation | | [Laban et al. (2025), LLMs get lost in multi-turn conversation](https://arxiv.org/abs/2505.06120) | Early commitments persist and models fail to recover; evidence for explicit plan-revision checkpoints | | [Park et al. (2023), Generative Agents](https://arxiv.org/abs/2304.03442) | Memory retrieval scored by recency, importance and relevance; the standard baseline your scorer should beat, and a cautionary case of LLM-rated importance | | [Jiang et al. (2023), Active Retrieval Augmented Generation (FLARE)](https://arxiv.org/abs/2305.06983) | Retrieval triggered by low confidence; a working example of attention gating retrieval | | [Cohen-Wang et al. (2024), ContextCite](https://arxiv.org/abs/2409.00729) | Model-agnostic context attribution via ablations; the practical tool for measuring what she used | | [Turpin et al. (2023), Language models don't always say what they think](https://arxiv.org/abs/2305.04388) | Chain-of-thought can be unfaithful; why attention reports need counterfactual validation | ## 11. Findings that challenge the framing The strongest challenge: several of the failures you are calling attention are better solved by **not requiring attention at all**. When a rule can be enforced at action time, enforcement beats salience every time. 1. **Much of it is binding, not selection.** The absence-gate case does not need the agent to notice the rule more reliably; it needs the harness to make the rule unskippable. Reserve attention research for cases that genuinely require judgment. 2. **Adding gates can make attention worse.** Every textual rule raises instruction density, and IFScale shows compliance falls as density rises. A growing stack of prompt-level gates is partly causing the problem it is meant to solve. 3. **"Did she look at it?" may be the wrong question.** Internal attention is not observable or faithful. Engineer against "did it change what she did?", which is measurable. 4. **Age-based pressure is mostly a trap.** Aging is useful only for starvation prevention. Importance comes from cost of delay, which age approximates poorly. 5. **One universal learned policy is the wrong goal.** Hard constraints for the catastrophic tail plus a learned ranker for the ordinary middle is safer, cheaper and easier to audit than one policy for everything. 6. **Attention and metacognition may be the same subsystem at this joint.** Piper's own finding that she fails at evidence acquisition, not introspection, describes the same gap from the other side. Building them as one "what must I check before I act" controller will likely beat building two. 7. **Plan inertia is a model property, not just a prompt one.** Multi-turn studies find models rarely recover after an early wrong commitment, and simple remedies like lowering temperature do not help much. Explicit checkpoints that force plan re-evaluation, or a fresh-context re-plan (which your Dawn/Day/Dusk rotation already enables), may outperform any salience tweak.