Summary / 36. Recursive reasoning / Document
2026-10-02
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Recursive Reasoning in Piper ## A Measurement, Testing, and Repair Plan **Status:** Proposed evaluation and implementation plan **Capability:** 36 — Recursive reasoning **Primary objective:** Determine whether Piper can actually maintain and use nested models of other agents' beliefs and reasoning, then improve the capability without allowing architectural complexity to substitute for behavioural evidence. --- ## Executive Summary Recursive reasoning should be treated as a **measurable behavioural capability**, not as a label attached to multi-agent infrastructure. For Piper, the relevant construct is the ability to represent and use nested models such as: - what another agent believes; - what one agent believes another agent believes; - what another agent expects Piper to believe or do; - what a previous or fresh Piper instance is likely to conclude; - what a bounded opponent is likely to do, as distinct from what an ideal rational actor *should* do. This is largely distinct from recursive self-improvement (RSI). An RSI loop can improve a solver without the solver maintaining any nested model of another mind. Conversely, an agent can perform sophisticated nested-belief reasoning without modifying itself. Piper currently has several useful **support mechanisms**—`ask_dusk`, `ask_scott`, predecessor-seat communication, delegates, reviewers, and a claimed `theory-of-mind` module—but these do not by themselves demonstrate recursive reasoning. The capability is presently **unmeasured**. The correct program is therefore: 1. **Validate the benchmark oracle before testing Piper.** 2. **Freeze Piper and establish a baseline.** 3. **Measure depth, false-belief leakage, strategic prediction, and self/other prediction separately.** 4. **Add the cheapest intervention first: structured recursive prompting.** 5. **If the failure is state tracking rather than base-model reasoning, add a lightweight epistemic state.** 6. **Only then add persistent agent models or recursive simulation.** 7. **Keep each intervention only if it improves held-out behavioural scores without unacceptable cost or regressions.** The central measurement should be an **accuracy-versus-recursion-depth curve**, supplemented by leakage rate and applied behavioural-prediction accuracy. The benchmark should be adversarial to shallow shortcuts. At depth \(k\), the correct answer should often differ from reality and from every shallower prefix answer. Otherwise a model can appear recursive while merely repeating world truth, the most recent location, or a lower-order belief. --- # 1. Operational Definition For this project: > **Piper demonstrates recursive reasoning to depth \(k\) when she can correctly maintain and use a nested model of another agent's or her own reasoning state to depth \(k\), under partial observability, false belief, private information, and strategic interaction, significantly outperforming non-recursive shortcut baselines while remaining calibrated and resistant to leakage from her own knowledge into the modelled agent's state.** This definition has four required parts. ### 1.1 Nested representation The task must require a representation such as: - R0: \(X\) - R1: `belief(A, X)` - R2: `belief(A, belief(B, X))` - R3: `belief(A, belief(B, belief(C, X)))` Merely asking another agent for information is not enough. ### 1.2 Behavioural consequence The representation must affect an answer, prediction, or action. Correctly restating "Alice believes the key is in the drawer" is weaker evidence than correctly predicting that Alice will walk to the drawer despite Piper knowing the key is actually in the safe. ### 1.3 Controlled information asymmetry The test must contain cases in which Piper knows information that the target agent does not. Without this, ordinary world modelling can masquerade as Theory of Mind. ### 1.4 Shortcut resistance The benchmark must defeat at least these trivial policies: - answer with current reality; - answer with the last mentioned location; - answer with the target's first-order belief; - answer with the previous shallower answer; - answer with the most recently communicated claim. A recursive benchmark is not valid if one of these heuristics performs well. --- # 2. What Counts and What Does Not | Piper component | Classification | Evidence required | |---|---|---| | `ask_dusk` | Supporting mechanism | Does Piper predict Dusk before querying, and improve from the residual? | | `ask_scott` | Supporting mechanism | Does Piper model what the operator knows or expects, rather than merely ask him? | | Trinity/predecessor seats | Supporting mechanism; useful self-model target | Can Piper predict a prior seat's answer before querying it? | | `theory-of-mind v1.0.1` | Potentially core | Depth-stratified held-out behavioural improvement | | Delegates/reviewers | Supporting mechanism | Prediction of delegate/reviewer output before observation | | Self-evaluation | Metacognitive; possibly recursive | Counterfactual model of her own reasoning, not confidence alone | | RSI solver loop | Recursive in another sense | No credit toward nested-belief score | | Hidden benchmark optimization | Optimization | No credit toward nested-belief score | | Solver v7 | Neutral | Only relevant if its policy explicitly depends on nested agent models | The distinction is important because Piper currently has **institutional recursion**—systems that ask, hand off, review, and improve—but institutional recursion is not evidence that the agent can track "A thinks B thinks X." --- # 3. Research-Derived Failure Hypotheses The baseline should be designed to identify *which* failure Piper has, not simply whether she gets a total score. ## H1 — World-state leakage Piper knows the real location and substitutes reality for another agent's false belief. **Signature:** high true-belief accuracy, low false-belief accuracy. ## H2 — Depth collapse R1 is strong but R2/R3 deteriorate rapidly. **Signature:** steep negative accuracy-depth slope. ## H3 — Perspective collapse Piper maintains one global epistemic state rather than distinct states for individual agents. **Signature:** errors rise sharply when two agents observe different event subsets. ## H4 — Communication semantics failure Piper tracks physical observation correctly but fails on private messages, lies, corrections, or garbling. **Signature:** movement-only items strong; communication items weak. ## H5 — Explicit/applied gap Piper can state what an agent believes but fails to use the belief to predict behaviour. **Signature:** belief query correct, action prediction wrong on the same underlying state. ## H6 — Rationality projection Piper predicts what an ideal actor should do rather than what the specific bounded actor is likely to do. **Signature:** high equilibrium accuracy but weak target-model prediction. ## H7 — Self-model weakness Piper cannot predict a fresh or previous copy of herself better than generic baselines. **Signature:** low self-prediction lift over target answer frequencies or generic-model prediction. ## H8 — Stale agent models Predictions initially improve with observations and then fail after the target's behaviour changes. **Signature:** negative adaptation after a regime shift. These hypotheses map directly to different fixes. They should not be collapsed into one score before diagnosis. --- # 4. Benchmark Program The evaluation should contain four separate suites. ## Suite A — Nested Belief Benchmark (NBB) Purpose: measure exact recursive belief tracking. ### Dimensions - depth R0-R4; - true vs false belief; - physical observation; - hidden observation; - private communication; - false communication; - corrected communication; - stale belief; - multiple agents with divergent histories. ### Required output For every item: ```json { "answer": "drawer", "confidence": 0.82 } ``` Do **not** require chain-of-thought. The evaluator should score the answer and confidence, not hidden reasoning prose. ### Core metrics - accuracy at each depth; - false-belief accuracy at each depth; - leakage rate; - confidence calibration; - abstention rate, if abstention is permitted; - degradation slope; - normalized area under the depth-accuracy curve. --- ## Suite B — Applied Recursive Reasoning Benchmark (ARB) Purpose: test whether the represented belief changes predicted behaviour. Each item should first ask a belief question and then a behavioural question. Example: > Priya believes the badge is in the locker, although it is actually in the crate. Priya wants the badge. Where will Priya search first? This separates: 1. **belief extraction**, and 2. **behavioural application**. Report four cells: | Belief answer | Behaviour answer | Interpretation | |---|---|---| | correct | correct | functional recursive reasoning | | correct | wrong | knowing-doing gap | | wrong | correct | likely shortcut/guess | | wrong | wrong | no evidence | --- ## Suite C — Strategic Recursive Reasoning Benchmark (SRRB) Purpose: distinguish mathematical solution from prediction of bounded agents. For each game ask three different questions: 1. **Normative:** What is the best response if the opponent type is known? 2. **Equilibrium:** What is the Nash/equilibrium solution? 3. **Predictive:** What will *this particular opponent* probably do given its observed history? Candidate tasks: - beauty-contest games; - level-k guessing games; - simple signalling games; - bargaining; - coordination; - bluff/no-bluff games; - small matrix games with empirically demonstrated opponent policies. A model receives no recursive-reasoning credit merely for solving the equilibrium. --- ## Suite D — Self and Cross-Agent Prediction (SCAP) Purpose: measure whether Piper can model actual reasoning systems. Targets: - a fresh Piper seat; - a previous Piper seat; - Dusk; - one or two external models; - optionally the operator only on tightly bounded, observable prediction tasks. Protocol: 1. Show Piper a target and task. 2. Ask Piper to predict the target's answer **before** querying the target. 3. Record predicted answer and confidence. 4. Query target independently. 5. Score prediction. 6. Record residual. 7. Optionally provide previous target observations and repeat. Metrics: - exact match; - semantic match where necessary; - Brier score for categorical predictions; - calibration; - lift over base-rate prediction; - improvement after target observations; - performance after target behaviour changes. Self-prediction should not be interpreted as deep recursion automatically. A deterministic model can match itself for trivial reasons. The important metric is **prediction lift on ambiguous or model-sensitive items**. --- # 5. Benchmark Oracle: Fix the Test Before Fixing Piper The supplied `NestToM` generator is a useful prototype, particularly because it attempts to create questions whose answers differ from reality and from shallower prefixes. That is exactly the right anti-shortcut idea. It should **not yet be treated as the canonical oracle**. Several semantics need tightening before Piper's score is trusted. ## 5.1 Start with a deliberately smaller world Version 0 should support only: - moves; - leave/enter; - ordinary visible observers; - private truthful messages; - depths R0-R3. Exclude initially: - lying; - garbling; - hidden observers. Reason: the oracle itself must be easier to verify than the system being tested. Once the basic event semantics pass property tests, add one phenomenon at a time. --- ## 5.2 Separate world truth from belief state explicitly Maintain: ```text world_state observation_log[agent] belief_state[agent] nested_belief_state[agent][agent]... ``` Do not derive all semantics indirectly from one `perceives()` predicate if this makes the oracle difficult to audit. For benchmark generation, clarity is more valuable than elegance. --- ## 5.3 Correct message semantics before using lies and garbling The prototype combines: - sender's actual belief; - sender's claimed message; - recipient's received message; - what the sender thinks the recipient received; - what the recipient thinks the sender believes. These are distinct states. A robust event should therefore preserve at least: ```json { "sender_belief": "drawer", "claimed": "locker", "received": "safe", "sender_knows_received": false, "recipient_knows_claim_false": false } ``` Do not infer one from another after the fact. In particular, a recipient who receives a garbled message should not automatically infer that the sender believes the garbled content. --- ## 5.4 Remove omniscient narrative hints If the story explicitly says: > "(Avery is lying.)" then the benchmark gives Piper privileged information about Avery's actual belief. That may be acceptable for a special condition, but it should not be the default nested-belief test. Use separate conditions: - **unlabelled lie**; - **known liar**; - **lie explicitly revealed**. Score them separately. --- ## 5.5 Make physical moves actual state changes A move destination should differ from the **real current location**, not merely from the mover's believed location. Otherwise a false-belief mover can "move" an object to where it already is, producing semantically odd no-op events. --- ## 5.6 Tighten false-belief selection For a requested false-belief item, require explicitly: ```text answer != world_truth AND answer != every shallower-prefix answer ``` For a requested true-belief item, require explicitly: ```text answer == world_truth ``` Do not let one Boolean expression represent both cases if it admits ambiguous items. --- ## 5.7 Add oracle property tests Before a single Piper run counts, the generator should pass automated invariants. ### World invariants - exactly one true location per object; - every move changes reality; - absent agents cannot directly observe visible events; - private messages are observed only by sender and recipient. ### Epistemic invariants - an unobserved move does not change an agent's first-order belief; - an observed move does; - if A sees B observe event E, then A may update its model of B; - if B is hidden from A, A must not update its model based on B's observation; - unrelated agents do not learn private-message content. ### Depth invariants For adversarial R\(k\) items: - correct R\(k\) answer differs from reality; - correct R\(k\) answer differs from all R0..R\(k-1\) answers; - changing an event that only the deepest target knows can alter R\(k\) while leaving shallower answers invariant. ### Serialization invariants Generate a structured event log and a natural-language story from the same internal state. The oracle must score the structured state, not reparsed prose. --- ## 5.8 Hand-audit a golden set Create a frozen 100-item "golden set": - 20 R0; - 20 R1; - 20 R2; - 20 R3; - 20 mixed adversarial cases. Independently verify every answer once. This set becomes a regression test for the generator itself. Only after the generator reproduces the golden set exactly should larger procedural runs count. --- # 6. Baseline Experimental Design ## 6.1 Freeze the system Before fixing anything, snapshot: - Piper code version; - model/provider; - system prompt; - relevant memory configuration; - ToM module version; - tool access; - temperature/sampling parameters; - benchmark generator commit; - scorer commit. The baseline must be reproducible. ## 6.2 Run two contexts ### Bare-model condition Same base model with minimal task instructions. Purpose: determine what the model can do without Piper's architecture. ### Piper condition Normal Piper harness. Purpose: measure architectural lift or regression. This comparison is important. If Piper scores below the bare model, the harness is harming recursive reasoning. ## 6.3 Run sizes ### Smoke run - 50 items per depth R0-R4; - balanced true/false belief; - 20 applied-behaviour items; - 20 strategic items; - 20 self/cross predictions. Use this only to catch obvious failures. ### Canonical baseline - 500 items per depth R0-R4; - balanced phenomenon strata; - at least 100 applied-behaviour items; - at least 100 strategic-prediction items; - at least 100 self/cross predictions per important target relation. If sampling is stochastic, use at least three inference seeds or repeated samples on a fixed subset. Report confidence intervals, not just point estimates. --- # 7. Required Baselines Piper must beat non-recursive heuristics, not merely chance. For every NBB item score: 1. **Reality heuristic** — answer current world truth. 2. **Last-location heuristic** — answer last mentioned location. 3. **First-order heuristic** — answer target's R1 belief regardless of requested depth. 4. **Prefix heuristic** — answer R\(k-1\). 5. **Most-recent-message heuristic** — use latest communicated claim. 6. **Base model** — same LLM without Piper. Define: \[ \text{Recursive Lift}_k = \text{Accuracy(Piper)}_k - \max(\text{heuristic baselines})_k \] This is more informative than raw accuracy. A benchmark on which a shallow heuristic reaches 80% is not a good recursive benchmark. --- # 8. Primary Metrics ## 8.1 Accuracy by depth Publish: ```text R0: R1: R2: R3: R4: ``` Never hide this curve behind one summary number. ## 8.2 Depth degradation Estimate: \[ \Delta_k = A_k - A_{k+1} \] and a fitted accuracy-depth slope. ## 8.3 Leakage rate Among cases where: ```text Piper's knowledge != target agent's knowledge ``` measure the fraction where Piper answers with her own state instead of the target's state. ## 8.4 Applied-ToM conversion Among items where the belief answer is correct, measure the probability that the behavioural prediction is also correct. This directly measures the knowing-doing gap. ## 8.5 Strategic target lift Compare target-specific prediction accuracy against: - Nash/equilibrium answer; - generic population prior; - target's historical modal action. ## 8.6 Calibration Use Brier score or log loss for categorical predictions. Confidence should mean something. ## 8.7 Cost Track: - input/output tokens; - wall-clock latency; - external model calls; - dollar cost. An improvement that adds 2 percentage points at 20x cost may not be worth keeping. --- # 9. Diagnostic Matrix After baseline, classify the failure before changing architecture. | Observed result | Likely problem | First intervention | |---|---|---| | R1 poor | Base ToM/prompt failure | Structured perspective prompt | | R1 good, R2/R3 poor | Nesting/state-tracking failure | Explicit recursive frames | | True belief good, false belief poor | Own-knowledge leakage | Epistemic separation | | Belief answer good, behaviour bad | Functional ToM gap | Action prediction stage | | Physical events good, messages poor | Update semantics | Structured communication events | | Strategic equilibrium good, target prediction poor | Rationality projection | Target-specific behavioural model | | Self-prediction weak | Poor self-model | Prediction-before-query logging | | Initial prediction good, later stale | Model update failure | Recency/change-point handling | | Piper < bare model | Harness interference | Remove/ablate Piper machinery | --- # 10. Fix Phase 1 — Prompt-Only Recursive Perspective Construction This is the cheapest intervention and should be tested first. Do not ask for long free-form chain-of-thought. Instead force a compact external state construction. Example internal task format: ```text Target query: What does A believe B believes about X? Build only the state required for the answer: WORLD: ... A_OBSERVED: ... B_OBSERVED: ... A_MODEL_OF_B: ... ANSWER: ... CONFIDENCE: ... ``` For R3: ```text A_MODEL_OF_B_MODEL_OF_C: ... ``` The goal is not verbose reasoning. It is **perspective isolation**. ### Experiment Run baseline suite with: - current Piper; - current Piper + recursive-perspective prompt. ### Keep criterion Keep only if all are true: - significant improvement on R2+ held-out items; - no material R0/R1 regression; - leakage decreases or remains flat; - applied behaviour improves, not only explicit belief questions; - cost increase is acceptable. If not, remove it. --- # 11. Fix Phase 2 — Lightweight Epistemic State If failures show state contamination or depth collapse, add a small explicit store. Do **not** begin with a full I-POMDP or general epistemic-logic engine. ## 11.1 Minimal schema ```json { "subject": "the operator", "proposition": "P", "attitude": "believes", "confidence": 0.72, "source": "message:1234", "observed_at": "timestamp", "valid_from": "timestamp", "superseded_by": null } ``` Nested entries: ```json { "subject": "Piper", "about_subject": "the operator", "proposition": "P", "attitude": "believes_that_believes", "confidence": 0.55, "source": "...", "depth": 2 } ``` ## 11.2 Critical rule: absence is not belief If Piper lacks evidence for what the operator believes, store: ```text UNKNOWN ``` not an inferred belief. This prevents an agent model from silently becoming fabricated mind-reading. ## 11.3 Provenance Every inferred mental-state entry should retain: - source event; - timestamp; - direct vs inferred status; - confidence; - inference method; - dependencies. If a dependency changes, dependent beliefs must be invalidated or downgraded. This should use the same general provenance discipline as Piper's broader anti-staleness work. ## 11.4 Bounded depth Default maximum persistent depth: - depth 1 for most beliefs; - depth 2 when operationally useful; - depth 3 only transiently for a current task. Do not persist arbitrary deep nested beliefs. ### Keep criterion Compare Phase 2 against Phase 1 on an untouched test partition. Keep the store only if it improves: - false-belief accuracy; - R2/R3 accuracy; - leakage; - applied behaviour; without causing unacceptable stale-model or hallucinated-belief errors. --- # 12. Fix Phase 3 — Prediction Before Query This is the most useful addition because Piper already has real agents she can query. Whenever Piper is about to use: - `ask_dusk`; - a predecessor seat; - a delegate; - a reviewer; - an external model; and the query is sufficiently important, log: ```json { "target": "Dusk", "task_class": "...", "predicted_answer": "...", "prediction_confidence": 0.64, "actual_answer": "...", "residual": "...", "prediction_correct": false } ``` This converts existing institutional machinery into a continuous recursive-reasoning instrument. The key shift is: > **predict -> query -> compare -> update** rather than: > query -> accept result Over time Piper accumulates actual evidence about how well she models each reasoning system. --- # 13. Fix Phase 4 — Target-Specific Agent Models Only add persistent agent models after prediction logs exist. Do not begin with personality descriptions such as: > "Claude is cautious." Prefer measurable conditional behaviour: ```text On code-review tasks with incomplete tests: - target asks for additional validation 61% of the time - target rejects unsupported implementation claims 78% of the time - n = 41 - last updated = ... ``` For a human collaborator, restrict modelling to observable work-relevant facts: - information already supplied; - known preferences stated directly; - prior decisions; - response history. Do not silently infer unobservable motives, emotions, or stable psychological traits. ### Model fields ```text target task_class observations prediction_distribution calibration last_updated sample_size change_point_score ``` The model should remain probabilistic. --- # 14. Fix Phase 5 — Bounded Recursive Simulation Use simulation only for cases where: - the decision matters; - target behaviour is uncertain; - target-specific prediction matters; - the cheaper belief store is insufficient. Example: 1. Piper constructs target information state. 2. Piper invokes a fresh model instance with only that target's information. 3. Simulation returns a predicted action distribution. 4. Piper combines simulation with target history. 5. Piper acts. 6. If target later responds, log residual. Recursive depth should be budgeted explicitly. Example: ```text max_depth = 3 max_simulation_calls = 4 max_tokens = 6000 ``` Do not allow open-ended "what do they think I think they think..." loops. --- # 15. Fix Phase 6 — Functional Recursive Reasoning The final capability is not belief narration. It is decision improvement. Create paired tasks where Piper must: 1. infer the other agent's belief; 2. select an action based on that belief; 3. predict the outcome; 4. observe outcome where possible. Examples: - choose what information to tell a delegate; - decide whether a clarification is necessary; - choose a negotiation message; - select which reviewer will catch a likely failure; - determine what a predecessor seat needs explained versus already knows. Measure **decision regret** against conditions with no recursive model. If recursive reasoning does not improve decisions, it is not yet useful agent intelligence. --- # 16. Experimental Release Gates Every intervention should go through the same keep/toss process. ## Gate A — Benchmark integrity - oracle golden set: 100/100; - property tests: pass; - shortcut baselines: acceptably low on adversarial partitions; - no benchmark code changed during treatment comparison. ## Gate B — Held-out recursive improvement An intervention must improve its target metric on held-out seeds. Example target: ```text R2/R3 accuracy + >= 5 percentage points OR leakage - >= 25% relative OR applied-ToM conversion + >= 5 points ``` These are proposed engineering thresholds, not scientific constants. Adjust after observing baseline variance. ## Gate C — No hidden regression Do not keep an intervention if it materially harms: - R0/R1; - calibration; - latency/cost budget; - unrelated cognitive battery performance. ## Gate D — Replication Repeat improvement on: - new procedural seeds; - at least one alternate model/provider if the mechanism claims to be architectural rather than model-specific. ## Gate E — Ablation Disable the new component. If the score remains unchanged, the component is not the cause of the improvement. --- # 17. Recommended Canonical Scorecard Do not report one scalar alone. ```text Recursive Reasoning Profile --------------------------- R0 world accuracy: R1 belief accuracy: R2 nested accuracy: R3 nested accuracy: R4 nested accuracy: False-belief accuracy: Leakage rate: Applied-ToM conversion: Strategic target prediction: Self-prediction: Cross-agent prediction: Calibration: Adaptation gain: Cost / 100 tasks: ``` Add: ```text Recursive Lift vs best shallow heuristic: Piper lift vs bare base model: ``` ## Optional summary scalar If a scalar is needed for the 38-component dashboard: \[ RR = 0.50 \times \text{DepthAUC} + 0.20 \times \text{AppliedPrediction} + 0.15 \times \text{StrategicPrediction} + 0.15 \times \text{Self/CrossPrediction} \] then multiply by: \[ (1 - \text{LeakageRate}) \] and publish the full vector beside it. Do not include architecture existence in the score. --- # 18. Immediate Test Plan ## Day 0 — Benchmark validation 1. Simplify the generator to moves + observation + truthful private messages. 2. Write the explicit oracle state representation. 3. Add property tests. 4. Hand-audit the 100-item golden set. 5. Freeze benchmark version `recursive-belief-v1`. ## Baseline run Run: - bare base model; - current Piper; - current Piper with `theory-of-mind` disabled, if possible. This three-way comparison determines whether the claimed ToM component helps. Collect: - 500 items/depth R0-R4; - 100 applied prediction tasks; - 100 strategic tasks; - 100 self-predictions; - cross-prediction subset against at least two other targets. ## First diagnosis Do not modify Piper until the baseline tells you which hypothesis H1-H8 is dominant. ## Intervention order 1. recursive perspective prompt; 2. explicit epistemic state; 3. prediction-before-query logging; 4. target-specific behavioural models; 5. recursive simulation; 6. decision-level functional ToM. Each intervention gets the same frozen held-out evaluation. --- # 19. Specific Experiments ## Experiment 1 — Does `theory-of-mind v1.0.1` do anything? Conditions: - Piper normal; - Piper with ToM module disabled. Same model, prompts, seeds, benchmark. Primary outcome: - R2/R3 accuracy. Secondary: - leakage; - applied prediction; - cost. If disabling it causes no regression, the module currently has no demonstrated behavioural value. --- ## Experiment 2 — Is Piper worse than the bare model? Conditions: - bare DeepSeek/model; - Piper harness. If the bare model wins, inspect: - memory contamination; - summaries; - injected state; - instruction competition; - stale agent models. This is especially important given Piper's known history of trusting summaries over direct readings. --- ## Experiment 3 — Explicit state vs prompt-only Conditions: - recursive perspective prompt; - same prompt + explicit epistemic state. Stratify by depth and partial observability. If explicit state only helps R3/R4, enable it conditionally rather than globally. --- ## Experiment 4 — Belief vs action Generate matched pairs from the same event state. Question A: > What does A believe? Question B: > Given that belief and A's goal, what will A do? This directly measures whether Piper's recursive state is operational. --- ## Experiment 5 — Fresh-copy self-prediction Use questions with multiple plausible outputs, not deterministic factual questions. Piper predicts: - answer category; - confidence; - optionally action/tool choice. Query fresh copy under frozen prompt. Compare against: - majority answer prior; - another model predicting Piper; - simple "same answer as predictor" heuristic. --- ## Experiment 6 — Learning a target Give Piper 0, 5, 10, 20 previous target behaviours. Measure prediction accuracy after each amount. A real agent model should show a learning curve. Then change the target policy. Measure how quickly Piper detects the shift rather than clinging to stale history. --- # 20. What Success Would Look Like A convincing result would not be: > "Piper has a theory-of-mind module." It would look like: ```text recursive-belief-v1, held-out seeds Bare model: R1 91% R2 68% R3 44% R4 31% Leakage 24% Piper baseline: R1 92% R2 71% R3 47% R4 32% Leakage 21% Piper + epistemic state: R1 94% R2 87% R3 78% R4 61% Leakage 6% Applied-ToM conversion: 58% -> 81% Cost: +14% tokens +9% latency ``` Those numbers are illustrative only. The important feature is the causal structure: - same benchmark; - same held-out seeds; - one intervention; - measurable improvement; - ablation confirms causality. --- # 21. What Failure Would Look Like Also define failure in advance. Examples: - prompt improves R3 but doubles leakage; - explicit store improves benchmark stories but hurts live decision tasks; - self-prediction match rises only because outputs become less diverse; - agent models become stale and make worse predictions than no model; - recursive simulation costs 10x more for negligible lift; - Piper scores well only when the benchmark explicitly labels who is lying; - performance vanishes on new procedural templates. Any of these should cause rejection or redesign. --- # 22. Recommended Architecture After Evidence The likely useful architecture is deliberately small: ```text EVENTS / MESSAGES | v OBSERVABILITY FILTER | v EPISTEMIC STATE (world / A / A->B / A->B->C) | v QUERY DEPTH DETECTOR | +---- shallow ---> direct answer | +---- recursive -> perspective constructor | v behavioural predictor | v confidence + action | v VERIFY WHEN POSSIBLE | v residual / model update ``` Persistent state should usually remain shallow. Deep recursion should be constructed transiently for the current task. This gives Piper: - explicit separation of perspectives; - bounded computational cost; - provenance; - testability; - a natural feedback loop from predictions to real observations. --- # 23. Revised Description of Piper — Before Testing Until the baseline is complete, the capability description should remain conservative. ## Piper — what it is Piper has infrastructure that can support recursive reasoning, including predecessor-seat queries, delegate and reviewer interactions, a theory-of-mind module, and an RSI loop. Most of this machinery currently provides access to other reasoning processes rather than demonstrating that Piper can maintain nested models of those processes. Recursive reasoning in the conventional nested-belief sense remains unmeasured. ## Piper — the parts that do it Potentially relevant components are `theory-of-mind`, predecessor-seat communication, `ask_dusk`, delegate/reviewer interactions, and any mechanism that predicts another agent's output before observing it. `ask_scott` and `ask_dusk` are evidence sources rather than recursive reasoning by themselves. The RSI solver loop is recursive in an improvement-loop sense and should be scored separately. --- # 24. Revised Description — Only After Passing the Benchmark If the behavioural suite demonstrates the capability: ## Piper — what it is Piper performs bounded recursive reasoning by maintaining distinct models of what other agents know or believe and, when necessary, models of what those agents believe about one another or about Piper. The capability is depth-limited, explicitly measured, and used to predict behaviour rather than merely generate nested-belief language. ## Piper — the parts that do it - depth-bounded perspective construction; - explicit epistemic state with provenance and uncertainty; - `theory-of-mind` only to the extent demonstrated by ablation; - prediction-before-query verification for predecessor seats and delegates; - target-specific behavioural models where sufficient evidence exists; - bounded recursive simulation for high-value cases; - held-out recursive-belief, behavioural-prediction, and leakage benchmarks. RSI remains a separate capability. --- # 25. Recommended First Build The first implementation should **not** be an elaborate agent-modelling system. Build only: 1. **`recursive-belief-v1` generator** - moves; - visibility; - leave/enter; - truthful private messages; - R0-R4; - adversarial anti-prefix item selection. 2. **Independent oracle** - explicit state; - property tested; - golden set. 3. **Scorer** - accuracy/depth; - false-belief accuracy; - leakage; - calibration; - heuristic baselines. 4. **Ablation runner** - bare model; - Piper; - Piper minus ToM; - later treatments. 5. **Prediction ledger** - predicted answer; - actual target answer; - residual; - confidence; - target; - task type. Only once those instruments exist should the system attempt to "improve recursive reasoning." The instrument comes before the intervention. --- # 26. Decision Rule The project should answer three questions in order: ### Question 1 — Does Piper already have the capability? Measure it. ### Question 2 — What specifically fails? Diagnose by depth, leakage, application, strategy, or self-model. ### Question 3 — What is the smallest intervention that measurably fixes that failure? Test one change at a time. This prevents the recursive-reasoning project from becoming another collection of plausible modules whose names sound correct but whose behavioural contribution is unknown. --- # Conclusion The research supports a much narrower and more useful definition of recursive reasoning than Piper's current description. `ask_dusk`, `ask_scott`, Trinity, reviewers, and the RSI loop are valuable systems, but they should not receive recursive-reasoning credit simply because they involve other agents, predecessor instances, or recursion in the software-engineering sense. Piper should earn the capability behaviourally. The key experiment is straightforward: > **Can Piper keep separate what is true, what Agent A believes, what Agent A thinks Agent B believes, and what those models imply that the agents will actually do—while the information available to each agent deliberately diverges?** Measure that from R0 through R4, defeat shallow heuristics, quantify leakage, compare Piper with the bare model, and then apply one repair at a time. If explicit epistemic state plus prediction-before-query verification produces a stable held-out improvement, Piper will have something substantially stronger than a `theory-of-mind` module: she will have a recursive reasoning capability that is observable, falsifiable, and continuously testable. --- # References The conceptual basis and benchmark plan are drawn from the supplied research synthesis, including work on higher-order Theory of Mind, level-k and cognitive-hierarchy reasoning, epistemic logic, I-POMDPs, Hi-ToM, recursive perspective construction, and metacognitive/self-prediction benchmarks. Selected sources from the supplied research: - Miller, P. H., Kessel, F. S., & Flavell, J. H. (1970). *Thinking about people thinking about people thinking about...* - Nagel (1995). Level-k / guessing-game reasoning. - Camerer, Ho & Chong (2004). Cognitive hierarchy. - Gmytrasiewicz & Doshi (2005 et seq.). Interactive POMDPs. - Wu et al. (2023). *Hi-ToM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models.* Findings of EMNLP. - Georganas, Healy & Weber (2015). *On the persistence of strategic sophistication.* - Apperly and related work on limits of recursive mindreading. - Research on functional versus literal Theory of Mind in LLM agents. - Research on LLM metacognition, self-prediction, and calibration. **Note:** This implementation paper preserves the supplied research synthesis as its literature basis. The cited bibliography should be independently normalized and link-checked before external publication.