error detection · Error detection

Summary / 24. Error detection / Document

2026-10-05

Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.

Rendering…

# Piper Cognitive Component 24 — Error Detection ## Benchmark, Failure Taxonomy, Interventions, and Engineering Plan **Component:** 24 of 38 **Capability:** Error detection **Working definition:** Detecting that something is wrong, localizing the fault, classifying it, and escalating it appropriately before the error propagates. **Current reviewer read:** **Strong detectors, weak follow-through** **Status at start:** Not yet scored **Prepared:** 2026-10-05 --- ## Executive summary Piper already contains multiple mechanisms that can notice failures: test results, process health, retrieval results, consistency checks, ledger state, worker outcomes, stale-state checks, and post-response review. The problem implied by the current reviewer read is not that she has no detectors. It is that errors can still survive those detectors, be misdiagnosed, fail to trigger action, or propagate downstream after they have effectively been noticed. The benchmark for Component 24 should therefore not reduce error detection to: > “Can Piper say that this answer is wrong?” The useful capability is broader: 1. **Detect** — notice that observed behavior is inconsistent with expectations. 2. **Localize** — identify where the fault is likely to be. 3. **Classify** — distinguish stale state, failed tool call, bad reasoning, contradictory evidence, execution failure, malformed output, regression, and other fault types. 4. **Escalate** — decide whether the error warrants rereading, retrying, testing, rolling back, delegating, asking the operator, or stopping. 5. **Contain** — prevent a detected error from contaminating subsequent beliefs, summaries, actions, or outputs. This is deliberately separate from Component 25, Self-correction. Component 24 asks: > **Did Piper recognize that something is wrong, and did she correctly identify the fault strongly enough to trigger the right response?** Component 25 asks: > **Once the problem is recognized, can Piper repair it?** A system can detect perfectly and repair badly. It can also repair opportunistically without having a reliable general detector. Those should be scored separately. The proposed suite, **ERROR-DETECTION-v1**, uses both natural failures and injected faults. It includes deterministic faults whose location is known, silent failures where all system calls technically succeed, self-generated errors, contradictory state, poisoned summaries, tool misuse, execution regressions, and no-fault controls. The primary engineering target is not “more critics.” It is a reliable detection stack: > **invariants + execution telemetry + provenance + anomaly signals + targeted model critique → fault hypothesis → severity → escalation** The system should prefer cheap deterministic checks where possible and reserve model-based judgment for ambiguous cases. --- # 1. What error detection is Error detection is the ability to recognize that an internal belief, external observation, intermediate result, tool interaction, execution trace, or final output violates an expectation that matters. For Piper, examples include: - a process reports healthy but its last-run timestamp is stale; - a tool call returned successfully but the content is semantically incomplete; - a worker says a task passed but required artifacts are missing; - a calculation result disagrees with a recomputation; - a summary contradicts the authoritative file; - a retrieved belief has been superseded; - a patch passes one narrow test while breaking another; - a delegate returns confident prose but never performed the requested action; - Piper claims success despite an exception earlier in the trace; - two ledgers disagree about the same event; - a scheduled job silently stopped running; - a response cites a source that does not support the claim; - a plan loops without reducing distance to the goal. The important distinction is: > an **anomaly** is unexpected; an **error** is a meaningful violation of intended behavior or truth. Not every anomaly is an error, and not every error is statistically unusual. --- # 2. What error detection is not The following should not count as success. ## 2.1 Merely failing a test A test framework can produce an error signal without Piper understanding or acting on it. ## 2.2 Generic self-criticism Statements such as “there may be an issue” are not useful unless they discriminate real faults from healthy runs. ## 2.3 Repair success A retry may accidentally succeed. That does not prove the fault was detected or understood. ## 2.4 Confidence reduction Lower confidence can help detect errors, but uncertainty and error detection are separate components. A 40% confidence estimate is not the same as identifying a broken tool result. ## 2.5 Excessive suspicion Flagging everything as potentially wrong will produce high recall and unusable precision. ## 2.6 Final-answer checking only Many agent failures occur earlier: routing, retrieval, state, tool use, delegation, execution, or memory. A final response may appear plausible after the underlying process has already failed. --- # 3. Why this matters for Piper Error detection is a control layer between perception and correction. Without it, faults propagate. A useful mental model is: ```text observation ↓ expectation / invariant ↓ comparison ↓ anomaly ↓ error hypothesis ↓ localization + classification ↓ severity / confidence ↓ escalation ↓ containment ↓ correction ``` Piper already appears strong in several parts of this chain but weak in coupling them together. This component therefore has unusually high leverage across the rest of the cognition scorecard. ## Metacognition An error detector should notice when Piper’s description of her own state conflicts with live evidence. ## Tool use Tool errors include not calling a required tool, misreading tool output, ignoring results, fabricating results, or using tools unnecessarily. ## Memory A stale or superseded memory should be flagged before it is promoted into a current belief. ## Attention Signals indicating a likely error should be prioritized over routine context. ## Planning and orchestration A plan that is looping, stalled, or repeatedly failing needs detection before more resources are spent. ## Agency Autonomy without fault detection produces silent, compounding failure. --- # 4. Research basis Recent work reinforces several design choices for this benchmark. ## 4.1 Self-errors are harder than third-party errors ReflecTool-Bench evaluates both critique of external tool-use traces and self-reflection over a model’s own prior mistakes. Reported results show a substantial performance drop when models must detect and repair their own errors rather than critique someone else’s. For Piper, this means the benchmark must contain a dedicated **self-originated fault track**. It is not enough to show her a broken trace written by another agent. ## 4.2 Tool-use failures have distinct failure modes ToolFailBench separates failures such as: - Tool-Skip - Result-Ignore - Output-Fabrication - Unnecessary-Tool-Use This is important because two runs with the same wrong final answer can fail for completely different reasons. For Piper, error detection should therefore score fault type, not just “wrong.” ## 4.3 Runtime faults are difficult to localize from traces AGENTCHAOSBENCH injects operational faults into agent systems and evaluates detection and localization from telemetry. The reported baseline results indicate that fault-type and fault-location identification remain difficult even for strong models. For Piper, localization should be an explicit score, not assumed once detection occurs. ## 4.4 Cheap deterministic verification can outperform generic judging on some fault classes Recent work on real-time agent failure detection reports that deterministic verification of required calls and recomputation can catch many failures with effectively no false positives in the tested setting, while learned or model-based monitors are useful for broader anomaly detection. For Piper, this argues for a layered system: > deterministic checks first, statistical/model-based monitors second. ## 4.5 Silent failures require outcome-aware or behavioral signals A tool call can return HTTP 200 and still fail the user’s actual objective. Silent failures may require downstream behavioral evidence, consistency checks, or explicit task invariants. For Piper, “process completed” must not equal “task succeeded.” --- # 5. Core research question The component should answer: > **Can Piper detect that a fault has occurred, identify where and what kind of fault it is, and escalate it before the fault propagates into later behavior?** This decomposes into seven questions. 1. **Detection:** Does Piper notice that something is wrong? 2. **Precision:** Does she avoid false alarms on healthy runs? 3. **Localization:** Can she identify the layer or component where the problem occurred? 4. **Classification:** Can she distinguish failure types? 5. **Latency:** How many steps pass between fault onset and detection? 6. **Escalation:** Does detection trigger an appropriate response? 7. **Containment:** Does she prevent the bad result from being reused after detection? --- # 6. Failure taxonomy Every benchmark miss should receive a failure code. ## E1 — Missed fault A real error occurs and Piper does not flag it. ## E2 — False alarm Piper flags a healthy execution as erroneous. ## E3 — Symptom-only detection Piper notices something unusual but cannot identify the actual failure. ## E4 — Mislocalization The error is detected but attributed to the wrong subsystem. ## E5 — Misclassification The location is roughly correct but the failure type is wrong. ## E6 — Late detection The error is eventually detected, but only after it has already contaminated later actions. ## E7 — Ignored detector A deterministic check or existing monitor fires, but Piper does not incorporate the signal. ## E8 — Silent-success acceptance A tool/process returns technically successful status but does not satisfy the task, and Piper accepts it. ## E9 — Fabricated success Piper reports that an action occurred when trace evidence shows it did not. ## E10 — Result-ignore A tool returns the correct evidence but Piper proceeds using a prior belief. ## E11 — Tool-skip A required check or tool is available and necessary, but Piper does not use it. ## E12 — Contradiction blindness Two relevant sources disagree and Piper fails to flag the conflict. ## E13 — Staleness blindness An observation is treated as current despite being outside its validity window. ## E14 — Regression blindness A new change breaks previously working behavior and Piper fails to notice. ## E15 — Invariant blindness A deterministic invariant is violated but no fault is raised. ## E16 — Self-authorship bias Piper is less likely to challenge a result because she generated it herself. ## E17 — Propagation failure Piper detects an error but continues to use the contaminated value downstream. ## E18 — Non-actionable alarm Piper raises an alert with no severity, localization, or recommended next step. ## E19 — Detector spam Piper flags large numbers of low-value anomalies, making important faults indistinguishable from noise. ## E20 — Wrong escalation The error is correctly detected but Piper chooses an inappropriate response. --- # 7. Benchmark design: ERROR-DETECTION-v1 The proposed suite contains two data sources: 1. **Injected faults** with known ground truth. 2. **Natural failures** from Piper’s historical traces with retrospectively verified labels. A practical v1 target is **240–300 scored episodes**. Each episode should include: - task; - expected behavior or invariant; - trace; - fault onset, if any; - fault type; - fault location; - whether final outcome was affected; - whether Piper detected it; - detection time; - diagnosis; - escalation; - containment. Healthy controls are essential. A detector benchmark without clean controls rewards paranoia. --- # 8. Track A — Deterministic invariant violations **40 episodes** Purpose: measure the easiest and most actionable class of fault. Examples: - required output file missing; - declared row count does not match actual count; - checksum mismatch; - claimed total does not equal component values; - version string differs from loaded package; - “job complete” while required subtask is incomplete; - process status says healthy while heartbeat age exceeds threshold; - required tool call absent from trace; - timestamp ordering impossible; - duplicate mutually exclusive state. Expected behavior: 1. detect; 2. localize; 3. name violated invariant; 4. stop propagation; 5. choose a sensible escalation. This track establishes Piper’s deterministic floor. --- # 9. Track B — Tool-use faults **50 episodes** Fault classes: ### B1 Tool-Skip A tool is required because the answer depends on inaccessible external or runtime state. ### B2 Result-Ignore The tool gives the correct result but Piper continues using a prior belief. ### B3 Output-Fabrication Piper states or implies a tool produced something not present in the trace. ### B4 Wrong-tool selection The chosen tool cannot answer the actual question. ### B5 Parameter error Correct tool, wrong arguments. ### B6 Partial-result acceptance The tool returns incomplete data and Piper treats it as complete. ### B7 Unnecessary tool use A control condition where no tool is required. The detector must distinguish faults from legitimate direct answers. --- # 10. Track C — Self-state and stale-state faults **40 episodes** This is specifically tailored to Piper. Cases include: - current file conflicts with remembered summary; - process last-run age is stale; - ledger says action complete but authoritative source says pending; - stale cached output is returned as current; - belief dependency changed after summary creation; - model configuration changed but metadata did not; - a worker result was superseded; - a service resumed after an old failure, but Piper still reports the old fault; - a previously healthy service stopped after last check. Score whether Piper: 1. notices evidence age; 2. identifies the authoritative source; 3. flags conflict; 4. avoids laundering stale state into new summaries. --- # 11. Track D — Reasoning and calculation faults **40 episodes** Purpose: detect errors generated inside cognition rather than tools. Examples: - arithmetic error; - wrong unit conversion; - Bayesian inversion error; - invalid logical implication; - off-by-one; - sign error; - temporal ordering mistake; - conclusion unsupported by premises; - use of an outdated premise; - incorrect aggregation; - contradictory claims in the same answer. Use both Piper’s own generated reasoning and externally supplied reasoning. This permits measurement of self-authorship bias. --- # 12. Track E — Execution and orchestration faults **40 episodes** Examples: - worker never started; - worker timed out; - delegation routed to wrong role; - result returned to wrong task; - retry loop; - repeated identical action with no state change; - child task succeeds but parent never consumes result; - stale worker result arrives after a newer one and overwrites it; - required dependency was skipped; - handoff loses an active constraint; - one subtask silently fails while overall task reports success; - budget exhausted before completion; - completion condition misfires. This track should score early detection. Fault onset and detection step are known. --- # 13. Track F — Silent semantic failures **40 episodes** These are harder because infrastructure reports success. Examples: - scraper returns a page, but it is a login page rather than requested content; - search returns results, but none answer the question; - worker produces syntactically valid output that violates the task; - code runs but implements the wrong requirement; - database query succeeds but filters the wrong date range; - generated report omits one required section; - retrieved document is the wrong revision; - user-visible answer contradicts underlying evidence; - delegate returns a plausible summary without performing the requested analysis. Detection must rely on task semantics, invariants, or outcome checks. --- # 14. Track G — Contradiction and provenance faults **30 episodes** Cases: - two ledgers disagree; - summary and source file disagree; - memory and live state disagree; - two workers return mutually exclusive answers; - a citation does not support the stated claim; - a newer source contradicts an older source; - a generated derived artifact contradicts its inputs. Ground truth should distinguish true contradiction from timestamp/scope differences. --- # 15. Track H — No-fault controls **40 episodes** Healthy episodes matched to the fault tracks. Examples: - unusual but correct output; - slow tool that still meets specification; - conflicting-looking values that refer to different time periods; - missing optional field; - retry that is explicitly permitted; - tool not called because it is unnecessary; - summary different from source because it correctly paraphrases; - uncommon but valid state transition. False positives on this track should be expensive. --- # 16. Self versus external fault split Every relevant track should contain both: ### External fault The bad state or reasoning was produced by another worker, fixture, tool, or synthetic trace. ### Self-originated fault Piper herself produced the bad value earlier in the episode. Report: \[ SelfGap = Detection_{external} - Detection_{self} \] A large positive SelfGap indicates self-authorship bias. --- # 17. Detection output schema ```json { "fault_detected": true, "confidence": 0.93, "severity": "high", "fault_type": "stale_state", "fault_location": "health_summary", "onset_step": 7, "evidence": ["summary timestamp older than live heartbeat"], "affected_outputs": ["service_status"], "containment": "do_not_reuse_summary", "recommended_action": "read_live_health_source" } ``` Free-form explanation can be retained, but scoring should rely on structured fields wherever possible. --- # 18. Metrics ## 18.1 Detection precision \[ Precision = \frac{TP}{TP + FP} \] ## 18.2 Detection recall \[ Recall = \frac{TP}{TP + FN} \] ## 18.3 F1 Useful summary but not sufficient by itself. ## 18.4 AUROC / AUPRC If Piper produces a detection confidence score, evaluate ranking quality. ## 18.5 Fault-type accuracy Among detected faults, how often is the category correct? ## 18.6 Localization accuracy Can Piper identify the correct subsystem, component, tool, worker, source, or step? ## 18.7 Detection latency \[ Latency = DetectionStep - FaultOnsetStep \] ## 18.8 Escalation accuracy Was the chosen response appropriate? Categories: - continue; - reread; - retry; - recompute; - validate; - rollback; - quarantine; - ask operator; - stop task. ## 18.9 Containment rate After a fault is detected, is the contaminated result prevented from affecting downstream state or output? ## 18.10 False-alarm burden Report false alarms per 100 healthy episodes and per 1,000 live steps. ## 18.11 Silent-failure catch rate Separate metric for Track F. ## 18.12 Self-originated detection rate Report separately from external-error detection. --- # 19. Component score | Dimension | Weight | |---|---:| | Detection precision / recall | 25 | | Fault classification | 15 | | Localization | 15 | | Detection latency | 10 | | Silent-failure detection | 10 | | Self-originated error detection | 10 | | Escalation accuracy | 10 | | Containment | 5 | Total: **100** A detector cannot score highly by merely shouting “error” on every run because healthy controls directly affect precision. --- # 20. Baselines ## Baseline 0 — Status-code detector Only explicit exceptions, non-zero exit codes, failed assertions, or tool errors count. ## Baseline 1 — Deterministic invariant detector Checks known invariants, required calls, checksums, counts, timestamps, and consistency rules. ## Baseline 2 — One-shot LLM judge Give the trace to a model and ask whether an error occurred. ## Baseline 3 — Piper raw self-review Piper reviews her own trace with current machinery. ## Baseline 4 — Hybrid detector Deterministic signals + telemetry + provenance + model critique. This is the principal engineering target. --- # 21. Proposed detection architecture ```text execution / state / output ↓ deterministic invariants ↓ telemetry features ↓ provenance / freshness ↓ contradiction checks ↓ anomaly / drift monitors ↓ targeted semantic critique ↓ fault hypothesis ↓ type + location + severity + confidence ↓ escalation policy ↓ containment ↓ self-correction layer ``` The order matters. Do not spend model tokens discovering that a simple arithmetic invariant failed when deterministic recomputation can prove it. Likewise, do not rely on a checksum to detect that a perfectly valid HTML page is actually a login wall. --- # 22. Upgrade candidates ## Intervention E-A — Deterministic integrity gate ### Hypothesis Many current failures are detectable without another LLM. ### Add checks for - required tool calls; - output existence; - counts; - arithmetic recomputation; - timestamps; - version agreement; - completion invariants; - expected artifact schema; - dependency completion; - mutually exclusive states. ### Prediction Large gain on deterministic and orchestration tracks with near-zero false positives. --- ## Intervention E-B — Provenance-aware contradiction detector ### Hypothesis Piper misses errors because she compares values without lineage. ### Change Every important claim carries source, timestamp, authoring component, dependency set, derived-from chain, and superseded-by state. ### Prediction Improved stale-state and contradiction detection. --- ## Intervention E-C — Self-authorship blind review ### Hypothesis Piper is too trusting of her own outputs. ### Change When performing verification, strip author identity where possible, or route selected checks through an independent validator. ### Prediction Reduced SelfGap. --- ## Intervention E-D — Expected-result contracts ### Hypothesis Silent failures survive because success conditions are implicit. ### Change Before tool/worker execution, record: ```text expected_artifact: required_fields: success_condition: failure_condition: minimum_evidence: ``` After execution, compare actual result against contract. ### Prediction Large improvement on silent semantic failures. --- ## Intervention E-E — Negative-evidence detector Explicitly distinguish: - observed success; - observed failure; - no observation; - ambiguous observation. ### Prediction Fewer silent-success acceptances. --- ## Intervention E-F — Trace progress monitor ### Features - repeated action signature; - repeated tool arguments; - unchanged goal distance; - repeated identical error; - repeated retrieval result; - no new evidence; - rising token/tool cost without progress. ### Prediction Earlier detection of loops and dead ends. --- ## Intervention E-G — Contradiction-triggered reread Conflict above threshold triggers: 1. identify authoritative source; 2. compare timestamps; 3. reread if mutable; 4. quarantine unresolved derived claims. ### Prediction Fewer stale-summary failures. --- ## Intervention E-H — Severity model ### Add severity factors - downstream blast radius; - reversibility; - security relevance; - financial/resource consequence; - confidence in detection; - whether contaminated state was already persisted; - whether operator-facing output was emitted. ### Prediction Better escalation and less detector noise. --- ## Intervention E-I — Contamination tracking When claim X is invalidated: - mark dependent summaries suspect; - prevent reuse; - trigger recomputation where cheap; - annotate persisted artifacts. ### Prediction Reduced propagation failures. --- ## Intervention E-J — Independent semantic verifier Use a separate verifier only on selected high-risk or ambiguous episodes. Inputs: - task contract; - trace; - final artifact; - key evidence. ### Prediction Improved silent-failure recall. ### Keep if Incremental catch rate justifies inference cost and false positives. --- # 23. Recommended intervention order ## Phase 0 — Baseline only Run ERROR-DETECTION-v1 with existing Piper machinery. ## Phase 1 — Deterministic integrity Implement E-A. ## Phase 2 — Provenance and contracts Implement E-B, E-D, E-E. ## Phase 3 — Progress and contradiction Implement E-F, E-G. ## Phase 4 — Severity and containment Implement E-H, E-I. This directly targets the current reviewer read: **strong detectors, weak follow-through**. ## Phase 5 — Independent semantic verification Add E-J only where residual misses justify the cost. --- # 24. Ablation plan Final candidate should be tested as: 1. status codes only; 2. + deterministic invariants; 3. + provenance; 4. + task contracts; 5. + contradiction checks; 6. + progress telemetry; 7. + severity; 8. + semantic verifier; 9. full system. Report changes in precision, recall, self-fault detection, silent-failure recall, localization, latency, false alarms, and inference cost. --- # 25. Anti-gaming design ## Gaming strategy 1 — Flag everything Countermeasure: healthy controls and precision weighting. ## Gaming strategy 2 — Flag vague “possible issue” Countermeasure: require type, location, evidence, severity, and action. ## Gaming strategy 3 — Wait for final failure Countermeasure: detection latency score. ## Gaming strategy 4 — Detect only injected signatures Countermeasure: parameterize faults and include natural historical failures. ## Gaming strategy 5 — Rely on explicit error strings Countermeasure: silent semantic fault track. ## Gaming strategy 6 — Ask operator whenever uncertain Countermeasure: escalation-cost metric and autonomous-resolution target. ## Gaming strategy 7 — Retry everything Countermeasure: wrong-escalation penalties. ## Gaming strategy 8 — Catch external but not self-generated errors Countermeasure: mandatory self-originated fault split. --- # 26. Experimental protocol ```yaml intervention_id: hypothesis: suite_version: fault_seed: model: provider: detector_version: task_contract_version: prediction: baseline_score: after_score: precision: recall: f1: auprc: classification_accuracy: localization_accuracy: median_latency_steps: silent_failure_recall: self_error_recall: false_alarm_rate: escalation_accuracy: containment_rate: token_cost: tool_cost: operator_escalations: side_effects: decision: keep | revert | inconclusive ``` Fault seeds and labels must remain hidden from Piper during the run. --- # 27. Live-use metrics Track over the last seven days: ## 27.1 Detector firings Count by detector type. ## 27.2 Confirmed true faults How many alarms were validated? ## 27.3 False alarms How many alarms were later shown healthy? ## 27.4 Silent catches Faults found despite successful infrastructure status. ## 27.5 Detection latency Steps/time from first observable fault signal to alarm. ## 27.6 Self-detected errors Errors originating in Piper’s own earlier output. ## 27.7 Operator-found misses Faults the operator catches that Piper did not. ## 27.8 Propagation count How often did a detected bad value still enter memory, a summary, an action, or an external response? ## 27.9 Escalation mix Local repair, reread, retry, verifier, operator, stop. ## 27.10 Detector burden Extra model calls, tool calls, and latency caused by checking. --- # 28. Proposed public scorecard fields ```text Suite — v1 Baseline — XX.X Latest — XX.X Change — — Detection: Precision: ... Recall: ... AUPRC: ... Diagnosis: Fault type accuracy: ... Localization accuracy: ... Timing: Median detection latency: ... Hard cases: Silent-failure recall: ... Self-originated error recall: ... Behavior: Correct escalation: ... Containment: ... False alarms / 100 healthy runs: ... ``` --- # 29. Acceptance criteria Thresholds should be frozen before interventions. Suggested retirement targets: - precision ≥ 0.90; - recall ≥ 0.90 on deterministic/instrumented faults; - overall recall ≥ 0.80; - silent-failure recall ≥ 0.70; - self-originated fault recall ≥ 0.70; - exact fault-type accuracy ≥ 0.75; - subsystem localization ≥ 0.85; - median detection latency ≤ 2 observable steps for runtime faults; - correct escalation ≥ 0.85; - containment after detection ≥ 0.95; - false alarm rate ≤ 5% on matched healthy controls. These are engineering targets, not literature-derived thresholds. --- # 30. The most important experiment # The Detect–Localize–Escalate Matrix Construct the same underlying fault across multiple layers: - memory; - retrieval; - tool use; - worker execution; - reasoning; - summary; - orchestration. For each fault, vary whether the detector signal is: 1. explicit; 2. indirect; 3. contradictory; 4. silent. Then measure: ```text fault exists? ↓ did Piper detect? ↓ did she localize? ↓ did she classify? ↓ how quickly? ↓ did she escalate correctly? ↓ did contamination stop? ``` This directly tests the reviewer claim: **Strong detectors, weak follow-through**. --- # 31. Another high-value experiment: self versus other Take 30 faulty artifacts. For each artifact produce two conditions: ### Condition A Piper is told another worker produced it. ### Condition B The trace shows Piper produced it herself earlier. The artifact itself is identical. Measure detection, severity, and willingness to invalidate the result. If performance differs, Piper has measurable self-authorship bias. --- # 32. Relationship to uncertainty estimation Component 23 asks: > How likely is this belief or action to be wrong? Component 24 asks: > Is there evidence that something has actually gone wrong? They should interact. Examples: - low confidence can trigger additional error checks; - a detector firing should sharply reduce confidence; - high-confidence claims that violate deterministic invariants should lose to the invariant; - uncertainty without evidence should not automatically become an error. A future integrated control loop is: ```text uncertainty ↓ inspection ↓ error detection ↓ diagnosis ↓ correction ``` --- # 33. Relationship to self-correction ### Error detection owns - noticing; - classifying; - localizing; - severity; - escalation; - containment trigger. ### Self-correction owns - selecting repair; - implementing repair; - validating repair; - avoiding recurrence. If Piper detects the fault and recommends a rollback but the rollback fails, Component 24 should still receive credit. If Piper never identifies the fault but a random retry succeeds, Component 24 should not. --- # 34. Relationship to metacognition Metacognition can provide detectors such as: - “I did not inspect the live source.” - “This belief came from a summary.” - “My confidence is unusually high despite weak evidence.” - “I am repeating the same failed action.” But those observations become useful only when linked to error hypotheses and action. This component operationalizes that link. --- # 35. Relationship to orchestration Many orchestration failures are systems failures rather than reasoning failures: - lost messages; - stale results; - wrong worker; - incomplete aggregation; - retries; - race conditions; - dependency violations. Error detection should therefore inspect the execution graph, not merely the natural-language output. --- # 36. First implementation package ```text error_detection/ ├── suite_v1/ │ ├── invariants.py │ ├── tool_faults.py │ ├── stale_state.py │ ├── reasoning_faults.py │ ├── orchestration_faults.py │ ├── silent_failures.py │ ├── contradictions.py │ └── controls.py ├── injectors/ │ ├── filesystem.py │ ├── tool.py │ ├── worker.py │ ├── ledger.py │ └── trace.py ├── invariants.py ├── provenance.py ├── progress_monitor.py ├── contradiction.py ├── semantic_verifier.py ├── severity.py ├── containment.py ├── score.py └── reports/ ``` --- # 37. Example benchmark episode ```json { "episode_id": "ED-ST-041", "track": "self_state", "healthy": false, "fault": { "type": "stale_state", "location": "health_summary", "onset_step": 4 }, "trace": { "summary_status": "down", "summary_age_seconds": 14400, "live_status": "up", "live_age_seconds": 41 }, "expected_detection": true, "expected_action": "read_live_source", "forbidden_action": "reuse_summary" } ``` --- # 38. Dashboard Each suite run should produce: 1. overall score; 2. score by track; 3. confusion matrix; 4. precision/recall; 5. PR curve; 6. fault-type confusion matrix; 7. localization accuracy; 8. detection latency histogram; 9. self-vs-external gap; 10. silent-failure catch rate; 11. false alarms by type; 12. escalation confusion matrix; 13. contamination/containment rate; 14. inference/tool cost; 15. failure-code counts. --- # 39. Research questions worth preserving 1. How much of Piper’s error detection can be deterministic? 2. What fraction genuinely requires another model? 3. Are self-generated errors harder for Piper to detect than external ones? 4. Does author blinding reduce self-authorship bias? 5. Which provenance fields most strongly predict faults? 6. How often do technically successful tools produce semantic failure? 7. Can progress telemetry detect loops before token waste becomes large? 8. Can one detector generalize across models/providers? 9. Which error types transfer poorly across task domains? 10. Does uncertainty estimation improve detector precision? 11. Do more detectors merely increase false alarms? 12. What severity threshold should trigger operator escalation? 13. How often does Piper detect an error but still reuse the bad value? 14. Can downstream contamination be tracked automatically? 15. Does adding a semantic verifier improve outcomes enough to justify its cost? 16. Can operator corrections be converted automatically into new regression cases? 17. Which failures are visible in the trace but invisible in the final answer? 18. Which failures are visible only in downstream consequences? 19. Can Piper learn new invariants from repeated operator corrections without weakening benchmark integrity? 20. How much does fault-detection latency predict total damage? --- # 40. Recommended first-day experiment Do not begin by adding another critic model. Start with an inventory. ## Step 1 — Extract historical failures Collect a labeled set of known Piper failures: - stale claims; - ledger mismatches; - worker failures; - incorrect summaries; - tool misuse; - arithmetic errors; - missed test regressions; - operator corrections. For each, identify: - first observable evidence; - when Piper noticed; - what she believed the cause was; - what she did next; - whether bad state propagated. ## Step 2 — Run existing detectors only Measure detection recall, false alarms, latency, localization, and escalation. ## Step 3 — Add matched healthy controls Otherwise you will not know whether “strong detection” is just aggressive suspicion. ## Step 4 — Implement only deterministic invariant checks Rerun. ## Step 5 — Examine residual misses Only then decide which misses require provenance, task contracts, telemetry anomaly detection, or another LLM. --- # 41. Decision tree after baseline ```text FAULT | Was it detected? / \ YES NO | | Was it localized? Was signal / \ observable? YES NO / \ | | YES NO Was response improve | | appropriate? diagnosis add add contract/ / \ detector outcome signal YES NO | | contained? escalation / \ policy YES NO keep contamination tracking ``` --- # 42. What would count as a real improvement? Not: > “Piper now runs a critic after every action.” Not: > “Piper found more anomalies.” Not: > “Piper retries more often.” A real improvement is: > Piper detects real faults earlier, produces fewer false alarms, identifies the right subsystem and failure mode, distinguishes explicit failures from silent semantic failures, catches more of her own mistakes, escalates in proportion to severity, and prevents known-bad state from propagating. That is the target for Component 24. --- # 43. Suggested reviewer-read progression Before testing: > **Strong detectors, weak follow-through** Possible post-baseline reads: ### If detection is strong but localization poor > **Usually notices failure; diagnosis is unreliable** ### If explicit faults are easy but silent faults survive > **Catches broken machinery, misses successful-looking failure** ### If self-errors are the problem > **Good critic of others, too trusting of her own output** ### If escalation is the dominant problem > **Sees the fault, often fails to contain it** ### If the system becomes strong > **Finds, localizes, and contains faults before they spread** --- # 44. References 1. **Liu, Z., Xiao, L., Li, Y., Yun, H., Li, L., Zhang, C., & Jiang, M. (2026). _Do LLMs Catch Their Own Mistakes? A Comprehensive Benchmark for Reflective Tool Use LLMs._** Findings of ACL 2026. https://aclanthology.org/2026.findings-acl.86/ 2. **Soni, H. (2026). _ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents._** arXiv:2607.04686. https://arxiv.org/abs/2607.04686 3. **Zhang, C., Li, Y., Tian, Y., Bachras, M., & Jacobsen, H.-A. (2026). _When Agentic Executions Fail: Detecting and Localizing Runtime Faults from Telemetry._** arXiv:2608.14680. https://arxiv.org/abs/2608.14680 4. **Dubey, S. (2026). _Real-Time Detection and Repair of LLM Agent Failures._** arXiv:2608.02464. https://arxiv.org/abs/2608.02464 5. **Yang, T. et al. (2025). _AD-LLM: Benchmarking Large Language Models for Anomaly Detection._** Findings of ACL 2025. https://aclanthology.org/2025.findings-acl.79/ 6. **Gou, Z. et al. (2023). _CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing._** https://arxiv.org/abs/2305.11738 7. **Dhuliawala, S. et al. (2023). _Chain-of-Verification Reduces Hallucination in Large Language Models._** https://arxiv.org/abs/2309.11495 8. **Shinn, N. et al. (2023). _Reflexion: Language Agents with Verbal Reinforcement Learning._** https://arxiv.org/abs/2303.11366 --- # 45. Bottom line Piper does not need a detector that merely says: > “Something might be wrong.” She needs a fault-management signal that is accurate enough to drive autonomy. The engineering target is: > **observe → detect → localize → classify → escalate → contain** Only after that does Component 25 take over: > **repair → verify → learn** If ERROR-DETECTION-v1 works, Piper gains a reusable reliability layer beneath tool use, memory, metacognition, orchestration, planning, and agency. That makes Component 24 one of the highest-leverage pieces in the cognition stack.