Summary / 08. Context construction / Document
Spec 2026-09-05
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Build Spec 08 — Context Construction (the ≈80% fraction) > Derived from `papers/08-context-construction.md`. This document drives the build of the > **context-construction stage**: the cross-cutting assembly engine that decides, for every model > invocation, what of the candidate pool the earlier components produced is actually *sent* to the > model — at what representation, in what order, and under what hard token budget. It sits after > component 07 (task decomposition & planning, which sequences the whole request into an executable > schedule of obligations) and before the components that apply, verify, and repair the model's > output. Where reconnaissance (03) says what the repository *is*, retrieval (04) what context is > *relevant* enough to enter the candidate pool, localization (05) *where the fault is*, impact > analysis (06) *what one edit will break*, and planning (07) *in what order to act* — context > construction (08) is the only stage that decides, for the *single next decision*, exactly what the > model sees: the observation. Retrieval decides what enters the candidate pool; context construction > decides the observation itself. It translates the paper's design into exact evidence-item schemas, > exact hard-gate admission rules, exact representation-policy tables, exact compression, dedup, > ordering, and token-budget mechanics, exact L-column caps, exact A/B instruments and thresholds, > exact ablation mechanics, and an exact build sequence — no further research required. Where the > paper says "≈80%", this spec pins *which* 80%. --- ## 1. Purpose & Scope Build a **reversible context-construction stage** — a deterministic-first, decision-specific, typed, aggressively selective, ephemeral assembly engine (evidence-pool assembly → hard relevance gate → representation choice → structural compression → deduplication → edge-anchored ordering → token budgeting) with a **single weak-LLM relevance-ranking decision**, as a drop-in *observation* stage that converts everything components 03–07 know — plus the current patch, live errors, tracebacks, and the latest tool/test output — into the **single `DecisionContext`** handed to the model for one decision before any further code is written or verified. It is distinct from its predecessors in *kind*, not merely in order: retrieval (04) optimizes *recall of what could matter*; context construction optimizes *the precision of what the model actually attends to*. Retrieval decides what enters the candidate pool; context construction decides which of those candidates, represented how, ordered how, against how large a budget, actually reaches the model's attention. The overall target is the paper's full 100% spec. **Round 1 lands the expected ≈80% of it**: everything deterministic (evidence-pool assembly, the hard relevance gate, representation choice, structural compression, deduplication, edge-anchored ordering, token budgeting) plus the *one* weak-LLM relevance-ranking call — targeting the reachable fraction of the corpus where the L column holds exactly one entry and everything upstream and downstream is deterministic software on CPU. The heavy learned / multi-call / trained-pair / unproven pieces (LLMLingua/LongLLMLingua perplexity-model compression, CompactionRL's RL-trained compactor, Scroll's lossless append-only Event Log with programmatic history querying, and the learned prompt predictor from Prompt Generation) are **not** built in Round 1 and are **not excluded** — they are the content of later rounds that climb from that ~80% landing toward 100% (see §2). Round 1 is kept only if it wins a measured A/B test on the *honest* context-construction metrics — led by **tokens-per-solved-instance** (the window economy) and **relevance precision** — *and* does not regress end-to-end resolve/cost/tokens (§7). --- ## 2. Round 1 vs later rounds (toward the 100% target) The goal is the paper's full **100%** spec. **~80%** is only the expected **first-round landing**; later rounds climb from there. Every element from the paper is classified below by *which round builds it*. - **BUILD-NOW = the Round 1 build** — the expected ≈80% deterministic landing this spec implements and A/B-tests (rows 1–7). - **DEFERRED = later rounds** — the learned / trained-pair / model-call-adding / unproven pieces (rows 8–11) that Round 1 intentionally does **not** build *yet*, because they add a model call to the pipeline against the L-column invariant, need a trained compactor (a model + harness pair with a single-context-transfer caveat), rest on unproven preprint techniques, or would supplant a deterministic stage that may already suffice. They are **not excluded from the project** — they are the content of later rounds, the climb toward 100%, gated by the same §7 A/B protocol once the Round 1 core is in place. | # | Component (paper §) | Classification | |---|---|---| | 1 | (a) Evidence-pool assembly — gather candidate material from 03/04/05/06/07 + the current patch, live errors, tracebacks, and latest tool/test output into a uniform addressable pool | **BUILD-NOW** | | 2 | (b) Hard relevance gate — obligation-driven + impact-DAG-driven admission only; everything else rejected without a model (P4 made structural) | **BUILD-NOW** | | 3 | (c) Representation choice — deterministic (evidence type × role) → rendering table (verbatim / signature / digest / truncated / omitted) | **BUILD-NOW** | | 4 | (d) Compression — structural first (signatures over bodies, AST slices, sliding-window contiguous-line extraction); deterministic masking under budget | **BUILD-NOW** | | 5 | (e) Deduplication — collapse duplicate references to one; drop mid-relevance filler | **BUILD-NOW** | | 6 | (f) Ordering — critical content pinned to the edges; nothing decision-critical mid-context (lost-in-the-middle) | **BUILD-NOW** | | 7 | (g) Token budgeting — hard ceiling + three tiers (stable core / working set / transient); evict transient first, account never overflow + the **single L relevance-ranking call** (see §4) on the hard gate's unresolved residue | **BUILD-NOW** | | 8 | LLMLingua / LongLLMLingua **perplexity-model compression** — score tokens/chunks for information content with a small perplexity model | **DEFERRED** — adds a model call to the pipeline against the L-column invariant; the Simple Observation Masking finding (§2.8) says deterministic masking may suffice in the weak-model regime; gated by §7 ablation, not a default | | 9 | CompactionRL's **RL-trained compactor** — jointly optimizes task execution and summary generation | **DEFERRED** — a model + harness *pair* (its own §5.2 caveat: gains do not consistently transfer to single-context settings, where a trained compactor can fall below its own base model); needs RL training; not a drop-in component | | 10 | Scroll's **lossless append-only Event Log** with programmatic history querying (grep/slice/compute over own history) | **DEFERRED** — one-week-old preprint; unproven; reframes the storage substrate of history rather than the observation; held loosely | | 11 | The learned **prompt predictor** from Repository-Level Prompt Generation (predict the best source×representation proposal) | **DEFERRED** — trained predictor reaches ~17% vs the 36% *oracle* upper bound over 63 proposals; the deterministic representation table (§3.3) already captures the source×representation decomposition at zero model cost | **Design decision: DETERMINISTIC-FIRST, LEARNED-EXCEPTIONAL.** Round 1 does not put a model anywhere in the observation path by default. It builds the full deterministic pipeline (rows 1–7) and the **single** relevance-ranking call (§4) that fires **only** when the hard gate (b) leaves genuinely ambiguous residue — candidates semantically related to the current decision but structurally unlinked by any obligation or impact edge. The deterministic path is the default; the L step is the exception. The perplexity-model compressor, the RL compactor, and the learned prompt predictor are all candidates for later rounds precisely because each would add a *second* model to the pipeline, break the one-entry L column, or replace a deterministic stage that the ablation (§7) may already prove sufficient. Any later-round candidate must clear the same bar as in earlier components: beat the deterministic stage on the honest metrics (§7), on held-out repositories, at a defensible cost, or be rejected. The cost of the deterministic choice is paid once, in the schemas (§3); the cost of a misplaced learned call is paid every decision, in L-column violations and distractor burden. ## 3. The Deterministic Pipeline (D) Seven stages, all **D** through the emission of the assembled `DecisionContext`. The LLM column holds exactly one thing: relevance ranking (§4), fired only when the hard gate (b) leaves unresolved residue. The pipeline: ``` CANDIDATE POOL (03 recon graph, 04 retrieved hits, 05 localization, 06 impact DAG, 07 plan node + obligations + acceptance criteria, current patch, live errors + tracebacks, latest tool/test output) → (a) evidence-pool assembly [D] → uniform addressable pool → (b) hard relevance gate [D] → admitted set (obligation- and impact-DAG-driven) → (§4) relevance ranking [L — the ONE entry, ONLY on the gate's unresolved residue] → (c) representation choice [D] → (d) compression [D] → (e) deduplication [D] → (f) ordering [D] → (g) token budgeting [D] → the single DecisionContext handed to the model ``` The window is rebuilt from authoritative evidence for **every** model decision — never emitted once at task start and accumulated across the run (§8 invariant 4). All artifacts land in `build/08-context-construction/`. Nothing writes into the live harness. ### 3.1 (a) Evidence-pool assembly (D) **Input:** every candidate the earlier components produced — source + symbols (03), retrieved hits (04), the fault/changed entity (05), graph edges + configs + downstream obligations (06), the current plan node + its obligations + acceptance criteria (07) — **plus** the live execution state: the current patch, live errors, tracebacks, and the most recent tool and test output. **Output:** a **uniform, addressable evidence pool**. **No language model runs here.** Assembly is deterministic collection and normalization, **not** selection — nothing is admitted to the window at this stage; this stage only makes everything available in a uniform, addressable form. **Evidence-item schema (exact):** ```json { "id": "E17", // unique within the pool, stable across the run "source": "04", // provenance origin: "03" | "04" | "05" | "06" | "07" | "patch" | "live" "kind": "source_code", // one of the kinds below "content": "…", // normalized text/JSON payload (bounded, see caps) "provenance": "04: retrieved by query 'SessionManager.load timeout', score 0.81" } ``` **Kinds (closed enum):** `source_code`, `symbol`, `retrieved_hit`, `fault_entity`, `changed_entity`, `graph_edge`, `config`, `downstream_obligation`, `plan_node`, `acceptance_criteria`, `patch`, `error`, `traceback`, `tool_output`, `test_output`. **Procedure (exact, deterministic):** 1. **Collect:** pull candidate material from every authoritative source in one pass (component 03's recon graph, component 04's retrieved hits, component 05's localization, component 06's impact DAG edges/configs/downstream obligations, component 07's current plan node + obligations + acceptance criteria, plus the current patch, live errors, tracebacks, and latest tool/test output). 2. **Normalize:** map each piece onto the schema above — assign an `id`, tag its `source` and `kind`, normalize its `content` into the shared textual/JSON form, and record `provenance` (where it came from and why it was produced, so later stages can attribute and the audit can trace it). 3. **Index:** every item is addressable by `id`; downstream stages refer to items by `id`, never by re-reading the source. 4. **No selection:** an item merely *enters the pool*; admission to the window is decided by stage (b), not here. **Caps:** pool ≤ **256** items per decision; each item's `content` ≤ **8 KB**; `provenance` ≤ **200** chars. Items over the pool cap are queued by source-priority (07 plan obligations first, 06 impact edges next, 04 hits next, 03 recon next, live state next) — but this is only a *pool* cap; actual admission is decided entirely by (b). The pool is rebuilt per decision; it is never accumulated across decisions (§8 invariant 4). ### 3.2 (b) Hard relevance gate (D) **Input:** the assembled pool (a). **Output:** the **admitted set** — the subset of the pool that is causally relevant to the *current* decision. **No language model runs here.** This is P4 executed structurally: the gate encodes "context must justify its presence for the current decision" in deterministic code, so no plausible-sounding but irrelevant file, rule, or overview can ride into the window on the intuition that it might help (paper §2.6). **Admission rule (exact — two independent drivers, disjunction):** An evidence item is **admitted** iff either driver reaches it: 1. **Obligation-driven** — the item is required to discharge the current plan node's obligations or acceptance criteria from component 07. Concretely: the item is the node's target entity, a dependency of the current obligation, a named acceptance-criterion predicate, or a named test / validation target that the obligation must satisfy. 2. **Impact-DAG-driven** — the item is a **direct consequence** the current change must satisfy: a direct edge in component 06's impact DAG whose head is the current change (a `CalledBy`, `ConfigConsumer`, `SchemaConsumer`, or `PreserveObligation` consequence of the entity being edited). **Rejection (exact):** an item that is **neither** required to discharge the current obligation **nor** a direct impact-DAG consequence of the current change — however plausible in the abstract — is **rejected here**, deterministically, **without invoking a model**. "Semantically similar to the task" is *not* an admission criterion at this gate; that is exactly the ambiguity the single L step (§4) is reserved for, and only for the residue the gate cannot resolve. **Inputs to the gate (exact, deterministic):** - the current plan node `{node_id, obligation, target_entity, preconditions, required_validation}` from 07 and its obligations + acceptance criteria; - the edges and consequences of 06's impact DAG reachable from the current change; - each pool item's `kind` + `content` + `provenance`. **Procedure (exact):** for each pool item, evaluate driver 1 (does an obligation/acceptance criterion reference this item's target?) then driver 2 (is this item a direct 06 consequence of the change?). Admit on either. **Emit** the admitted set as an ordered list of `id`s plus a **residual set** — the pool items the gate could *not* resolve either way (semantically near the decision but structurally unlinked). The residual set is passed to §4 **only if non-empty and ≤ 32 items**; a larger residue is folded into the nearest obligation bucket or rejected outright (§4 caps). **Caps:** the admitted set ≤ **64** items per decision. A pool that yields more than 64 admitted items is over-supplied by its sources; the gate rejects the lowest-priority admitted items deterministically by the same source-priority as (a) until the cap holds — never by asking a model to choose. Every admission and rejection is logged with its driver (or absence) for the §7 relevance-precision and ablation instruments. ### 3.3 (c) Representation choice (D) **Input:** each admitted item from (b) + its `kind` + its **role** in the current decision. **Output:** each admitted item assigned a **rendering** by deterministic policy. **No language model runs here.** This is P2 + P3 executed: the same retrieved entity shown as a full body or a signature produces a different decision, so representation is as decisive as selection — and the *cheapest* form that preserves the information this decision requires is the default, with expansion the gated exception (paper §2.2, §3 P3). **The representation-policy table (exact — evidence `kind` × decision `role` → rendering).** The rendering is one of `verbatim` | `signature` | `digest` | `truncated` | `omitted`: | Evidence kind | Role in this decision | Rendering | |---|---|---| | change target (fault/changed entity, 05/06) | the code being edited | **verbatim** | | insertion-point code | the code immediately before/at the edit site | **verbatim** (the strongest edit-conditioning signal, §3.6) | | requirement + acceptance criteria (07) | the goal the current obligation serves | **verbatim** | | related symbols (03 recon, 04 hits) | surrounding / referenced / similar | **signature** (name, params, return, one-line doc) — never body unless the decision must reason over the body | | impact neighborhood (06 direct edges) | direct consequences of the change | **signatures** + **only the call sites of direct obligations** (never full consumer bodies) | | wider impact DAG (06 transitive) | downstream reach of the change | **structured digest** — a fixed one-line form, e.g. `"11 callers, 14 config/schema consumers, 22 indirect dependents, 15 tests — no direct evidence requires source expansion yet"` | | a related test (07 validation / 04 hit) | must the decision reason over the test body? | **body if reasoned over, else its name + one-line expectation** | | config / schema consumer (06) | does the change alter a consumed shape? | **signature** of the consumed schema + the consumer binding line | | patch / error / traceback / test output (live) | the current failing state | **verbatim** (freshly relevant, short-lived) | | stale or superseded item (live history) | already-falsified / old version | **omitted** (P6 — dropped before ordering) | **Procedure (exact, deterministic):** each admitted item's `kind` and role index the table above. The rendering assigned is fixed; the table is code, not prose. **Expansion is a gated exception, never a default:** an item assigned `signature` or `digest` may be escalated to `verbatim` only when the decision genuinely requires the body — the gate is a deterministic predicate (the decision's `required_validation` names a test whose body must be reasoned over, or the obligation's target is the entity itself), never a model's judgment that more text might help. **Caps:** the representation table is a fixed mapping — no free-form choice. An item whose `kind` × role pair is not in the table defaults to `signature` (cheapest safe) and is logged as a table-miss for the §7 audit. ### 3.4 (d) Compression (D) **Input:** the admitted, representation-assigned items from (c). **Output:** each large item reduced to the smallest span that preserves the information the current decision needs. **No language model runs here.** The perplexity-model compression of the LLMLingua lineage is **deferred** (§2 row 8): the Simple Observation Masking finding — that a deterministic masking strategy matches an LLM summarizer while halving cost (paper §2.8) — says deterministic masking and structural compression may suffice in the weak-model regime, and a learned compressor would add a model call against the L-column invariant. **Procedure (exact — structural first, then deterministic masking only under budget):** 1. **Signatures over bodies:** any item assigned a body rendering but whose *content* is a whole function/class is replaced by its signature unless the decision must reason over the body (§3.3's gated exception). 2. **AST slices:** where the decision touches a specific span (a method, a branch, a block), slice the AST to that span (component 03 tree-sitter / stdlib `ast`) instead of inserting the enclosing entity. 3. **Sliding-window contiguous-line extraction (RepoCoder's real mechanism, paper §2.3):** for a large file/entity the decision needs only part of, a sliding window traverses the entity and extracts the most relevant contiguous *lines* that fit the remaining budget, scored by token overlap with the current decision's terms. This is the deterministic analogue of the DEFERRED perplexity scorer — it uses the decision's own obligation/acceptance terms, not a learned model. 4. **Deterministic masking (under budget only, §3.7):** when the window is over its hard ceiling, older / lower-signal environment observations are replaced by compact pointers (e.g. `"tool output offloaded to scratch — <15-token pointer>"`) in the style of the observation-masking result — never by an LLM summary, and never losing the raw ground truth if it must remain recoverable (§8). **Caps:** a body is compressed only down to its smallest decision-relevant span; an item that cannot be reduced below **1 KB** while preserving the decision's required predicate is flagged for the budgeter (g), not silently dropped. Compression that would require a model (perplexity-based) is **deferred, never defaulted** (§8 invariant 8). ### 3.5 (e) Deduplication (D) **Input:** the admitted, compressed items from (d). **Output:** a **deduplicated** working set where each distinct fact appears once. **No language model runs here.** This is P4 at the token level: a window cluttered with near-identical renderings of one fact is paying distractor cost for no information. **Duplicate classes collapsed to a single reference (exact):** 1. **Same definition via two paths** — the same symbol reached through 03 recon and 04 retrieval is rendered once; the second occurrence is a pointer to the first `id`. 2. **Same failure reported twice** — the same live error/traceback surfaced in two sources (e.g. both a test output and a log line) is rendered once. 3. **Same symbol in target + related fields** — a symbol that is both the change target and appears in a related-symbol list is rendered in its target role only. 4. **Same evidence under multiple obligations** — an item that discharges several obligations is rendered once and pointed to by each obligation. **Procedure (exact):** dedup keys on normalized content identity (definition site, failure text, symbol identity), not on `id`. Each distinct fact is kept exactly once; every duplicate is replaced by a one-line pointer to the surviving `id`. **Mid-relevance filler is dropped** — items that survived (b) but are only weakly attached (score in the gate's lower tier, or rendered `signature`/`digest` because they are context rather than load-bearing) are removed when they do not change any decision the item in question depends on. **Caps:** ≤ **1** rendering per distinct fact; dedup runs before ordering (f) and before budgeting (g) so the budgeter measures the *deduplicated* window, never pays twice for one fact. ### 3.6 (f) Ordering (D) **Input:** the deduplicated working set (e). **Output:** the items **placed** in the window under P5. **No language model runs here.** Ordering is a free correctness lever: lost-in-the-middle says performance is highest when the relevant content sits at the edges and collapses when forced mid-context, even in a window that technically fits everything (paper §2.5, §3 P5). LongLLMLingua's document reordering operationalizes this into measurable gains (paper §2.7). **Placement rule (exact):** 1. **Edges (highest attention):** the **insertion-point neighborhood** (the code immediately before and at the edit site — the strongest conditioning signal for an edit decision, kept **adjacent to the edit target**), the **freshly failing test output**, and the **critical definitions** the current edit depends on. These are pinned to the start and/or end of the window. 2. **Middle (supporting):** signatures, digests, and related-but-not-load-bearing material fill the middle. 3. **Nothing decision-critical mid-context:** no item the decision depends on (the change target, the failing assertion, the acceptance criterion, the key definition) is allowed to sit in the middle span where lost-in-the-middle would bury it. **Procedure (exact):** items are tagged `edge-critical` or `supporting` by the same deterministic role used in (c). The budgeter (g) allocates the window from the edges inward: `edge-critical` items fill the front block, then the back block (so the freshest failure output sits at the very end, the position of highest recency attention), then `supporting` material fills the middle only after both edge blocks are placed. The rule is load-bearing, and the §7 lost-in-the-middle probe verifies it — deliberately burying critical content mid-context must degrade the result, or the ordering rule is not earning its place. ### 3.7 (g) Token budgeting (D) **Input:** the ordered working set from (f). **Output:** the assembled **`DecisionContext`** — measured against a **hard ceiling**, never overflowed by accumulation. **No language model runs here.** This is the economy of the paper's §1 made explicit: instructions, code, and tool output compete for the same finite tokens (tool observations routinely consume 70–80% of the budget, paper §2.8), and the constructor — not the model — decides who wins. **Allocation across three tiers (exact):** - **Stable core** — the task, its acceptance criteria, and the current plan obligation. **Never evicted mid-decision.** - **Working set** — the current file and the needed definitions (the verbatim change target, insertion-point code, and critical definitions of §3.6). - **Transient state** — the last tool output and the current errors. **Hard ceiling:** the assembled window must stay under **`TOKEN_CEILING` = 24,000 tokens** for Round 1 (the harness's own weak model window with headroom; the ceiling is a build constant, configurable only via the build's config file, never changed by the constructor itself). The ceiling is measured after (d) and (e), on the *deduplicated compressed* window — the constructor **accounts, never overflows**. **Eviction ladder (exact, under the ceiling):** 1. **Evict transient state first** — collapse older tool output to compact pointers (§3.4 masking), keeping only the freshest failure output at the edge. 2. **Then compress the working set by representation** — escalate a `verbatim` body to `signature` (§3.3) where the decision can survive without the body; widen AST slices only if the predicate still holds. 3. **As a last resort, trim the stable core** — and only by truncating *supporting* acceptance detail, never the task goal or the current obligation (the two items P1 cannot drop). **Procedure (exact):** order the tiers by eviction priority; if the assembled window exceeds the ceiling after (a)–(f), apply ladder steps 1 → 2 → 3, re-measuring after each, until the window fits. If the window still exceeds the ceiling after all three steps (an irreducibly large stable core — a pathological request), the constructor emits the stable core **only** and flags the run: it never emits an over-budget window. **Emit:** the single `DecisionContext` = `{ task, acceptance_criteria, current_obligation, window: [ordered items], token_count, ceiling, budget_log }`. `token_count` ≤ `TOKEN_CEILING` always. `budget_log` records the eviction steps taken (for the §7 token-budget-adherence instrument). The window is **ephemeral**: it is rebuilt for the next decision and never persisted as accumulated context (§8 invariant 4, invariant 6). ## 4. The LLM Step (L, ONE entry) The **single** learned node in the entire stage: **relevance ranking**. The weak model ranks the candidates the deterministic hard gate (b) could not resolve — the genuinely ambiguous residue — for the *current decision*. It emits, per candidate, an identifier and a relevance score, as schema-constrained JSON in **ONE call at temperature zero**, scoped strictly to the unresolved residue. This is the sole entry in the L column; every upstream and downstream step is deterministic software on CPU (paper §4). The deterministic path is the default; this step fires **only** when the hard gate (b) leaves genuinely ambiguous material — candidates semantically related to the current decision but structurally unlinked, so no obligation or impact edge reaches them and the constructor cannot tell deterministically whether they are signal or distractor. It is the *exceptional* entry, never a planner that runs continuously over the whole run. **Firing condition (exact):** the step fires **iff** the hard gate (b) leaves a non-empty residual set of unresolved candidates **and** that set is ≤ **32** items. A larger residue — more than 32 unresolved candidates — is **not** ranked: it is folded into the nearest obligation bucket (each residual item is assigned to the obligation it is semantically nearest and admitted/rejected under that obligation's driver) **or, if it cannot be folded, rejected outright**. The L column is never invoked to adjudicate an oversized residue. **Prompt inputs (assembled deterministically, in this exact order):** 1. **`current_obligation`** — the current plan node's obligation + acceptance criteria (07). 2. **`decision_question`** — the one-line question this decision answers (from the plan node). 3. **`residue`** — each unresolved candidate as `{ id, kind, signature }` (signatures only, **never bodies** — the L step ranks, it does not read code). **Input → output schema (exact):** ```json { "rankings": [ { "candidate_id": "E31", "relevance_score": 0.82 }, { "candidate_id": "E44", "relevance_score": 0.17 } ] } ``` **Ranking record schema (exact — the model must emit nothing outside it):** ```json { "candidate_id": str, // must reference a residual candidate id from (b) "relevance_score": number } // in [0,1]; higher = more relevant to the current decision ``` **Caps:** **ONE call**, `temperature = 0`, `max_completion_tokens = 1024`, ≤ **32** candidates in the residue (the firing-condition cap). `temperature = 0` makes the single call deterministic in practice so re-ranking results are comparable. The output is schema-constrained; the harness, not the model, decides what the scores mean. **Admission decision (exact):** a residual candidate is admitted to the window **iff** the harness applies a deterministic threshold to the model's score: `relevance_score ≥ 0.5` → admit to (c); below → reject. The threshold is a build constant (configurable via the build config, never a model output). The score itself never bypasses the representation table (c) or the budgeter (g) — a ranked candidate still travels the full deterministic path. **Parse-failure fallback (exact):** any JSON parse failure, schema violation, out-of-range score, or >32-candidate output is a **no-op**: the stage **rejects all unresolved residual candidates** and hands control back to the deterministic path — **never a re-prompt, never a second call**. The L column is reached **once** per decision; a failed parse is not retried with a "please try again" call. This is the mechanical enforcement of the one-entry L column and the anti-over-prompting rule. On a failed parse, the residual candidates are treated as rejected (deterministic fallback — the constructor defaults to the frugal window, never to stuffing). **Re-invocation on the next decision (exact):** relevance ranking is re-considered **only** when the *next* decision produces its own residue; each invocation is scoped strictly to that decision's unresolved residue and re-uses none of a prior decision's ranking. It remains the **single conceptual learned entry** — a bounded, once-per-decision, per-residue ranking — **never a model that runs over the whole run** and never a loop that re-derives what the deterministic gate already settled (paper §4, §3.2). **Invariant 3 (§8): the L column holds exactly one entry.** ## 5. Implementation Order Eight phases, each independently testable and reversible. Each lands in `build/08-context-construction/` as a standalone importable module. Rollback = delete/flag-off that phase's module; nothing earlier breaks because phases only add modules and never mutate live harness code. | Phase | Delivers | Isolated test | Rollback | |---|---|---|---| | **1. Evidence-pool assembly** | `assemble.py` (§3.1 collection + normalization + indexing; pulls from 03/04/05/06/07 + patch/live/tool/test state) | Feed a fixture of recon/retrieval/localization/impact/plan outputs + a patch + a live error: every piece lands as a schema-valid `{id,source,kind,content,provenance}` item, `kind` ∈ the closed enum, caps enforced (≤256 pool, ≤8 KB content, ≤200-char provenance); a >cap pool queues by source-priority; assert **no selection** (nothing leaves the pool) and **no LLM consulted** (probe the LLM client to raise). | Delete `assemble.py` (pure read). | | **2. Hard relevance gate** | `gate.py` (§3.2 obligation- and impact-DAG-driven admission + rejection + residual-set emission) | Fixture pool + plan node + 06 edges: an item required to discharge the current obligation is admitted; a direct 06 consequence is admitted; a plausible-but-unlinked overview/rule is rejected **without a model** (assert zero LLM calls); the residual set (structurally unlinked but semantically near) is emitted; a >64 admitted pool is capped by source-priority; every admission/rejection is logged with its driver. | Delete `gate.py`. | | **3. Relevance ranking (L)** | `rank.py` (§4 firing condition + ONE call + output schema + ≥0.5 admission threshold + parse-failure fallback) | Mock the LLM client: assert the step fires **only** on a non-empty ≤32 residue; a >32 residue folds into obligation buckets or rejects (no call); the call is **one**, `temperature=0`, `max_completion_tokens=1024`, prompt carries obligation + question + residue signatures but **no code blobs**; output schema validated; a malformed / out-of-range / >32-candidate response → **all residual rejected** with **no second call**; a scored ≥0.5 candidate is admitted to the deterministic path. | Delete `rank.py` (no live-harness change). | | **4. Representation choice** | `render.py` (§3.3 the (evidence kind × role) → rendering table + gated-expansion predicate) | Feed every (kind × role) pair in the table: assert the exact rendering assigned (change target + insertion point + requirement/acceptance = verbatim; related symbols = signature; wider impact DAG = the structured one-line digest with the caller/config/test counts; a related test = body only when the decision reasons over it, else name); an out-of-table pair defaults to signature and is logged as a miss; expansion fires only on the deterministic predicate. | Delete `render.py`. | | **5. Structural compression** | `compress.py` (§3.4 signatures over bodies + AST slices + sliding-window contiguous-line extraction + deterministic masking) | Feed a large file + a decision term: a whole-function item is signatured unless reasoned over; an AST slice extracts only the decision-relevant span; the sliding window extracts the contiguous lines with best token overlap to the decision terms; under budget pressure older output is replaced by a compact pointer (never an LLM summary); an item that cannot be reduced below 1 KB while preserving the predicate is flagged for the budgeter, not silently dropped. | Delete `compress.py`. | | **6. Deduplication** | `dedup.py` (§3.5 collapse same-definition/same-failure/same-symbol/same-evidence duplicates to one reference + drop mid-relevance filler) | Fixture: the same symbol via 03 and 04 renders once; the same failure via test output and log renders once; a symbol in both target and related roles renders once in its target role; one fact under two obligations is rendered once and pointed to; dedup runs before ordering and budgeting (assert the budgeter measures the deduplicated window). | Delete `dedup.py`. | | **7. Ordering** | `order.py` (§3.6 edge-anchored placement: edge-critical front + back blocks, supporting middle, nothing decision-critical mid-context) | Fixture working set: the insertion-point neighborhood, freshly failing test output, and critical definitions land at the edges; supporting material fills the middle; **no** decision-critical item sits mid-context; the code immediately before the edit point stays adjacent to the edit target. | Delete `order.py`. | | **8. Token budgeting + A/B harness** | `budget.py` (§3.7 three tiers + hard 24k ceiling + eviction ladder + `DecisionContext` emit + budget_log), `ab.py` (feature-flag router + tokens-per-solved capture + relevance-precision capture + ablation runner + lost-in-the-middle probe + decision) | End-to-end on 3 warm-up tasks: `budget.py` never emits a window over 24k tokens (assert `token_count ≤ ceiling` always); transient evicts first, working-set compress by representation second, stable-core trim last; `ab.py` runs constructed-vs-baseline on the §7 sample, emits tokens-per-solved, relevance precision, resolve/cost/tokens, the ablation report, the lost-in-the-middle probe, and the §7 decision. | Feature flag `off` → live harness untouched; `ab.py` is read-only. | The A/B runner (Phase 8) is the only phase that ever *invokes* the current harness — and only to measure it, never to modify it. ## 6. Dependencies & Supply-Chain Per Piper's rule: nothing is installed unless >72h old AND (≥10 stars OR deep-scrutinized). This build is **zero-install for Round 1 (Python-only, stdlib + in-repo)**: | Dependency | Why | Status / age-stars | Install needed? | |---|---|---|---| | stdlib `json`, `re`, `sqlite3` | evidence-item schemas (§3.1), normalization, dedup identity keys (§3.5) | CPython stdlib. | **No** | | stdlib `ast` / component 03 tree-sitter | AST slices (§3.4), structural grounding of `target_entity` references | Reuse component 03's `recon.db` (already built/flagged there); no new parser. | **No** | | Component 03 recon graph (`recon.db`) | §3.1 pool source, §3.3 related-symbol signatures | In-repo, built by spec 03. | **No** | | Component 04 retrieved hits | §3.1 pool source (the candidate pool this stage narrows) | In-repo, built by spec 04. | **No** | | Component 05 localization | §3.1 fault/changed-entity pool source | In-repo, built by spec 05. | **No** | | Component 06 impact DAG + configs + downstream obligations | §3.2 impact-DAG-driven admission, §3.3 impact rendering | In-repo, built by spec 06. | **No** | | Component 07 plan node + obligations + acceptance criteria | §3.2 obligation-driven admission, §3.3 verbatim requirement, §3.7 stable core | In-repo, built by spec 07. | **No** | | Component 01 tool layer (read/edit/test output capture) | §3.1 live patch/error/traceback/tool-output pool sources | In-repo, already vetted. | **No** | | Harness weak model (`deepseek-v4-pro`) | §4 the single relevance-ranking call | The harness's own model, already vetted. | **No** | **Preferred path: zero-install.** Round 1 is Python-only and deterministic: the constructor reuses component 03's recon graph, component 04's retrieved hits, component 05's localization, component 06's impact DAG, component 07's plan node + obligations, and component 01's live tool/test state; the single LLM call (§4) goes to the harness's own weak model. **No new package is installed for Round 1, and nothing is flagged for a later install** — the four DEFERRED rows (§2 rows 8–11: LLMLingua/ LongLLMLingua perplexity-model compression, CompactionRL's RL-trained compactor, Scroll's Event Log, and the learned prompt predictor) are code and/or training on existing weights, not new third-party installs; any future model download or GPU dependency they imply must clear the 72h + star/scrutiny + sandbox-test gate before use, and is out of Round 1 scope entirely. ## 7. Acceptance Criteria — the A/B Protocol **Sample (concrete).** Reuse the **same 20 held-out SWE-bench-Verified instances as spec 01** (Python-only, single-repo, no-Docker reproduction). Held out = never seen by any tuning, prompt, or ranking pass. Seeded RNG. **Fallback:** `coding-cortex/piperbench` sampled to 20 tasks, held out identically. **Baseline (concrete).** The **"stuff everything"** baseline — the fused candidate pool (everything components 03–07 + live state produced) sent to the model with **no constructor at all**: no hard gate, no representation choice, no dedup, no ordering, no budget — the raw accumulation a harness without component 08 would send. The constructed window must beat *this* on tokens-per-solved and relevance precision, and the ablation (7.4) removes constructor mechanisms against the *honest* metrics below. ### 7.1 (a) Tokens-per-solved-instance — the KEY measure (the window economy) The key context-construction instrument, inherited from the paper's §6: **instances solved ÷ total context tokens consumed** across the run — the window economy. Every constructed window must respect the hard ceiling of §3.7, and the run must be scored on solve rate per token of context actually sent. **The constructed window must STRICTLY BEAT the "stuff everything" baseline on tokens-per-solved with NO solve-rate regression.** This is the instrument that makes P1–P3 falsifiable: if sending the full candidate pool costs the same per solved instance as sending the frugal window, the constructor is not earning its place, regardless of how elegant its principles read on paper. **This is the KEY measure — a relevance-precision or resolve win that does not beat the baseline on tokens-per-solved is not credited.** ### 7.2 (b) Relevance precision — the honest gate score The **fraction of the context actually sent that was causally needed** for the decision — required to discharge the current obligation, satisfy an impact edge, or answer the decision's question. This is the direct score of the hard relevance gate (b) and of P4: a window full of plausible-but-unneeded material scores low even if the resolve eventually succeeds, because it succeeded *despite* the distractor burden, not because of it. It is measured against the human- or test-verified set of facts each decision truly depended on. **The constructed window must STRICTLY IMPROVE relevance precision** versus the "stuff everything" baseline (which, by sending everything, scores near-zero precision by construction — it pays distractor cost for every irrelevant item it carries). ### 7.3 (c) End-to-end regression guard The coarse instrument kept for parity with the program's other components — **deliberately not** the gate for constructor changes, because resolve rate cannot separate context quality from execution quality: 1. **Resolve rate** = `resolved / total` (spec 01's score). 2. **Cost** = `$ / instance` (model spend only; deterministic stages are ~free). 3. **Tokens** = `tokens / instance`. ### 7.4 Decision rule (explicit thresholds) Run candidate-vs-current on the *identical* sample with the *identical* model (the harness's own weak model, `deepseek-v4-pro`) and hardware (this box). The candidate **wins iff**: ``` Tokens-per-solved STRICTLY beats the "stuff everything" baseline (the KEY measure) AND relevance precision STRICTLY improves AND resolve_candidate ≥ resolve_current (no solve-rate regression) AND cost_candidate ≤ 1.5 × cost_current AND tokens_candidate ≤ 1.5 × tokens_current ``` Tokens-per-solved must **strictly beat the baseline** and relevance precision must **strictly improve**; resolve must not regress; cost and tokens are bounded at 1.5×. A candidate that improves resolve but leaves tokens-per-solved flat against the baseline is **not adopted**. If the candidate wins, it replaces the current path (no-constructor stuffing); if it loses, the build tree is kept for the record and the live harness is unchanged. ### 7.5 Ablation rule (mechanical) The instrument that enforces P4 and makes the AGENTS.md lesson [6] mechanical — **never add context because it sounds sensible; prove it earns its tokens.** Two passes, everything else held fixed, the delta measured on the honest metrics (tokens-per-solved and relevance precision, plus resolve/cost/tokens): 1. **Remove each context source** in turn — the impact-DAG digest, the retrieved-code field, the history field, the constraints field, the plan obligation. 2. **Remove each representation choice** in turn — verbatim vs signature vs digest. A source or choice is **DELETED** iff its removal does **not** degrade the honest metrics beyond noise (`Δ = 0`, or `|Δ| ≤ 5%` where continuous). A source whose *addition* degrades the metrics is a demonstrated harm and is removed outright. Applied first to the weakest a-priori case (the retrieved history field), then to any other source/choice — regardless of how sensible it looks. ### 7.6 Lost-in-the-middle probe (P5 load-bearing check) The instrument that verifies §3.6's ordering rule is load-bearing, not decorative: 1. **Verify placement:** confirm that critical content — the insertion-point neighborhood, the failing-test output, and the critical definitions — is actually placed at the edges of the constructed window. 2. **Verify the ordering rule matters:** deliberately **bury** that critical content mid-context and re-measure. The result must **degrade** (worse on the honest metrics). If burying critical content does not degrade the result, the ordering rule is not earning its place and the ordering mechanism (f) fails the ablation — attention is not being directed by placement in this regime. --- ## 8. Guardrails & Reversibility **Revocability.** - The build lives entirely in `build/08-context-construction/` (a separate tree); it never writes into the live harness's source. - A **feature flag** (env var `CTX=ctx08|current`, default `current`) routes the A/B; the flag alone switches between the new context-construction stage and the current "stuff-everything" path — no code deletion required to revert. - `ab.py` is a read-only driver: it *measures* the current harness, it never modifies it. Running old-vs-new is always possible in the same session, same repo, same model. **Invariants (must hold, checked at the end of every phase).** 1. **No new `.md` files outside this folder** (`projects/self-improvement/{papers,specs,build}`). Everything else is code or DB rows. 2. **No change to the live harness** until the §7 A/B gate passes. The current tool surface stays byte-identical during the entire build. 3. **The L column holds exactly ONE entry** — relevance ranking (§4), fired only on the deterministic gate's unresolved residue, never re-prompted on parse failure. Any phase that introduces a language model elsewhere in the pipeline (assembly, hard gate, representation, compression, dedup, ordering, budgeting) is rejected outright. 4. **The window is rebuilt for every decision** — never emitted once at task start and accumulated across the run. The pool (a), the gate (b), and the budgeter (g) all run fresh per decision; no constructed window is appended to a prior one (paper §1, P1). 5. **No context earns permanent admission (distractor hygiene)** — repository instructions, history, and retrieved code are admitted only when they justify their presence for the *current* decision; a source or representation choice that fails the §7.5 ablation is **deleted**, never kept because it sounds sensible. 6. **Critical content is pinned to the edges** — nothing decision-critical is allowed mid-context. The insertion-point neighborhood, failing-test output, and critical definitions sit at the edges; any item the decision depends on that drifts to the middle is a rejected deliverable. 7. **A hard token budget is enforced by accounting, never overflowed by accumulation.** Every window is measured against the `TOKEN_CEILING` (24k) and evicted via the §3.7 ladder; no window over the ceiling is ever emitted, and no sequence of decisions accumulates past it. 8. **Deterministic stages stay deterministic** — compression that requires a model (perplexity-based) is **deferred, not defaulted**; the deterministic structural + masking path (§3.4) is the only compression in Round 1, and any later learned compressor must clear the §7 bar on held-out repositories at a defensible cost before it may enter the pipeline. --- *End of spec. This document is the build contract; the next document records the A/B outcome.*