Summary / 04. Code and context retrieval / Document
Spec 2026-09-04
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Build Spec 04 — Code/Context Retrieval (the ≈80% fraction) > Derived from `papers/04-code-context-retrieval.md`. This document drives the build of the > **code/context-retrieval stage**: the continuous, task-driven pulling of the smallest > high-precision slice of the repository *during execution*. It translates the paper's design into > exact extraction regexes, an exact ripgrep invocation, exact merge/rank/prune thresholds, exact > token caps, an exact vetting rule, and an exact build sequence — no further research required. > Where the paper says "≈80%", this spec pins *which* 80%. --- ## 1. Purpose & Scope Build a **reversible code/context-retrieval mechanism** — a deterministic, multi-signal pipeline (ripgrep lexical recall → symbol/API-description match → call-graph neighbor expansion → structure-aware dedup/merge → identifier-weighted BM25/IDF re-rank → goal-scoped line-level prune → hard token cap → vetting gate) that, at **every decision point during execution**, hands the model a curated working set of ≤4,000 tokens instead of a repository. It is the *execution-time pulling* counterpart to component 03's once-per-task reconnaissance *map*: 03 built the manifest and line-level graph and knows what the repo is; 04 spends that map at each step to decide what the model needs to see *right now*. The overall target is the paper's full 100% spec. **Round 1 lands the expected ≈80% of it**: everything deterministic (lexical recall, the symbol/API-description index, dedup, the BM25/IDF re-rank, the goal-scoped line prune, the token cap, and the vetting rule), targeting the reachable fraction of the change surface — named symbols, error strings, and the APIs the code must call. The statement-level CFG/CDG/DDG dependence re-rank, the trained CoRet encoder, the 0.6B neural SWE-Pruner skimmer, and AllianceCoder's LLM-decomposed query are **not** built in Round 1 and are **not excluded** — they are the content of later rounds that climb from that ~80% landing toward 100% (see §2). Round 1 is kept only if it wins a measured A/B test on the retrieval metrics *and* does not regress end-to-end resolve/cost/tokens (§7). --- ## 2. Round 1 vs later rounds (toward the 100% target) The goal is the paper's full **100%** spec. **~80%** is only the expected **first-round landing**; later rounds climb from there. Every element from the paper is classified below by *which round builds it*. - **BUILD-NOW = the Round 1 build** — the expected ≈80% deterministic landing this spec implements and A/B-tests (rows 1–7). - **DEFERRED = later rounds** — the deep learned/structural pieces (rows 8–11) that Round 1 intentionally does not build *yet*, because they need training data, weight access, or the statement-level graph that Round 1 deliberately postpones. They are **not excluded from the project** — they are the content of later rounds, the climb toward 100%, gated by the same §7 A/B protocol once the Round 1 core is in place. | # | Component (paper §) | Classification | |---|---|---| | 1 | ripgrep lexical recall (zero-index, fixed-string, never stale) | **BUILD-NOW** | | 2 | Symbol/API-description index + description↔description matching (AllianceCoder's load-bearing signal) | **BUILD-NOW** | | 3 | Structure-aware dedup/merge (collapse by structural identity, merge by containment) | **BUILD-NOW** | | 4 | Identifier-weighted BM25/IDF re-rank (symbols weighted over prose, top-k=20) | **BUILD-NOW** | | 5 | Goal-scoped line-level pruning (deterministic relevance score, never break syntax) | **BUILD-NOW** | | 6 | Hard token cap (≤4K tokens, enforced in code before the model runs) | **BUILD-NOW** | | 7 | The vetting rule (retrieved content is untrusted data; verbatim reproduction flagged) | **BUILD-NOW** | | 8 | Full statement-level CFG/CDG/DDG program-dependence graph re-rank (GraphCoder's −19.82%-if-removed signal) | **DEFERRED** — Round 1 uses the `invoke`/`import`/`contain` edges already built in component #3's `recon.db` | | 9 | The trained CoRet encoder (contrastive "what-must-be-inspected" neural retriever) | **DEFERRED** — needs training | | 10 | The 0.6B neural SWE-Pruner skimmer (the learned line-keep decision) | **DEFERRED** — Round 1 re-implements the line-keep *decision* deterministically | | 11 | AllianceCoder's LLM-decomposed query (model-paraphrased retrieval key) | **DEFERRED** — Round 1 builds the query by deterministic regex/symbol extraction | **Vetting-rule decision: BUILD-NOW-DETERMINISTIC** (a Round 1 build item). The paper's vetting rule is already non-learned — a harness-side check that flags verbatim reproduction of retrieved content. There is nothing learned to defer. The only *learned* node in the whole stage is the final consumption of the curated slice (the model reasoning over it); retrieval itself is 100% deterministic. --- ## 3. The Deterministic Pipeline (D) One pipeline, seven stages, all **D** through the production of the minimal context set. The LLM column holds exactly one thing: reasoning over the curated slice (§3.7's consuming step). The pipeline is: ``` TASK → (a) query/goal construction → (b) multi-source recall → (c) dedup/merge → (d) identifier-weighted BM25/IDF re-rank → (e) goal-scoped line prune → (f) hard token cap → MINIMAL CONTEXT SET → LLM (reasoning only) ↑ (g) vetting gate wraps (b)→(f) ``` All artifacts land in `build/04-code-context-retrieval/`. Nothing writes into the live harness. ### 3.1 (a) Query/goal construction (D) **Input:** `task_text: str` (the issue/request verbatim) + optional `repro_target: str | None` (the failing test node id from component 01 Stage-1 SBFL) + optional `latest_error: str | None` (truncated traceback from `run_test`). **Output:** a `Query` object, built by deterministic regex/symbol extraction — no model paraphrasing, no embedding: ```json { "identifiers": [ "memstore.MemStore.remember", "recall", ... ], "error_strings": [ "KeyError: 'kind'", ... ], "test_names": [ "tests/test_x.py::TestY::test_z", ... ], "file_paths": [ "src/memstore.py", ... ], "goal_hint": "context_focus_question string" } ``` **Extraction rules (exact):** | Field | Regex / rule (exact) | |---|---| | `identifiers` | Dotted identifiers `[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)+`; bare CamelCase `\b[A-Z][a-zA-Z0-9_]+\b`; snake_case tokens `\b[a-z_][a-z0-9_]*\b` — each kept **only if** it resolves in component 03's `name_index` (unresolved names are kept as *unresolved* query terms, a §3.2 recall input, never silently dropped). | | `error_strings` | Quoted strings `'…'` / `"…"` from `latest_error`/`task_text`; plus any line matching `Error:` / `Exception:` / `Traceback` / `assert`. | | `test_names` | `repro_target` if set; plus any token matching `\S+\.py::\S+` or `\S+::\S+::\S+` in `task_text`. | | `file_paths` | Any token matching `[\w\-/]+\.(py|js|ts|go|rs|java|c|h|toml|json|yaml|yml)` in `task_text`. | | `goal_hint` | The first sentence of `task_text` (up to the first `.` or `\n`), with leading issue/request verbs stripped (`fix`, `implement`, `add`, `change`, `when`, `given`, `please`, `we need to`). This is SWE-Pruner's `context_focus_question` re-implemented deterministically. | **Cap:** ≤ **32 identifiers**, ≤ **8 error_strings**, ≤ **8 test_names**, ≤ **8 file_paths**, each truncated at 120 chars. The `goal_hint` ≤ 200 chars. ### 3.2 (b) Multi-source recall (D) Three recall sources fire in parallel, each deterministic, each emitting candidates with provenance. **Source 1 — ripgrep lexical recall (primary).** For each `identifier` / `error_string` / `file_path` term, run (exact invocation): ``` rg --no-heading --line-number --color never -F '<term>' <repo_root> \ --glob '!vendor/**' --glob '!node_modules/**' --glob '!site-packages/**' \ --glob '!third_party/**' --glob '!external/**' --glob '!build/**' --glob '!.git/**' \ -g '*.py' | head -40 ``` - `-F` = fixed-string (literal terms, never regex — avoids escaping and injection). - `-g '*.py'` = the language glob from component 03's manifest `languages` (Python-only in Round 1). - `head -40` = the **result cap per query: ≤40 raw hits**. If `head` truncates, the candidate set carries `truncated: true` and the excess is *not* stored (a cap is a cap, not a queue). - Each hit → candidate `{ source: "ripgrep", term, file, line, text: "<that line>" }`. **Source 2 — symbol/API-description matching.** The `goal_hint` (and the full `task_text`) is scored against the §4 index's `desc_text` (signature + docstring) with the *same* BM25/IDF scorer as §3.4 — description↔description, **not** text↔code (AllianceCoder: match what the code will *call* against what the task *means*). Each match → candidate `{ source: "api-desc", qualname, kind, signature, docstring, file, line, score }`. **Cap: ≤20 symbols.** **Source 3 — call-graph neighbor expansion.** For every `identifier` that resolves to a `def` node in component 03's `recon.db`, expand **one hop** over `invoke`/`import`/`contain` edges. Each neighbor `def` → candidate `{ source: "callgraph", qualname, kind, signature, file, line }`. **Cap: ≤20 nodes.** (Round 1 uses these existing edges — the statement-level CFG/CDG/DDG re-rank is DEFERRED, §2 row 8.) Similar-code matching is **not** a Round-1 source (P3: similar code is the noise; the paper's title warns against it). It is admitted only as a later-round, lowest-weight source behind an A/B gate. ### 3.3 (c) Structure-aware dedup/merge (D) **Input:** the union of candidates from the three sources, each with provenance. **Output:** a deduplicated candidate set, one entry per structural identity. **Rules (exact):** 1. **Collapse by structural identity** — two candidates resolving to the same `qualname` (function / class / method) collapse to one entry; their provenance tags are concatenated, not discarded. 2. **Merge by containment, not text overlap** — if one candidate's `file:line` span lies inside another's (e.g. a line hit inside a function body), the inner span is absorbed into the containing function/class span. Overlapping line ranges merge into their **union span**. 3. **Dedup by identifier** — two hits naming the same identifier at the same `file:line` are one. 4. **Preserve syntactic boundaries** — every merged span is *extended* out to the nearest whole statement / function / class boundary (using component 03's `recon.db` node `end_line`), so no span ever cuts mid-statement or mid-body. ### 3.4 (d) Identifier-weighted BM25/IDF re-rank (D) **Input:** the deduplicated candidate set. **Output:** the same candidates with a `bm25_score`, ranked, truncated to **top-k = 20**. **Score (exact):** standard BM25 over tokenized candidate text with an identifier-weight multiplier (GrepRAG's identifier-weighted BM25): ``` score(d, q) = Σ_{t ∈ q∩d} w(t) · idf(t) · [ tf(t,d)·(k1+1) / (tf(t,d) + k1·(1 − b + b·|d|/avgdl)) ] idf(t) = ln( (N − n(t) + 0.5) / (n(t) + 0.5) + 1 ) w(t) = 3.0 if t matches [A-Za-z_][A-Za-z0-9_]* AND t ∈ name_index (a symbol token) = 1.0 otherwise (prose/comment token) k1 = 1.5, b = 0.75, N = # candidates, avgdl = mean candidate length ``` Identifiers and symbols carry **3×** weight over natural-language prose, so a candidate whose *symbols* match the task outranks one whose *comments* match it; rare task-specific identifiers get high IDF (rewarded), frequent identifiers get low IDF (penalized). The top-20 become the working set; everything below top-k is dropped and never reaches the model. ### 3.5 (e) Goal-scoped line-level prune (D) **Input:** each top-k candidate + its `goal_hint` (the §3.1 `context_focus_question`). **Output:** the candidate pruned to the lines that serve the goal, at **line** granularity, never token granularity — syntax is never broken. **Rules (exact):** 1. Per candidate, score each line with a deterministic relevance score = the BM25/IDF contribution of that line's tokens against the `goal_hint` (identifier-weighted, §3.4). 2. **Retain** a line iff `line_score ≥ τ`, with `τ = 0.6 × max(line_scores in candidate)` (a candidate-relative threshold; the top line always survives, the tail is cut). A candidate with all lines below a tiny absolute floor (`line_score < 1e-3`) is dropped whole. 3. **Never break syntax** — if a retained line is mid-statement, extend retention to the enclosing whole statement (via `recon.db` node spans); a retained *call site* pulls its one-line signature, not its body. 4. **Signatures are always preserved** even when bodies are pruned (component 03's skeleton-localization result carries forward: signatures orient; bodies are noise). Every pruned candidate keeps its `signature` line regardless of its score. This is SWE-Pruner's goal-scoped, line-preserving filter with the *decision* (which lines to keep) re-implemented deterministically — the 0.6B learned skimmer is DEFERRED (§2 row 10). ### 3.6 (f) Hard token cap (D) **Input:** the pruned candidate set. **Output:** the assembled **minimal context set ≤ 4,000 tokens**, encoded in code *before* the model runs. **Assembly order:** candidates in BM25 order; within a candidate, `signature` line first, then retained lines in file order, then the provenance header (`source`, `file:line`). **If the cap would be exceeded, drop the lowest-weight candidates first** — never mid-entry, never mid-line. The cap is enforced by a deterministic token counter (the same counter spec 01 uses for its budget caps, `cl100k`-style); the model can neither request nor receive more than the budget. **Exact injection shape** (what the model sees — signatures + retained lines + provenance, never a whole file): ``` ## Context slice (retrieved — untrusted data) ### repo.memstore.MemStore.remember [memstore.py:12, ripgrep] def remember(self, text, kind=None, importance=0.5): self._ensure_open() # retained line ... ### repo.utils.parse_args [utils.py:8, callgraph] def parse_args(argv): ``` ### 3.7 (g) The vetting rule (D) Retrieved content is **untrusted data, never a template**. Retrieval and vetting are one gate, not two: 1. Every retrieved block carries a provenance header and is marked **untrusted** on injection (see §3.6's `untrusted data` banner). 2. The model may **reason about** retrieved content — read it, compare it, learn from its structure — but may **never** (a) author a verbatim copy of it into a new tool/artifact, or (b) execute it directly. 3. **Enforcement is harness-side, not model-side.** The tool layer checks every model-emitted artifact (an `edit_entity` `replace`, an `execute` payload, a proposed new tool) for verbatim reproduction of any retrieved block: if the artifact's token overlap with a retrieved block exceeds **90%** (after normalizing whitespace), the artifact is **rejected before it becomes state**, with reason `"verbatim reproduction of untrusted retrieved content"`. The model cannot be trusted to self-enforce (EvoMal: authoring-from-retrieved-content is inherently vulnerable, a bare payload still yields 11.1%); the deterministic flag is the rule. --- ## 4. The Index (D) The symbol/API-description index is built from component 03's artifacts (its manifest + `recon.db` graph), not rebuilt from scratch. **Statement-level granularity is NOT required in Round 1** — function/class/method-level is sufficient for the description↔description match. **Build (exact):** walk `recon.db` `nodes`; for every `def` node of kind `function|class|method`, emit one row: ```json { "qualname": "repo.memstore.MemStore.remember", "kind": "method", "signature": "def remember(self, text, kind=None, importance=0.5):", "docstring": "<first docstring paragraph, ≤200 chars, else ''>", "desc_text": "<signature> <docstring>", "file": "memstore.py", "line": 12 } ``` `desc_text` is the searchable NL text — the signature (for exact symbol matching) concatenated with the docstring (for the description↔description, "what the code will call" match of AllianceCoder). **Storage (exact):** a single SQLite file `build/04-code-context-retrieval/api.db` with table `api_index(qualname TEXT PRIMARY KEY, kind TEXT, signature TEXT, docstring TEXT, desc_text TEXT, file TEXT, line INT)`. **No vector DB** — matching is BM25/IDF over `desc_text`, computed by the same ~50-line stdlib scorer as §3.4 (optionally backed by SQLite's stdlib FTS5 `bm25()`, which needs no pip install). The index is built **once per repo** (reusing component 03's already-parsed graph), keyed by `sha256(file_bytes + path)` per file so later runs re-index only changed files. --- ## 5. Implementation Order Seven phases, each independently testable and reversible. Each lands in `build/04-code-context-retrieval/` as a standalone importable module. Rollback = delete/flag-off that phase's module; nothing earlier breaks because phases only add modules and never mutate live harness code. | Phase | Delivers | Isolated test | Rollback | |---|---|---|---| | **1. Query construction + ripgrep recall** | `query.py` (regex/symbol extraction + goal-hint), `recall_ripgrep.py` (the §3.2 invocation + ≤40 cap) | Fixture repo + a scripted `task_text` naming a known function: assert `identifiers` extracts the dotted name, `goal_hint` is the first sentence with verbs stripped, and the ripgrep call returns ≤40 hits with the correct `file:line`. A term matching nothing returns `[]` (not an error). | Delete the two modules (pure read). | | **2. Dedup/merge** | `dedup.py` (structural-identity collapse, containment merge, boundary preservation) | Fixture where the same function is hit by ripgrep (a body line) and api-desc (the def): assert they collapse to one entry; a line hit *inside* the function is absorbed into the function span; overlapping line ranges merge to the union; no merged span cuts a statement (uses `recon.db` `end_line`). | Delete `dedup.py`. | | **3. BM25/IDF re-rank** | `bm25.py` (~50-line stdlib scorer + identifier weighting + top-k=20) | Unit: a candidate whose *symbol* matches the query outranks one whose *comment* matches (weight 3× vs 1×); a rare identifier outranks a frequent one (IDF); output never exceeds 20. Known-vector fixtures assert the exact §3.4 formula. | Delete `bm25.py`. | | **4. Symbol/API-description index + description↔description match** | `api_index.py` (build `api.db` from `recon.db`), `recall_apidesc.py` (§3.2 Source 2) | Fixture repo with a documented function: assert `api.db` has one row per exported `def`, `desc_text` = signature + first docstring para; a task whose *description* matches a docstring (but shares no tokens with the code) still ranks that symbol in the ≤20. | Delete `api_index.py` + `api.db` + `recall_apidesc.py`. | | **5. Goal-scoped line prune** | `prune.py` (§3.5 line scoring + τ + syntax-preserving extension + signature-kept) | Fixture candidate: assert lines below `0.6 × max` are dropped, a retained mid-statement line extends to the whole statement, and the `signature` survives even when its body is pruned; a whole-candidate drop when all scores `< 1e-3`. | Delete `prune.py`. | | **6. Token cap + vetting** | `assemble.py` (≤4K assembly, lowest-weight-first drop), `vet.py` (verbatim-reproduction flag) | Assert the assembled set ≤ 4,000 tokens on a deliberately-oversized candidate set (and that it drops lowest-weight entries first, never mid-entry); a model artifact reproducing a retrieved block at >90% token overlap is rejected with the exact §3.7 reason; a non-reproducing artifact passes. | Delete `assemble.py` + `vet.py`. | | **7. A/B runner** | `ab.py` (feature-flag router + retrieval-metric capture + oracle diagnostic + decision) | End-to-end on 3 warm-up tasks; `ab.py` runs old-vs-new on the §7 sample, emits the retrieval metrics + resolve/cost/tokens + the §7 decision. | Feature flag `off` → live harness untouched; `ab.py` is read-only. | The A/B runner (Phase 7) is the only phase that ever *invokes* the current harness — and only to measure it, never to modify it. --- ## 6. Dependencies & Supply-Chain Per Piper's rule: nothing is installed unless >72h old AND (≥10 stars OR deep-scrutinized). This build is **zero-install for Python-only Round 1**: | Dependency | Why | Status / age-stars | Install needed? | |---|---|---|---| | `ripgrep` (`rg`) | lexical recall (Source 1) | **Confirmed installed** — `ripgrep 15.2.0` at `/home/agent/.pi/agent/bin/rg` (per the grep-guard rule). | **No** | | BM25/IDF | identifier-weighted re-rank (§3.4) + description match (§4) | **A ~50-line stdlib impl** (pure Python, no pip); optionally SQLite stdlib FTS5 `bm25()`. | **No** | | symbol/API-description index | built from component 03's `recon.db` | Reuses component 03's stdlib `ast` graph (Python-only). | **No** | | call-graph edges | neighbor expansion (Source 3) | Component 03's `recon.db` `edges` table (`invoke`/`import`/`contain`). | **No** | | stdlib `re`, `json`, `sqlite3`, `hashlib`, `subprocess` | extraction, storage, hashing, ripgrep subprocess | CPython stdlib. | **No** | | `tree-sitter` + `tree_sitter_{...}` grammars | multi-language symbol index | **FLAG: only if a later A/B expands past Python-only.** Must be **>72h old** AND (≥10 stars OR deep-scrutinized via `~/mailtool/security_director.py screen "gh:tree-sitter/py-tree-sitter"`) plus `package_guard.py test`. | **Flagged** — see below | | SWE-bench-Verified dataset | A/B sample (§7) | Not a pip package; a data fetch. Not installed locally. | **Flagged (data, not code)** — see §7 fallback | **Preferred path: zero-install.** Round 1 is Python-only, so the symbol index reuses component 03's stdlib `ast` graph and nothing is pip-installed. `tree-sitter` is adopted **only** if the A/B (§7) expands to a multi-language repo and passes the 72h + star/scrutiny + sandbox-test gate; until then the fallback stands. --- ## 7. Acceptance Criteria — the A/B Protocol **Sample (concrete).** Reuse the **same 20 held-out SWE-bench-Verified instances as spec 01** (Python-only, single-repo, no-Docker reproduction). Held out = never seen by any tuning, prompt, or ranking pass. Seeded RNG. **Fallback** (if the dataset cannot be fetched locally): `coding-cortex/piperbench` sampled to 20 tasks, held out identically. ### 7.1 (a) Retrieval metrics — measured SEPARATELY (RepoBench-R style) Retrieval is graded on retrieval, not on the patch it feeds. For each `(repo, task)` pair, gold context = the gold patch's file/symbol region (or, in SWE-Explore's manner, ≥2 successful agent trajectories). The retriever returns a ranked candidate list under the §3.6 token cap; score: 1. **recall@1 / recall@3 / recall@5** — fraction of gold regions recovered in the top-1 / top-3 / top-5. 2. **MRR** — mean reciprocal rank of the gold region in the ranked list. 3. **required-file recall** — fraction of gold *files* surfaced. 4. **required-symbol recall** — fraction of gold *symbols* surfaced. 5. **irrelevant-context ratio** — share of retrieved tokens outside the gold region (the "noise" metric; **lower is better**). 6. **tokens retrieved** — budget actually spent (**lower is better**). These are run **standalone** (no patch, no Stage 2/3 of component 01), on the *same* held-out sample. ### 7.2 (b) The oracle-context diagnostic Each evaluated task runs **twice** on the harness's own weak model (`deepseek-v4-pro`): once with the pipeline's retrieved context, once with **oracle** context (the gold slice, hand-supplied). The outcome partitions the space: | oracle+Piper | retrieved+Piper | Verdict | |---|---|---| | succeed | fail | **retrieval failure** — the pipeline failed to surface what the model could have used; the fix belongs to the retriever. | | fail | fail | **reasoning failure** — the model could not use even perfect context; no retriever improvement helps. | | succeed | succeed | pipeline is **not** the bottleneck. | | fail | succeed | curation **outperforms** the gold slice — the gold slice is too noisy (the more-context-is-noise finding). | This keeps the retriever honest: it is only credited for what the weak model demonstrably *could* use. A retrieval-metric win that shows up only in the "reasoning failure" cell is **not** credited to the retriever. ### 7.3 (c) End-to-end regression guard 3. **Resolve rate** = `resolved / total` (spec 01's score). 4. **Cost** = `$ / instance` (model spend only). 5. **Tokens** = `tokens / instance`. ### 7.4 Decision rule (explicit thresholds) Run candidate-vs-current on the *identical* sample with the *identical* model (`deepseek-v4-pro`) and hardware (this box). The candidate **wins iff**: ``` ( recall@1 OR recall@3 OR recall@5 OR MRR OR required_file_recall OR required_symbol_recall ) strictly improves (≥1 metric up) AND recall@1/3/5, MRR, required_file/symbol_recall do NOT regress (each ≥ current) AND irrelevant_context_ratio_candidate ≤ irrelevant_context_ratio_current AND tokens_retrieved_candidate ≤ tokens_retrieved_current AND resolve_candidate ≥ resolve_current AND cost_candidate ≤ 1.5 × cost_current AND tokens_candidate ≤ 1.5 × tokens_current ``` At least one retrieval metric must **strictly** improve, and **none** may regress; the two noise-suppression metrics (irrelevant-context ratio, tokens retrieved) must not rise; end-to-end resolve must not regress; cost and tokens are bounded at 1.5×. A candidate that improves retrieval but regresses resolve or blows a cost/token bound is **not adopted**; the tradeoff is recorded explicitly. If the candidate wins, it replaces the current context path (the model browsing/searching the repo itself); if it loses, the build tree is kept for the record and the live harness is unchanged. **Ablation rule (mechanical).** Each pipeline step (§3.2–3.6) is removed in isolation, everything else held fixed, and the delta measured on the retrieval metrics (§7.1). A step is **deleted** iff its removal changes none of recall@1/3/5, MRR, required-file/symbol recall, irrelevant-context ratio, or tokens-retrieved beyond noise (`Δ = 0`, or `|Δ| ≤ 5%` where continuous). Applied first to the weakest a-priori cases (the call-graph neighbor expansion, and the goal-hint verb-stripping), then to any other step — regardless of how sensible it looks. --- ## 8. Guardrails & Reversibility **Revocability.** - The build lives entirely in `build/04-code-context-retrieval/` (a separate tree); it never writes into the live harness's source. - A **feature flag** (env var `RETRIEVE=ctx04|current`, default `current`) routes the A/B; the flag alone switches between the new retriever and the current "model looks around" path — no code deletion required to revert. - `ab.py` is a read-only driver: it *measures* the current harness, it never modifies it. Running old-vs-new is always possible in the same session, same repo, same model. **Invariants (must hold, checked at the end of every phase).** 1. **No new `.md` files outside this folder** (`projects/self-improvement/{papers,specs,build}`). Everything else is code or DB rows. 2. **No change to the live harness** until the §7 A/B gate passes. The current tool surface stays byte-identical during the entire build. 3. **Context is always bounded**: the assembled slice ≤ 4,000 tokens, enforced in code before the model runs; the model is never handed the whole repo or a full-file dump. 4. **The retriever is deterministic**: no neural retriever, no embedding index, no GPU, no learned skimmer, no LLM-decomposed query in Round 1. The LLM column holds exactly the final reasoning over the curated slice. Any phase that introduces a neural/embedding step into a D node is rejected outright (those are §2 rows 8–11, deferred, gated). 5. **Retrieved content is untrusted data, never a template** — the vetting rule (§3.7) flags verbatim reproduction; retrieved content is never blindly imitated or executed. 6. **Never break syntax** — dedup/merge and pruning operate at statement/function granularity; a token-level cut that breaks a statement is a rejected change. --- *End of spec. This document is the build contract; the next document records the A/B outcome.*