Summary / 06. Dependency and impact analysis / Document
Spec 2026-09-04
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Build Spec 06 — Dependency & Impact Analysis (the ≈80% fraction) > Derived from `papers/06-dependency-impact-analysis.md`. This document drives the build of the > **dependency-and-impact-analysis stage**: the forward-looking projection from a proposed change to > everything else it may break, computed *before* any further code is written. It sits after > component 05 (localization: where is the fault) and answers the question localization deliberately > does not: *given the edit I am about to make, what else does it reach.* It translates the paper's > design into exact change-type classifiers, exact relation-set tables, exact walk caps, exact > scoring formulas, exact triage thresholds, exact schemas, and an exact build sequence — no further > research required. Where the paper says "≈80%", this spec pins *which* 80%. --- ## 1. Purpose & Scope Build a **reversible dependency-and-impact-analysis stage** — a deterministic-first, change-type-aware, causal projector (AST change-type classification → relation-set selection → depth-capped reverse graph walk over hard edges → git co-change recall → test-impact map → hard-gate/soft-rank → MUST/MAY/ LOW-RISK triage → dependency-consistent batch assembly → cause-labeled impact DAG) with a **single weak-LLM next-edit decision read off the map** — as a drop-in *forward-looking* stage that converts a proposed change into downstream **change and validation obligations** before any further code is written. It is distinct from its three predecessors in kind, not merely in order: component 03 answers "what the repository *is*", component 04 "what context is *relevant*", component 05 "where the fault *is*" — component 06 answers "what the repository *will be*" after the edit, and it is the first stage in the pipeline whose arrow points forward rather than backward. The overall target is the paper's full 100% spec. **Round 1 lands the expected ≈80% of it**: everything deterministic (the change-type classifier, the CodePlan relation-set table, the reverse graph walk, the TDAD-style test-impact map, git co-change recall with anti-leakage, the hard-gate/soft-rank, triage, batch assembly, and the cause-labeled impact DAG) plus the *one* weak-LLM next-edit decision — targeting the reachable fraction of the corpus where the L column holds exactly one entry and everything upstream is deterministic software on CPU. The heavy learned pieces (full escape analysis, Athena's CodeBERT conceptual coupling, Ripple's K× self-consistency sampling) are **not** built in Round 1 and are **not excluded** — they are the content of later rounds that climb from that ~80% landing toward 100% (see §2). Round 1 is kept only if it wins a measured A/B test on the *honest* impact metrics — led by the PASS_TO_PASS regression rate — *and* does not regress end-to-end resolve/cost/tokens (§7). --- ## 2. Round 1 vs later rounds (toward the 100% target) The goal is the paper's full **100%** spec. **~80%** is only the expected **first-round landing**; later rounds climb from there. Every element from the paper is classified below by *which round builds it*. - **BUILD-NOW = the Round 1 build** — the expected ≈80% deterministic landing this spec implements and A/B-tests (rows 1–10). - **DEFERRED = later rounds** — the learned/expensive stages (rows 11–13) that Round 1 intentionally does not build *yet*, because they need a GPU, fine-tuning, or multiple LLM calls. They are **not excluded from the project** — they are the content of later rounds, the climb toward 100%, gated by the same §7 A/B protocol once the Round 1 core is in place. | # | Component (paper §) | Classification | |---|---|---| | 1 | Atomic change-type classification — AST diff before/after the proposed edit, deterministic pattern-matching, no LLM | **BUILD-NOW** | | 2 | Atomic-change-type → relation-set table (CodePlan Table 1) — fixed lookup, relations not in the table are never traversed | **BUILD-NOW** | | 3 | Reverse graph walk over hard edges (call/import/inherit/class-member) + TESTS edges, depth-capped (≤3 hops) | **BUILD-NOW** | | 4 | Git co-change recall — (file → co-changed-files) pairs from component 05's history index (BM25 over commit messages + churn), L=1 hop, anti-leakage (pre-date commits only) | **BUILD-NOW** | | 5 | Test-impact — TDAD-style 4-node × 5-edge graph + confidence-weighted merge, emitting `test_map.txt` (the targeted validation set) | **BUILD-NOW** | | 6 | Hard-gate / soft-rank — hard edges decide MEMBERSHIP; deterministic lexical-coupling score ranks within membership (no transformer) | **BUILD-NOW** | | 7 | MUST-INSPECT / MAY-IMPACT / LOW-RISK triage by combined confidence | **BUILD-NOW** | | 8 | Dependency-consistent batch assembly — connected components under hard-edge transitive closure; each component is ONE change unit | **BUILD-NOW** | | 9 | Impact DAG with cause-edge labels, serialized (JSON) for the LLM | **BUILD-NOW** | | 10 | Weak-LLM next-edit decision read off the map (the sole L entry) | **BUILD-NOW** | | 11 | Full escape analysis for method-body changes (interprocedural points-to of escaping objects) | **DEFERRED** — Round 1 uses a simplified conservative gate (§3b) | | 12 | Athena's transformer conceptual coupling (CodeBERT embeddings + fine-tuning for soft edges) | **DEFERRED** — needs a GPU / model download + fine-tuning; too heavy for CPU | | 13 | Ripple's self-consistency sampling (K× LLM calls, majority vote over next-edit predictions) | **DEFERRED** — K× LLM cost; Round 1 uses exactly ONE LLM call | **Escape-analysis decision: BUILD-NOW-CONSERVATIVE-GATE.** Round 1 does not build interprocedural points-to escape analysis. It substitutes a conservative deterministic gate (§3b): a method-body change propagates to its callers **iff** the change touches an object that could be observed by a caller — i.e. the changed statement (a) assigns to a parameter, (b) assigns to an attribute of `self`/a parameter, (c) mutates a global/module-level binding, or (d) changes the return expression. A body change confined to purely-local temporaries (neither returned nor written into any reachable object) terminates propagation immediately, matching the paper's "no observable escape → no propagation" rule in its cheap form. Full escape analysis is later-round row 11. --- ## 3. The Deterministic Pipeline (D) Nine stages, all **D** through the production of the cause-labeled impact DAG. The LLM column holds exactly one thing: the next-edit decision (§4), reached after the DAG is complete. The pipeline: ``` PROPOSED CHANGE → (a) change-type classification → (b) relation-set selection → (c) reverse graph walk (hard edges + TESTS, ≤3 hops) → (d) git co-change recall (L=1, anti-leakage) → (e) test-impact map (test_map.txt) → (f) hard-gate/soft-rank → (g) triage (MUST/MAY/LOW-RISK) → (h) dependency-consistent batches → (i) impact DAG (cause-labeled JSON) → (§4) LLM next-edit decision (once) ``` All artifacts land in `build/06-dependency-impact-analysis/`. Nothing writes into the live harness. ### 3.1 (a) Atomic change-type classification (D) **Input:** the proposed edit as a *before-tree* and *after-tree* (or a diff) of the touched file(s), plus the file path. **Output:** a list of atomic changes. **No language model reads the edit.** ```json { "changes": [ { "change_type": "method-signature", "entity": "repo.auth.LoginService.login", "file": "src/auth/login.py", "line_range": [12, 16], "before_sig": "def login(self, user):", "after_sig": "def login(self, user, mfa_code):" } ] } ``` **Procedure (exact, deterministic AST pattern-matching):** 1. Parse `before` and `after` with stdlib `ast` (Python) or the tree-sitter parser reused from component 03 (multi-language). Walk both trees in parallel, matching nodes by `(kind, parent path, name, position)`. Every matched node that differs, and every unmatched node (present in one tree only), becomes a candidate atomic change. 2. Classify each candidate by the **precedence ladder** (first match wins): - node present in `before` only → **`delete`**. - node present in `after` only → **`add`**. - a `ClassDef` whose `name`, `bases`, or `keywords` differ → **`class-decl`**. - a `FunctionDef`/`AsyncFunctionDef` whose name is `__init__` (or whose enclosing scope's constructor semantics match) and whose parameter list differs → **`constructor`**. - a `FunctionDef`/`AsyncFunctionDef` whose parameter names/count/defaults/annotations or decorators differ → **`method-signature`** (signature dominates body; a node where both signature and body differ is classified `method-signature`, not both). - a `FunctionDef`/`AsyncFunctionDef` whose body statements differ but args/decorators identical → **`method-body`**. - an `Import`/`ImportFrom` whose `names`/`module` differ → **`import`**. - an `Assign`/`AnnAssign`/`AugAssign` at class or module scope (a field/constant binding) whose value or annotation differs → **`field`**. - anything else that differs → **`method-body`** (the conservative default, which never under-propagates a structural change). 3. A multi-part edit (several distinct changed nodes) is **decomposed** into one atomic change per changed node; the composite impact is the **deduplicated union** of each atomic change's impact set. Each atomic change is analyzed separately (§3.2–3.9) and unioned at §3.9. **Emit:** one `{change_type, entity, file, line_range}` per atomic change. The classification is pattern-matching over the AST diff — no LLM is consulted to decide what kind of change is being made (the paper's single most important transferable decision, made mechanical). **Caps:** ≤ **64 atomic changes** per proposed edit (a larger edit is a composition; each is capped at 64 and the rest are folded into the nearest same-type change's line range); `entity` is a `qualname` capped at 200 chars; `line_range` is `[start, end]` from the diff hunks. ### 3.2 (b) Relation-set selection (D) **Input:** each atomic change's `change_type`. **Output:** the relation set to traverse (a table lookup). This is the **P1 gate**: relations not in the table are never traversed. A change-type-blind N-hop walk is forbidden (guardrail §8). **The fixed table (exact — CodePlan Table 1, with `add`/`delete` completed):** | change_type | relations traversed (reverse direction) | |---|---| | `method-signature` | `CalledBy`, `Overrides`, `OverriddenBy` | | `method-body` | `Callers` — **escape-gated** (below) | | `field` | `UsedBy`, `ConstructedBy`, `SubclassOf` (class-hierarchy) | | `class-decl` | `InstantiatedBy`, `SubclassOf`, `SuperclassOf` (inheritance) | | `constructor` | `InstantiatedBy` | | `import` | `ImportedBy` | | `add` | *(none — no reverse edges exist for a brand-new node)* | | `delete` | `CalledBy`, `UsedBy`, `ImportedBy`, `InstantiatedBy`, `Overrides`, `OverriddenBy` (union — anything that referenced the deleted entity) | **Escape-gate for `method-body` (exact, conservative Round-1 form):** a body change propagates to its callers **iff** the changed statement does any of: - (a) writes to a parameter (`param = …` or `param.attr = …`), - (b) writes to an attribute of `self` or of a parameter (`self.x = …`, `param.x = …`), - (c) writes to a global / module-level binding (`global g; g = …`), - (d) changes the function's `return` expression. Otherwise (all changed statements are purely-local temporaries, neither returned nor written into any reachable object) propagation **terminates immediately** — the body change has no observable escape and no caller is traversed. This is the paper's escape-analysis gating in its cheap deterministic form; the full interprocedural points-to analysis is DEFERRED (§2 row 11). **`add` note:** an added node emits **no reverse obligation** (nothing references it yet). The stage emits `{change_type: "add", obligations: []}` and marks the change `LOW-RISK`; the new node's own outgoing edges are recorded for the *next* cycle's graph, never traversed backward here. ### 3.3 (c) Reverse graph walk (D) **Input:** the edited node + the selected relation set + component 03's `recon.db` graph (nodes + `call`/`import`/`inherit`/`class-member` edges) augmented with `TESTS` edges (from §3.5). **Output:** the reachable impact set, each member tagged with its admitting relation(s) and hop distance. **Procedure (exact):** 1. Map the relation names of §3.2 to `recon.db` edge types: `CalledBy` = reverse of `call`; `Overrides`/`OverriddenBy` = `inherit` (method-override edges) in either direction; `UsedBy` = reverse of `class-member` (field refs); `InstantiatedBy` = reverse of `call` to a constructor def; `ImportedBy` = reverse of `import`; `SubclassOf`/`SuperclassOf` = `inherit`. 2. **BFS backward** — against the direction of the dependency — from the edited node over *only* the selected relation's edge types. Each visited node records `{entity, distance, admitting_relations}`. 3. **Depth cap:** `MAX_DEPTH = 3` hops (configurable `MAX_DEPTH`, default 3). Nodes beyond 3 hops are **not** collected (they would drown a weak model in irrelevant sites). A body change with no escape (§3.2) terminates at depth 0. 4. **Also walk `TESTS` edges** from every reached node (forward, toward tests) — the test impact is computed in §3.5, but the walk here seeds which tests are structurally reachable. **Emit:** `impact_set` = list of `{entity, file, distance, admitting_relations}`. **Caps:** ≤ **200** nodes per atomic change; if exceeded, keep the 200 nearest by distance and record `"truncated": true` with the omitted count. ### 3.4 (d) Git co-change recall (D) **Input:** the edited file(s) + component 05's git-history index (the `git log` derived churn/message index). **Output:** structurally-decoupled co-change candidates with a `co_change_score ∈ [0,1]`. **Procedure (exact):** 1. Mine `(file → co-changed-files)` pairs from history: for each commit, its changed-file set forms all-pairs co-change edges. **BM25 over commit messages** weights each commit's contribution by message relevance to the current task text; **churn** weights it by how often the pair recurs. `co_change_score(f, g) = normalized( Σ_commits_touching_both bm25(msg, task) · churn_weight )`. 2. **L = 1 hop only**: only direct co-change partners of the edited file are recalled (the paper's 86.4%-reduction/76.8%-recall phase is L=1). No transitive co-change closure. 3. **Anti-leakage (exact):** only **pre-date** commits — commit time strictly before the proposed change's date (equivalently, before the ground-truth fix commit when running the §7 benchmark). Post-date commits are excluded so history is a genuine prior, never a memorized answer. **Emit:** `co_change_candidates` = list of `{file, co_change_score}` sorted descending. **Caps:** ≤ **40** candidates per edited file; score truncated at 4 decimal places. ### 3.5 (e) Test-impact — `test_map.txt` (D) **Input:** the reachable impact set (§3.3) + the graph's `TESTS`/`CONTAINS`/`CALLS`/`IMPORTS`/ `INHERITS` edges. **Output:** a grep-able static file `test_map.txt` mapping each changed file to its affected tests with a combined confidence — **the targeted validation set** (which tests to re-run before submission, TDAD's selection-as-verification-gate). **The graph (exact — TDAD's 4-node × 5-edge shape):** four node kinds (`module`, `class`, `method`, `test`) joined by five edge types: `CONTAINS`, `CALLS`, `IMPORTS`, `TESTS`, `INHERITS`. These are the edges already in component 03's graph plus a `TESTS` edge from a unit to each test that exercises it (discovered from test collection: a test `def test_*` calling into a def creates a `TESTS` edge; containment under a class/module creates `CONTAINS`). **Confidence-weighted merge (exact):** a test `T` is affected by changed file `F` iff there is a graph path from `F` (or any entity in `F`'s impact set) to `T` over the five edge types. The combined confidence is the **max single-edge weight** on any admitting path: | Edge type | Weight | |---|---| | `TESTS` (direct test edge) | **1.0** | | `CALLS` (transitive call) | **0.56** | | `CONTAINS` (coverage/containment) | **0.5** | | `IMPORTS` | **0.45** | | `INHERITS` | **0.56** (transitive-call-equivalent) | `confidence(T | F) = max( w_e )` over every edge `e` on any path from `F` to `T`. A test reached only via a direct `TESTS` edge scores 1.0; a test reached via a two-hop `CALLS` chain scores 0.56. Tests with confidence **≥ `TEST_MIN = 0.45`** are included in the targeted validation set. **`test_map.txt` format (exact, one line per (file, test) pair):** ``` src/auth/login.py tests/test_login.py::TestLogin::test_mfa 0.56 src/auth/login.py tests/test_session.py::test_validate 1.0 ``` **Emit:** `test_map.txt` (the static file the LLM and the validation gate can read) plus an in-memory `test_impact` structure. **Caps:** ≤ **256** (file, test) pairs; pairs sorted by confidence descending within each file. ### 3.6 (f) Hard-gate / soft-rank (D) **Input:** the merged candidate set (structural walk §3.3 ∪ co-change §3.4 ∪ test-impact §3.5), each member tagged with how it was admitted. **Output:** the membership set plus a deterministic ranking within membership. **Hard-gate (exact):** a candidate is a **member** iff it was admitted by a **hard edge** — `call` / `import` / `inherit` / `class-member` / `TESTS` (§3.3, §3.5). A candidate admitted **only** by the soft co-change signal (§3.4) is **not a member** by itself; it becomes a member only if it also has a hard edge. This is Athena's two-class architecture with the learned soft edge replaced by a deterministic proxy. **Soft-rank (exact, deterministic lexical coupling — no transformer):** ``` lex(e) = w_id · jaccard(ids(e), ids(seed)) + w_str · jaccard(strs(e), strs(seed)) + w_imp · jaccard(imports(e), imports(seed)) w_id = 0.5, w_str = 0.3, w_imp = 0.2 jaccard(A, B) = |A ∩ B| / |A ∪ B| (0 if both empty) ``` - `ids(e)` = the set of identifier tokens in `e`'s `qualname` / signature / class name (split on `.`, `_`, and camel-case boundaries). - `strs(e)` = the set of string-literal tokens appearing in `e`'s skeleton signature/body-preview (from component 03's signature index). - `imports(e)` = the set of module names `e` imports. - `seed` = the edited entity. `lex(e) ∈ [0,1]`; the ranking is `lex` descending. The score is cheap, deterministic, and stands in for Athena's embedding-based conceptual coupling (which is DEFERRED, §2 row 12). **Emit:** `ranked_members` = list of `{entity, distance, admitting, lex, co_change_score, test_confidence}` sorted by `lex` (within each distance bucket) descending. ### 3.7 (g) Triage (D) **Input:** `ranked_members` + the atomic change's `change_type` + the §3.2 relation set. **Output:** the three-bucket blast-radius summary. **Buckets (exact):** | Bucket | Condition (exact) | |---|---| | **MUST-INSPECT** | hard-edge reachable **at distance 1** through a relation the change type **provably perturbs** — a deterministic binding check confirms breakage: a `method-signature` change with a caller whose argument count/names no longer match the new signature; a `delete`/`field` change with a `UsedBy`/`ImportedBy` reference to the removed symbol; a `class-decl` change with an `InstantiatedBy` site whose constructor args no longer bind. | | **MAY-IMPACT** | hard-edge reachable but **not provably** perturbed (distance ≥ 2, or distance 1 with no binding break), **OR** admitted only by a soft signal with `co_change_score ≥ COCHANGE_MIN = 0.3`, **OR** `test_confidence ≥ TEST_MIN = 0.56`. | | **LOW-RISK** | everything else the walk touched (distance > 0, lex below `LEX_MIN = 0.1`, and no MUST/MAY condition met). | **Combined confidence (exact, for reporting and the §3.9 ordering):** ``` conf(e) = max( hard_conf(e), co_change_score(e), test_confidence(e) ) hard_conf(e) = lex(e) · 0.5^(distance - 1) for hard-edge members; 0 otherwise ``` `MUST-INSPECT` iff (hard member at distance 1 with a binding break) — which corresponds to `conf ≥ 0.70` in practice; `MAY-IMPACT` iff `0.35 ≤ conf < 0.70` (or the soft/test thresholds above); `LOW-RISK` iff `conf < 0.35`. The thresholds `MUST_MIN = 0.70`, `MAY_MIN = 0.35`, `COCHANGE_MIN = 0.3`, `TEST_MIN = 0.56`, `LEX_MIN = 0.1` are configurable constants. **Emit:** `triage = { "must_inspect": [...], "may_impact": [...], "low_risk": [...] }`, each entry carrying `{entity, file, conf, reason}` where `reason` is the admitting relation's cause string (§3.9). ### 3.8 (h) Dependency-consistent batches (D) **Input:** the `MUST-INSPECT ∪ MAY-IMPACT` set + the hard-edge graph. **Output:** the change units. **Procedure (exact):** group the MUST+MAY members into **connected components** under hard-edge transitive closure (two members are in the same batch iff a hard-edge path connects them, ignoring direction). Each connected component is **ONE change unit** (DepWareTrans's dependency-consistent batch): it must be edited and validated *together*, never file-by-file. Within a batch, order members by **distance from the seed** (topological), so the seed/signature change precedes its callers — the temporal-consistency ordering that keeps the repository valid after every step. **Emit:** `batches` = list of `{ batch_id, members: [ordered entity list], size }`. A batch with a single member is still a batch (unit-of-change uniformity). **Caps:** ≤ **32** batches, each ≤ **50** members; if a component exceeds 50 members it is split at the 50-nearest boundary and flagged `"split": true`. ### 3.9 (i) Impact DAG emission (D) **Input:** `triage` + `batches` + the admitting relations. **Output:** the obligations materialized as an explicit **DAG**, serialized to JSON for the LLM. **Node:** a downstream site (entity) with `{entity, file, kind, one_line_signature, batch_id}`. **Edge:** a cause label connecting a change to its consequence, e.g. `"signature changed → LoginService now inconsistent via CalledBy"` — the paper's compiler-blames-the-override antidote made explicit: a later compiler error at an override is attributed to the seed change, never mistaken for a new fault. **Serialized schema (exact):** ```json { "seed": { "change_type": "method-signature", "entity": "repo.auth.LoginService.login", "file": "src/auth/login.py", "line_range": [12, 16] }, "obligations": { "change": [ { "batch_id": 0, "members": ["repo.auth.LoginService.login", "repo.auth.Session.validate", "repo.auth.Token.issue"] } ], "validate":[ "tests/test_login.py::TestLogin::test_mfa", "tests/test_session.py::test_validate" ] }, "dag": { "nodes": [ { "entity": "repo.auth.Session.validate", "kind": "method", "file": "src/auth/session.py", "line": 41, "signature": "def validate(self, token, mfa_code):" } ], "edges": [ { "from": "repo.auth.LoginService.login", "to": "repo.auth.Session.validate", "cause": "signature changed (added param mfa_code) → caller must re-bind via CalledBy" } ] } } ``` **Cause-edge label template (exact):** `"<change_type> changed (<what changed>) → <consequence site> <obligation verb> via <relation>"`. The verb is `must re-bind` (signature/import), `must be updated` (field/constructor/class-decl), `must be re-checked` (method-body escape), `must be re-tested` (test edge). The DAG encodes **sequencing** (topological order by distance) and **cause** (which edge justifies which obligation). **Caps:** the DAG is ≤ **200 nodes** and ≤ **400 edges**; serialized JSON ≤ **12,000 tokens** (a token counter enforces; the largest summaries truncate first). --- ## 4. The LLM Step (L, ONE entry) The **single** learned node in the entire stage. The weak model reads the impact DAG + per-method **summaries (signatures, not bodies)** + the change plan, and decides the **next edit location**. It **never sees code blobs** and **never re-derives the graph** — the map is handed to it, pre-cohered. This is the paper's "the model contributes one decision per cycle, handed a map, never a code blob." **Prompt inputs (assembled deterministically, in this exact order):** 1. **`change_plan`** — the proposed seed change, one line (`"add mfa_code param to LoginService.login"`). 2. **The impact DAG** (§3.9 serialized JSON) — nodes as `{entity, kind, signature}` (signature-only), edges as cause labels, plus the `obligations.change` batch list. 3. **Per-method SUMMARIES** — for each MUST-INSPECT node, its one-line signature from component 03's signature index. **Signatures, never bodies** — no statement bodies, no line-span diffs, no full files. **Output schema (exact — one JSON object):** ```json { "next_edits": [ { "entity": "repo.auth.Session.validate", "reason": "login signature gained mfa_code; validate call must re-bind" }, { "entity": "repo.auth.Token.issue", "reason": "issue instantiates LoginService, must pass the new argument" } ] } ``` **Caps:** **one call**, `temperature = 0`, `max_completion_tokens = 1024`, ≤ **50** entities in the map (each summary ≤ 200 chars, signature-only). The model **emits identifiers and one-line reasons** — it does not explore, does not see bodies, does not call tools. Any JSON parse failure is a **no-op**: the stage falls back to the deterministic batch order (§3.8's distance ordering) and emits `{"next_edit": "failed", "fallback": "batch_order"}` — never a re-prompt, never a second call. --- ## 5. Implementation Order Eight phases, each independently testable and reversible. Each lands in `build/06-dependency-impact-analysis/` as a standalone importable module. Rollback = delete/flag-off that phase's module; nothing earlier breaks because phases only add modules and never mutate live harness code. | Phase | Delivers | Isolated test | Rollback | |---|---|---|---| | **1. AST change-type classifier** | `classify.py` (§3.1 diff + precedence ladder + atomic decomposition) | Fixture before/after AST pairs: a param-list change → `method-signature` (not `method-body`); a body-only change → `method-body`; a `__init__` param change → `constructor` (dominates signature); a class-bases change → `class-decl`; an import change → `import`; a node only-in-after → `add`, only-in-before → `delete`; a multi-node edit decomposes into ≥2 atomic changes. Assert **no LLM is consulted** (probe the LLM client to raise — the classifier must not touch it). | Delete `classify.py` (pure read). | | **2. Relation-set table + reverse graph walk** | `relations.py` (§3.2 table + escape-gate), `walk.py` (§3.3 BFS, reuses component 03's `recon.db`) | Fixture graph: a `method-signature` change reaches its caller and overrider, and does **not** traverse a `field` relation; a `method-body` change with a purely-local edit reaches **no** caller (escape-gate terminates at depth 0); a body change writing `self.x` reaches callers; BFS never exceeds `MAX_DEPTH=3`; `add` emits zero obligations. Assert a change-type-blind N-hop walk is impossible (the walk takes the relation set, not a blanket hop count). | Delete `relations.py` + `walk.py`. | | **3. Test-impact / `test_map.txt`** | `testmap.py` (§3.5 4×5 graph + weighted merge + file emit) | Fixture graph: a direct `TESTS` edge scores **1.0**, a two-hop `CALLS` path scores **0.56**, an `IMPORTS` path **0.45**; a test with confidence < 0.45 is excluded; the emitted `test_map.txt` lines match `file test_id conf` exactly. | Delete `testmap.py` + the emitted `test_map.txt` (build-tree artifact only). | | **4. Git co-change recall** | `cochange.py` (§3.4 pairs + BM25 + churn + L=1 + anti-leakage, reuses component 05's history index) | Fixture git repo with planted history: a historically-co-changed partner surfaces with a non-zero score; a **post-date** commit's pair is excluded (anti-leakage); L=1 only (no transitive partner of a partner); an unrelated file scores 0; output ≤ 40 candidates. | Delete `cochange.py` (pure read). | | **5. Hard-gate / soft-rank + triage** | `rank.py` (§3.6 lexical score + membership gate), `triage.py` (§3.7 buckets + thresholds) | Scripted candidate set: a soft-only candidate is **not** a member; a distance-1 caller of a signature change lands **MUST-INSPECT**; a distance-2 neighbor lands **MAY-IMPACT**; a low-lex distance-1 non-binding neighbor lands **LOW-RISK**; thresholds `MUST_MIN=0.70/MAY_MIN=0.35` partition exactly. Assert `lex` uses the §3.6 weights and no embedding is computed (probe to raise). | Delete `rank.py` + `triage.py`. | | **6. Dependency-consistent batch assembly** | `batch.py` (§3.8 connected components + distance ordering) | Fixture graph with two disconnected MUST+MAY clusters: emits **2** batches; within a batch the seed precedes its callers (topological); a >50-member component is split and flagged; a singleton is still a batch. | Delete `batch.py`. | | **7. Impact DAG emission + LLM next-edit** | `dag.py` (§3.9 cause labels + JSON), `nextedit.py` (§4 prompt + one call + output schema + parse-failure fallback) | Mock the LLM client to return a scripted ordered-entity list: assert **one** call, `temperature=0`, ≤1024 tokens, the prompt contains the DAG + signatures but **no statement body text**; assert the DAG JSON carries cause-edge labels; a malformed response → `{"next_edit":"failed"}` and the batch order stands, **no second call**. | Delete `dag.py` + `nextedit.py`. | | **8. Impact A/B harness** | `ab.py` (feature-flag router + PASS_TO_PASS capture + oracle-impact diagnostic + hidden-channel probe + decision) | End-to-end on 3 warm-up tasks; `ab.py` runs old-vs-new on the §7 sample, emits impact recall/precision, the PASS_TO_PASS regression rate, the oracle-impact gap, the hidden-channel report, and resolve/cost/tokens + the §7 decision. | Feature flag `off` → live harness untouched; `ab.py` is read-only. | The A/B runner (Phase 8) is the only phase that ever *invokes* the current harness — and only to measure it, never to modify it. --- ## 6. Dependencies & Supply-Chain Per Piper's rule: nothing is installed unless >72h old AND (≥10 stars OR deep-scrutinized). This build is **zero-install for Round 1 (Python-only, stdlib + in-repo)**: | Dependency | Why | Status / age-stars | Install needed? | |---|---|---|---| | stdlib `ast`, `json`, `re`, `sqlite3`, `hashlib`, `difflib`, `collections` | AST diffing (§3.1), DAG serialization, lexical score, storage, hashing | CPython stdlib. | **No** | | tree-sitter (multi-language parse) | reuse component 03's parser where a repo is non-Python | Reuse component 03's `recon.db` (already built/flagged there); no new parser. | **No** | | `git log` | §3.4 co-change pairs | **Confirmed installed** — `git` is present (used by recovery/backup and code_check flows). | **No** | | Component 03 index/graph (`recon.db`: `nodes`, `edges`, `name_index`) + `TESTS`/`CONTAINS` edges | §3.3 reverse walk + §3.5 test map | In-repo, built by spec 03; `TESTS` edges are added by this build's §3.5. | **No** | | Component 05 git-history index | §3.4 co-change recall (churn + message BM25) | In-repo, built by spec 05. | **No** | | Component 03 signature index (skeleton signatures) | §3.9/§4 summaries | In-repo, built by spec 03. | **No** | | stdlib BM25/IDF scorer (component 04's `bm25.py`) | §3.4 message BM25 | In-repo, already vetted. | **No** | **Preferred path: zero-install.** Round 1 is Python-only and deterministic: the AST classifier and graph reuse stdlib `ast` and component 03's `recon.db`, `git log` is installed, co-change recall reuses component 05's history index, and the test map reuses the `TESTS`/`CONTAINS` edges. **No new package is installed for Round 1, and nothing is flagged for a later install** — the three DEFERRED rows (§2 11–13: full escape analysis, Athena CodeBERT coupling, Ripple self-consistency) are code and/or fine-tuning on existing weights, not new third-party installs; any future model download they imply must clear the 72h + star/scrutiny + sandbox-test gate before use, and is out of Round 1 scope entirely. --- ## 7. Acceptance Criteria — the A/B Protocol **Sample (concrete).** Reuse the **same 20 held-out SWE-bench-Verified instances as spec 01** (Python-only, single-repo, no-Docker reproduction), plus a **curated hidden-channel corpus** (§7.4). Held out = never seen by any tuning, prompt, or ranking pass. Seeded RNG. **Fallback:** `coding-cortex/piperbench` sampled to 20 tasks, held out identically. ### 7.1 (a) Impact-set recall/precision Ground truth = the **co-changed file set** of **untangled bug-fix commits** (the files a maintainer actually touched together in a single, manually separated change — Alexandria's methodology, a human's causal account, not a model's claim). Against that ground truth, report: 1. **recall** = `|MUST ∪ MAY files ∩ gold co-changed files| / |gold co-changed files|`. 2. **precision** = `|MUST ∪ MAY files ∩ gold co-changed files| / |MUST ∪ MAY files|`. 3. **distance-to-co-changed-file** — structural distance (hops in `recon.db`'s edge graph) from each predicted file to the nearest gold co-changed file; rewarding "wrong file, right cluster" over a correct-looking wrong answer far away. **Lower is better.** ### 7.2 (b) The PASS_TO_PASS regression rate — the KEY measure The empirical shadow of the blast radius, counted at the validation stage: **how many tests that passed before the edit fail after it, per generated patch.** TDAD's template is 562 → 155 failures (72% fewer, regression rate 6.08% → 1.82%). This instrument is how the forward-looking stage is shown to be *working*: the downstream broken-test count **falls** when the stage is enabled, because the targeted validation set (§3.5) and the dependency-consistent batches (§3.8) catch breakage *before* submission. **This is the KEY measure — a recall/precision win that does not lower the regression rate is not credited.** ### 7.3 (c) The oracle-impact diagnostic Each evaluated task runs repair **twice** on the harness's own weak model (`deepseek-v4-pro`): once with the **oracle** impact set (the true co-changed files supplied) — the *upper bound* — and once with the **computed** impact set. The **gap** (oracle-repair regression rate vs computed-repair regression rate) is the **impact ceiling**: the portion of downstream breakage recoverable by improving impact analysis alone. A recall/precision win that leaves this gap unchanged is **not** credited — the stage must *close* the gap, not merely reorder within it. ### 7.4 (d) The hidden-channel probe A curated corpus of deliberately hidden propagation: **string-keyed references** (attribute lookups whose name is computed at runtime), **duck-typed call sites**, reflection-based instantiation, and registry-driven dispatch. Static analysis is **blind** to these by construction, so the probe measures not whether the analyzer *finds* them (it cannot) but whether **git co-change (§3.4) catches them** — and, failing that, whether the stage **flags the region as un-analyzable** rather than silently reporting LOW-RISK. Report: `co_change_recall_on_hidden` (fraction of hidden edges recovered by §3.4) and `unanalyzable_flag_rate` (fraction of hidden regions the stage honestly flags). A stage that confidently reports a false negative is scored down; one that reports "this boundary is invisible to me" is scored up. ### 7.5 (e) End-to-end regression guard 4. **Resolve rate** = `resolved / total` (spec 01's score). 5. **Cost** = `$ / instance` (model spend only; deterministic stages are ~free). 6. **Tokens** = `tokens / instance`. ### 7.6 Decision rule (explicit thresholds) Run candidate-vs-current on the *identical* sample with the *identical* model (`deepseek-v4-pro`) and hardware (this box). The candidate **wins iff**: ``` PASS_TO_PASS regression rate strictly drops (the KEY measure, must improve) AND impact recall strictly improves (≥1 of recall / distance-to-co-changed-file up) AND impact precision does NOT regress (candidate ≥ current) AND resolve_candidate ≥ resolve_current (no end-to-end regression) AND cost_candidate ≤ 1.5 × cost_current AND tokens_candidate ≤ 1.5 × tokens_current ``` The regression rate must **strictly drop** and impact recall must **strictly improve**; precision and resolve must not regress; cost and tokens are bounded at 1.5×. A candidate that improves recall but leaves the regression rate flat is **not adopted**. If the candidate wins, it replaces the current path (edit-without-projection); if it loses, the build tree is kept for the record and the live harness is unchanged. **Ablation rule (mechanical).** Each evidence channel is removed in isolation — structural walk (§3.3), git co-change (§3.4), test-impact (§3.5), lexical soft-rank (§3.6) — everything else held fixed, and the delta measured on impact recall/precision **and** the regression rate. A channel is **deleted** iff its removal changes none of recall / precision / regression rate beyond noise (`Δ = 0`, or `|Δ| ≤ 5%` where continuous). Applied first to the weakest a-priori case (the lexical soft-rank), then to any other channel — regardless of how sensible it looks. The paper's "a stage may be removed when it does not move the honest metrics" rule, made mechanical. --- ## 8. Guardrails & Reversibility **Revocability.** - The build lives entirely in `build/06-dependency-impact-analysis/` (a separate tree); it never writes into the live harness's source. - A **feature flag** (env var `IMPACT=impact06|current`, default `current`) routes the A/B; the flag alone switches between the new impact stage and the current "edit-without-projection" path — no code deletion required to revert. - `ab.py` is a read-only driver: it *measures* the current harness, it never modifies it. Running old-vs-new is always possible in the same session, same repo, same model. **Invariants (must hold, checked at the end of every phase).** 1. **No new `.md` files outside this folder** (`projects/self-improvement/{papers,specs,build}`). Everything else is code or DB rows. 2. **No change to the live harness** until the §7 A/B gate passes. The current tool surface stays byte-identical during the entire build. 3. **Impact traversal is change-type-aware — never a blanket N-hop walk.** The walk (§3.3) takes the relation set selected by the change type (§3.2), never a uniform hop count. A change-type-blind `walk_all_edges(3)` is a rejected change, not a tuning knob; the escape-gate (§3.2) further narrows method-body traversal. 4. **The LLM reads the map, never code blobs.** The only LLM input is the §3.9 DAG + §4 signatures; no statement bodies, no full files, no line-span diffs. Any phase that injects a body into the model-facing text is rejected outright. 5. **The graph is patched incrementally, never rebuilt whole.** After an accepted edit, only the changed file's nodes/edges are re-parsed and patched into `recon.db` (component 03's `sha256`-keyed invalidation); the graph is never reconstructed from scratch. A full rebuild is a rejected change. 6. **Temporal consistency — every intermediate state valid.** The batch ordering (§3.8) and the DAG edge direction (§3.9) keep the repository valid after *every* step: the seed/signature change precedes its callers, never the reverse. An unordered batch or a reversed cause edge is a rejected deliverable. 7. **Deterministic stages stay deterministic**: stages (a)–(i) of §3 are D; the L column holds exactly one entry (the §4 next-edit decision), reached once, never re-prompted on parse failure. Any phase that introduces an LLM into a D node (classification, walk, ranking, triage, batching) is rejected outright. 8. **Anti-leakage always on**: git co-change (§3.4) and any historical signal use pre-date commits only; a post-date or overlapping commit in the recall set is a rejected deliverable. --- *End of spec. This document is the build contract; the next document records the A/B outcome.*