Summary / 10. Editing mechanism / Document
Spec 2026-09-05
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Build Spec 10 — Editing Mechanism (the ≈80% fraction) > Derived from `papers/10-editing-mechanism.md`. This document drives the build of the > **editing mechanism**: the write-path stage in which an intended change — already chosen by > component 09's candidate generation — is reliably transformed into repository bytes. It sits > immediately after component 09 (patch/code generation, which decided **WHAT** should change) and > before the components that execute, validate, and repair the landed result (11 and 17–18). It is > the **WRITE PATH**: generation decides *what*, editing decides *whether it lands* — whether the > right bytes are replaced, at the right place, once, with nothing collateral touched, and with a > repository that still parses and still lints afterward. The two stages stay distinct because they > fail differently: generation mispicks semantics, editing mis-applies them. This spec translates the > paper's design into exact scope-ladder classes, an exact representation-selection ladder, exact > schema-constrained adjudication caps, exact mechanical-application and uniqueness rules, exact > parse/lint-gate mechanics, an explicit **collateral-change** gate, exact commit-and-anchor-refresh > rules, exact L-column caps, exact A/B instruments and thresholds, exact ablation mechanics, and an > exact build sequence — no further research required. Where the paper says "≈80%", this spec pins > *which* 80%. --- ## 1. Purpose & Scope Build a **reversible editing stage** — a deterministic-first, apply–verify–revert transformation service (edit-scope classification → edit-representation selection → mechanical edit application → uniqueness/parse/lint gate → collateral-change check → commit + anchor refresh) wrapped around a **single, schema-constrained, ambiguity-only weak-LLM adjudication call**, as a drop-in *write-path* stage that converts component 09's candidate change — the semantic content of the intended modification — into the **validated, parseable, uniquely-targeted, in-scope repository state** handed to the downstream execution and validation components (11, 17–18). It is distinct from its predecessor in *kind*, not merely in order: component 09 decided **what** the change ought to mean; editing is where that decision is turned into safe repository bytes, and everything that performs the byte-level transformation, verifies it, and commits it is deterministic software on CPU. The paper's central thesis governs the scope: **editing is a deterministic transformation service behind the model, not free-form repository manipulation.** The overall target is the paper's full 100% spec. **Round 1 lands the expected ≈80% of it**: all six deterministic stages (scope classification, representation selection, mechanical application, the uniqueness/parse/lint gate, the collateral- change check, and commit + anchor refresh) plus the *one* ambiguity-only representation-adjudication call — targeting the reachable fraction of the design where the L column holds exactly one entry and every other step is deterministic software on CPU. The heavy learned / trained-artifact / extra-model- call / GPU pieces (CODESTRUCT's full structured action space across languages, a GRPO-trained adaptive editor model, BlockDiff/FuncDiff structure-aware diff formats, a trained small applier model) are **not** built in Round 1 and are **not excluded** — they are the content of later rounds that climb from that ~80% landing toward 100% (see §2). Round 1 is kept only if it wins a measured A/B test on the *honest* editing metrics — led by **apply-success rate** (which must STRICTLY IMPROVE) and **collateral-change rate** (which must STRICTLY DROP), with solve/cost/tokens guarded end-to-end (§7) — where a green apply that leaks out of scope is scored as a failure, never as a win. --- ## 2. Round 1 vs later rounds (toward the 100% target) The goal is the paper's full **100%** spec. **~80%** is only the expected **first-round landing**; later rounds climb from there. Every element from the paper is classified below by *which round* builds it. - **BUILD-NOW = the Round 1 build** — the expected ≈80% deterministic landing this spec implements and A/B-tests (rows 1–8). - **DEFERRED = later rounds** — the trained-artifact / extra-model-call / GPU / parser-coverage pieces (rows 9–12) that Round 1 intentionally does **not** build *yet*, because each adds a trained model, a GPU/training dependency, a full per-language parser surface, or an extra model call against the L-column invariant. They are **not excluded from the project** — they are the content of later rounds, the climb toward 100%, gated by the same §7 A/B protocol once the Round 1 core is in place. | # | Component (paper §) | Classification | |---|---|---| | 1 | (a) Edit-scope classification — deterministic classification of the intended modification into the scope ladder (token/expression, statement, function/entity, file, multi-file) over the intent + the repository structure (reuse component 03's recon graph + AST) | **BUILD-NOW** | | 2 | (b) Edit-representation selection — deterministic ladder: lowest-complexity representation capable of the classified scope; search/replace default, AST/entity transform where the stdlib parser supports the entity, bounded rewrite the gated exception (P4) | **BUILD-NOW** | | 3 | (c) Representation adjudication — **the single learned call**, see §4 | **BUILD-NOW** (the L step; ONE call, ambiguity-only) | | 4 | (d) Edit application — deterministic transformer outside the LLM: exact-match search/replace with a uniqueness check (reject zero/multiple matches before writing), AST/entity transform against the parser, or bounded whole-entity rewrite; reparse (P5) | **BUILD-NOW** | | 5 | (e) Uniqueness / parse / lint gate — reject ambiguous targets (zero/multiple matches), non-parsing results, non-linting results; atomic rollback on rejection (P5) | **BUILD-NOW** | | 6 | (e2) Collateral-change check — verify the post-apply diff falls strictly inside the declared scope; any bleed is a rejected edit (P6) | **BUILD-NOW** | | 7 | (f) Commit + anchor refresh — snapshot/commit the transaction; refresh anchors (line numbers, byte offsets) so no later edit reuses stale coordinates | **BUILD-NOW** | | 8 | A/B harness + the single ambiguity-only adjudication call (§4) | **BUILD-NOW** (measured by §7, never a default that runs alone) | | 9 | CODESTRUCT's **full structured action space** — readCode/editCode over named AST entities across languages, replacing text-span editing everywhere | **DEFERRED** — needs a parser per language beyond Python's stdlib `ast` (component 03's tree-sitter is flagged, not a full multi-language action surface); Round 1 applies AST/entity transforms only where the stdlib `ast` parser for Python already exists; gated by §7 on held-out instances once Round 1 is in place | | 10 | A **GRPO-trained adaptive find-replace / whole-file editor model** (SWE-Edit — GRPO on Qwen3-8B with an adaptive policy, +12.5pp edit success, 8B editor to GPT-5-nano parity; edit-success 93.4%→96.9%, cost −17.9%) | **DEFERRED** — a trained-weights artifact + GPU training that would replace or sit beside deterministic stages (b) and (d) that Round 1 already implements; gated by §7 ablation once it exists | | 11 | **BlockDiff / FuncDiff structure-aware diff formats** + per-sample adaptive switching (To Diff or Not to Diff — block-level rewrites aligned to tree-sitter AST nodes, AdaEdit switches per-sample) | **DEFERRED** — needs tree-sitter alignment + training; Round 1 represents structure via the stdlib `ast` transform path, not a learned diff format; gated by §7 | | 12 | A **trained small applier model** (Cascaded Code Editing — sketch/apply split, a specialized model applies the large model's sketch) | **DEFERRED** — a trained applier replacing deterministic software as the applier; the paper's own sketch/apply split is realized in Round 1 with deterministic software in the applier role; gated by §7 once a trained applier exists | **Design decision: DETERMINISTIC-FIRST, LEARNED-EXCEPTIONAL.** Round 1 does not put a model anywhere in the write path by default. It builds the full deterministic pipeline (rows 1–2, 4–7) and the **single** ambiguity-only adjudication call (§4) that is the one genuinely learned decision in the stage. The deterministic path is the default; the L step fires only on genuine ambiguity. The CODESTRUCT full action space, the GRPO-trained editor, the structure-aware diff formats, and the trained applier are all candidates for later rounds precisely because each would add a trained artifact, a GPU dependency, a full multi-language parser surface, or a *second* model to the pipeline, break the one-entry L column, or replace a deterministic stage that the ablation (§7) may already prove sufficient. Any later-round candidate must clear the same bar as in earlier components: beat the deterministic stage on the honest metrics (§7), on held-out instances, at a defensible cost, or be rejected. The cost of the deterministic choice is paid once, in the schemas (§3); the cost of a misplaced learned call is paid every edit, in L-column violations and budget spent on a mechanism the deterministic stages may already beat. --- ## 3. The Deterministic Pipeline (D) The write path is a sequence of stages through the landing of an edit as a validated repository state. Every node is labeled D (deterministic software) or L (a learned model); the invariant that governs the whole is: *the L column contains exactly one entry — a single, schema-constrained, representation-adjudication call that fires only when the deterministic machinery leaves a genuine ambiguity — and every other step is deterministic software on CPU* (paper §4). The deterministic path is the default; the single learned call is the exception. The flow, from edit intent to committed repository state: ``` EDIT INTENT (09 candidate change: the semantic modification over a known location, its target span, and the repository structure from component 03's recon graph + AST) → (a) edit-scope classification [D] → a scope label + the affected entity/span → (b) edit-representation selection [D] → lowest-complexity representation capable of the scope → (§4) representation adjudication [L — the ONE entry, fires ONLY on genuine ambiguity] → (d) edit application [D] → the repository change (applied outside the LLM) → (e) uniqueness / parse / lint gate [D] → the applied change passes (atomic rollback on rejection) → (e2) collateral-change check [D] → the diff stays strictly inside the declared scope → (f) commit + anchor refresh [D] → the transaction committed; anchors refreshed ``` The repository change is applied to an authoritative snapshot for **every** edit — the intent is read from the current candidate change, never accumulated across edits, and anchors are refreshed after every mutation so no edit reuses stale coordinates (§8 invariant 7). All artifacts land in `build/10-editing-mechanism/`. Nothing writes into the live harness. The model-facing representation and the developer-facing representation are **not** the same thing: the model emits an LM-friendly operation (or the deterministic stages derive it from the scope), and the harness generates the developer-facing unified diff **afterward** (P2). The weak model never touches repository bytes directly; application, verification, gating, and commit are deterministic software on CPU (P5). ### 3.1 (a) Edit-scope classification (D) **Input:** the edit intent inherited from component 09 — the candidate change's semantic content, its declared target entity and `[start_line, end_line]` span over a known file — **and** the repository's structure from component 03 (the recon graph + the parsed AST of the target file). **Output:** one **scope label** from the fixed ladder **`{token_expression, statement, function_entity, file, multi_file}`** plus the **affected entity/span** that the change touches. **No language model runs here.** This is deterministic classification over the change's own shape and the repository's structure, mirroring the readCode/editCode entity boundary of CODESTRUCT (paper §2.3, P4): how much of a single syntactic unit the change touches, whether it crosses a unit boundary, whether it spans a whole entity or file — all read from the AST the harness already holds, never re-derived by the model. **Scope-classification rule (exact — deterministic, over the intent and the repo AST):** 1. Resolve the change's span and target entity against component 03's recon graph and the target file's AST. 2. Count the distinct syntactic units (AST nodes) the change's span intersects, and measure the span extent in contiguous lines. 3. Classify onto the ladder by this exact order (first true rule wins): | Scope label | Exact predicate | |---|---| | `multi_file` | the change touches **> 1 file** (from the recon graph) | | `file` | the change's span covers a whole file, **or** intersects **≥ 3 distinct top-level entities** in one file | | `function_entity` | the change stays within **one** function/entity (or ≤ 2 adjacent entities) **and** spans **> 15 and ≤ 60 contiguous lines**, or crosses a statement-block boundary within one entity | | `statement` | the change stays within a single enclosing block (one function/entity) **and** spans **≤ 15 contiguous lines** and touches ≤ 3 sibling statements of that block | | `token_expression` | the change is confined within a **single expression** (no statement boundary crossed) **and** spans **≤ 3 contiguous lines** | **Boundary-crossing flag:** whenever a change classified `function_entity` or `statement` ends within **≤ 1 line** of an enclosing entity boundary, or touches two entities at a shared boundary, the classifier additionally emits an **`at_unit_boundary: true`** flag. This flag is the *only* input that can make step (b) genuinely ambiguous and thereby arm the single L adjudication (§4) — it is the deterministic signal for where a model's judgment may beat the deterministic tie-break. **Output schema (exact):** ```json { "scope": "token_expression|statement|function_entity|file|multi_file", "entity": "str", // the resolved entity/entities the change touches (03) "file": "str", // absolute path "span": [int, int], // [start_line, end_line] of the change site (09) "at_unit_boundary": bool } // true only when within ≤1 line of an entity boundary or across two entities ``` **Caps:** one scope label per edit, fixed deterministically before any representation is selected; the classification reads the span + the repo AST and never asks a model which scope "feels" right. The classification is logged (label + boundary flag) for the §7 representation-robustness and ablation instruments. ### 3.2 (b) Edit-representation selection (D) **Input:** the scope label + affected entity/span from (a). **Output:** the **edit representation** — the lowest-complexity representation capable of expressing the classified scope, from the fixed set **`{search_replace, ast_transform, bounded_rewrite}`**. **No language model runs here.** This is P4 executed structurally (paper §3 P3/P4, §2.2, §2.5): the edit format is a measured variable, not a fixed convention, and the default for a weak-model harness is **search/replace over an exact-match span** — the cheapest representation that applies and reparses reliably (paper §2.1, §2.5, §2.8) — with **AST/entity transform preferred wherever the parser supports the target entity** (per P4, structure buys most where models are weakest; §2.3) and **bounded rewrite admitted only when the change is genuinely structural and too pervasive for localized hunks** (per §2.4). **Format-selection rule (exact — deterministic, a ladder from lowest to highest complexity):** 1. **`search_replace` (default):** the candidate change is expressed as an ordered list of `{search, replace}` block pairs over exact-match spans. This is the default for every `token_expression` and `statement` classification (paper §2.1 — the ecosystem's converged `str_replace_editor` default). 2. **`ast_transform`:** chosen when the classified scope is `function_entity` or `file` **and** the language parser supports the target entity — Round 1: the stdlib `ast` parser for Python only. The change is expressed as an entity operation (locate the named entity, transform its body) rather than a text span (paper §2.3 — structure-aware editing buys most where models are weakest). 3. **`bounded_rewrite` (gated exception):** chosen **only** when the classified scope is `file` or `function_entity` **and** the change is genuinely structural/pervasive — concretely, a span **> 40 contiguous lines** that cannot be expressed as localized hunks, or a change that touches the entity's structural skeleton (signature, block structure) rather than its interior (paper §2.4 — pervasive changes may be cheaper and more reliable as a whole-entity rewrite). 4. `multi_file` is decomposed into per-file changes; each per-file change is classified and formatted by rules 1–3 independently. **Deterministic default (no L):** the ladder always resolves to a single representation. The only case it leaves open is when rules 1 and 2 are **both** viable for the same change — a `function_entity`/`file` scope with `at_unit_boundary: true` where the change could be expressed either as a localized search/replace *or* as an entity transform of the whole unit, and scope classification is genuinely uncertain about crossing the unit boundary. In that single case the choice is delegated to the §4 adjudication (the L entry). Everywhere else the ladder's answer is final and no model runs. **Caps:** one representation per edit, fixed by the classification before any application; the choice is logged (scope + chosen representation + whether the L adjudication fired) for the §7 representation-robustness probe and ablation instruments. Bounded rewrite is **never** the default and **never** chosen for a localized edit. ### 3.3 (c) Representation adjudication (L — the ONE entry) The single learned node; the full spec is §4. **Input:** the classified scope (a) and the set of representations the deterministic ladder (b) left genuinely ambiguous. **Output:** ONE representation choice from the fixed set `{search_replace, ast_transform, bounded_rewrite}`. It fires **ONLY** when the deterministic ladder of (b) leaves the choice genuinely ambiguous — the lone `at_unit_boundary: true` case where a change is expressible either as a localized search/replace or as an entity transform. This is the deliberate **only** learned step because every other operation in the write path is something deterministic software does more reliably and more cheaply than a weak model (P1–P7, §2.5). It is deliberately narrow — choose among a small enumerated set of representations — because that is the one case where the two viable representations differ in reliability enough that adjudicating is worth a model call, and even then the model is not touching repository bytes. **Inline caps (the governing numbers; §4 is authoritative):** ONE call; `temperature = 0`; `max_completion_tokens = 256`; fires **only** on genuine ambiguity (default = deterministic); output schema-constrained to one enumerated value. Parse failure → **deterministic fallback (search/replace)** — never a re-prompt, never a second call, never a model write of repository bytes. --- ### 3.4 (d) Edit application (D) **Input:** the chosen representation from (b)/(§4). **Output:** the repository change applied **outside the LLM** exactly as the representation specifies — locate, verify, transform, reparse — or a **reported failure**. **No language model runs here.** This is P5 made concrete (paper §3 P5, §2.1, §2.9): the harness, not the model, performs the byte-level transformation, so a correct semantic decision is not lost to a formatting slip, and the developer-facing unified diff is generated by the harness **afterward** rather than demanded of the model (P2). Application is deterministic and auditable: the **same representation always produces the same repository change or a reported failure**. **Procedure (exact, deterministic, by representation):** 1. **`search_replace`:** for each `{search, replace}` pair, find the `search` block in the target file's target span; verify it matches **exactly once** (a zero-match or multi-match is a reported failure, never an ambiguous apply); replace at the uniquely matched location; reparse. 2. **`ast_transform`:** locate the named entity in the file's parsed AST, apply the semantic transformation to that entity's subtree, regenerate the source for the entity, splice it back; reparse the whole file. 3. **`bounded_rewrite`:** replace the whole named entity (function/entity or file) with the complete bounded replacement; reparse the whole file. **Reparse rule (exact):** every apply is followed by a reparse of the modified file with the language's parser (stdlib `ast` for Python in Round 1). A reparse failure is a **reported failure** handed to the gate (e), which rolls the repository back to its pre-application snapshot. **Caps:** application is byte-exact and idempotent per representation; each apply is against the same pre-edit snapshot of the repository (never against a prior edit's applied state, so edits remain independent for validation). A representation that cannot be located or uniquely matched is rejected with a structured failure reason (logged for the §7 apply-success and representation-robustness instruments). The model never emits repository bytes as free-form text — the transformer owns the bytes. ### 3.5 (e) Uniqueness / parse / lint gate (D) **Input:** each applied change from (d). **Output:** the **applied changes that are uniquely targeted, parse, and lint**; a change that fails any check is **rejected and the repository is rolled back atomically to its pre-application state**. **No language model runs here.** This is P5's apply–verify– revert discipline executed structurally (paper §3 P5, §2.1, §2.9): the gate rejects a malformed or ambiguous edit at the cheapest possible stage, before the expensive downstream validation (11, 17–18) ever sees it, so a malformed edit never persists and the agent never reasons over a corrupted file. **Procedure (exact):** for each applied change, run three deterministic checks in order: 1. **Uniqueness:** a target that matched **zero** times, or **more than once**, is a reported failure — the edit is rejected, never applied ambiguously. 2. **Parse:** reparse the modified file (stdlib `ast` for Python in Round 1; component 03's parser where flagged). A file that **fails to parse** is rejected. 3. **Lint:** run the deterministic lint checks only (component 03's existing lint surface; Round 1 uses no learned linter). An **error-level hard-lint** finding (one that changes the program's meaning — undefined name, unused import) rejects; warnings do not. Rejection is always **apply-then-verify with atomic rollback**: the repository is returned to its pre-application snapshot on any rejection, so a malformed or ambiguous edit never persists and never reaches downstream validation. Every rejection is logged with its reason (for the §7 apply-success and representation-robustness instruments). ### 3.6 (e2) Collateral-change check (D) **Input:** the post-apply diff of the accepted change from (e) **and** the localized scope declared in (a) — the entity, file, and span. **Output:** a pass/fail verdict: the diff **must fall strictly inside the declared scope**. Any bleed — lines outside the target entity/span changed, an unrelated symbol touched, a whole-file rewrite that reformatted surrounding code, a fuzzy search/replace that matched the wrong occurrence — **flags the edit as a failure and rolls it back**. **No language model runs here.** This is P6 executed as its own named, explicit stage (paper §3 P6, §2.6): an edit is not reliable merely because the intended change landed; it is reliable only if *nothing else moved*. The Edit-But-Verify audit (paper §2.6) shows the benchmarks used to compare edit tools mostly cannot see out-of-scope modification — 59% of low-coverage suites would not detect it, 56% of tests scope exclusively to the edited code — precisely because editing tools routinely introduce it, and whole-file rewrites and fuzzy search/replace are the mechanisms that leak. This gate is what makes the word "reliably" in this stage's mandate measurable: it is the single instrument that a solve rate cannot see. **Procedure (exact, deterministic):** 1. Compute the unified diff of the repository against its pre-application snapshot for this edit (the harness generates the diff; the model never does). 2. For each changed hunk, resolve the affected entity and span against the declared scope from (a). 3. **Pass** iff every changed line falls inside the declared entity and span, **and** the set of touched entities equals the declared affected entity/entities. **Fail** otherwise — any added, removed, or reformatted line outside the declared scope, or any touched symbol outside the declared entity set, is a bleed and the edit is rejected. **Caps:** the check runs **after** the uniqueness/parse/lint gate (e) and **before** commit (f); it rejects on any bleed, deterministically; every rejection is logged with its reason (for the §7 collateral-change-rate instrument). A green apply that leaks out of scope is scored as a **failure**, never accepted. ### 3.7 (f) Commit + anchor refresh (D) **Input:** the accepted, in-scope change from (e2). **Output:** the transaction **snapshotted and committed**, and **anchors refreshed** so no subsequent edit reuses stale line numbers or byte offsets. **No language model runs here.** This is the closing stage of the write path (paper §4 stage 7): line numbers are the weakest anchor — the file changes under the agent's own edits, and indices from an earlier round are lies by a later one — so every landed edit refreshes the coordinate space the next edit will address. **Procedure (exact, deterministic):** 1. **Snapshot/commit:** record the pre-edit snapshot and commit the accepted change as a discrete transaction (an edit log entry is appended; the commit is diffable for the audit). 2. **Anchor refresh:** after the commit, re-resolve every live anchor (line numbers, byte offsets) for the repository against the new state, so no subsequent edit reuses a coordinate from before this mutation. **Caps:** exactly one commit per accepted edit; anchors are refreshed after **every** mutation, never batched; the commit is atomic with the preceding gates — a change is committed iff it passed (e) and (e2). The edit log is diffable as it changes, so an editing regression (a rising apply-failure rate, a collateral-change leak, a drift toward whole-file rewrites) is visible in the produced diffs long before it surfaces as a failed downstream solve (paper §5). **The L column holds exactly one entry.** Stages (a), (b), (d), (e), (e2), and (f) are all deterministic software on CPU. The single learned node is stage (c), the representation-adjudication call (§4), which fires only on genuine ambiguity and never touches a repository byte. Any proposed addition that would put a language model elsewhere in the write path — a learned representation chooser that runs by default, a model-in-the-loop rewriter of a failed apply, a model that emits unified diffs directly — is a regression against P1–P7 and the L-column invariant, and must clear the same bar as in earlier components: beat the deterministic stage on the honest metrics of §7, on held-out instances, at a defensible cost, or be rejected (paper §4). --- ## 4. The LLM Step (L, ONE entry) The **single** learned node in the entire stage: **representation adjudication**. The weak model emits ONE choice from the fixed enumerated set `{search_replace, ast_transform, bounded_rewrite}` — the representation in which the change will be applied — **only** when the deterministic ladder of (b) leaves the choice genuinely ambiguous, at temperature zero, in one call (paper §4 step 3). This is the sole entry in the L column; every upstream and downstream step is deterministic software on CPU (paper §4). It fires **only on ambiguity**: the default is deterministic, and this step is the exception that the deterministic ladder cannot resolve. It is deliberately narrow — choose among a small enumerated set of representations — because that is the one case where the two viable representations differ in reliability enough that adjudicating is worth a model call (paper §2.5), and even then the model is not touching repository bytes. **Firing condition (exact):** the step fires **iff** stage (a) emitted `at_unit_boundary: true` **and** stage (b) found the ladder genuinely ambiguous — that is, **both** `search_replace` and `ast_transform` (or `bounded_rewrite`, when the change is pervasive) remain capable of expressing the classified scope, so the deterministic tie-break would have to choose arbitrarily between two reliable representations. In every other case the deterministic ladder's answer is final and **no model call is made**. The step never fires to explore, to apply an edit, to reparse, to verify, or to touch a repository byte. **Prompt inputs (assembled deterministically, in this exact order):** 1. **`scope`** — the classification from (a), verbatim: the scope label, the affected entity, the file, and the span. 2. **`candidate_representations`** — the ambiguous set left open by (b): the representations that are each capable of the change (e.g. `["search_replace", "ast_transform"]`). 3. **`edit_intent`** — the semantic content of the candidate change from 09, in brief (bounded by the caps below), so the adjudicator can judge which representation the change's structure favors. **Input → output schema (exact):** ```json { "representation": "search_replace | ast_transform | bounded_rewrite" } ``` The output is schema-constrained to a **single enumerated value** — the model cannot drift into free-form prose, an edit, or anything the deterministic applicator (d) cannot consume. **Caps:** **ONE call**, `temperature = 0` (a build constant, configurable via the build config), `max_completion_tokens = 256`, fires **only** on genuine ambiguity (default = deterministic). The output is schema-constrained to one of the three enumerated values; the harness, not the model, decides what the choice means. `temperature = 0` is mandatory because this is a **decision**, not a sampling problem: there is exactly one correct answer per ambiguous case, and diversity is not wanted. **Parse-failure fallback (exact):** any JSON parse failure, schema violation, or out-of-set value is a **no-op**: the stage falls back to the **deterministic default `search_replace`** and hands control back to the deterministic path. **NEVER a re-prompt in a loop, NEVER a second call, NEVER a model write of repository bytes.** The L column is reached **once** per edit; a failed parse is not retried with a "please try again" call. This is the mechanical enforcement of the one-entry L column and the anti-over-prompting rule (§8 invariant 3). **Re-invocation on the next edit (exact):** representation adjudication is re-considered **only** when the *next* edit assembles its own scope classification (a) and finds the ladder ambiguous again; each invocation is scoped strictly to that edit's ambiguity and re-uses none of a prior edit's choice. It remains the **single conceptual learned entry** — a bounded, once-per-ambiguous-edit call — **never a model that runs over the whole write path** and never a loop that re-derives what the deterministic stages already settled (paper §4). **Invariant 3 (§8): the L column holds exactly one entry.** --- ## 5. Implementation Order Eight phases, each independently testable and reversible. Each lands in `build/10-editing-mechanism/` as a standalone importable module. Rollback = delete/flag-off that phase's module; nothing earlier breaks because phases only add modules and never mutate live harness code. | Phase | Delivers | Isolated test | Rollback | |---|---|---|---| | **1. Edit-scope classification** | `scope.py` (§3.1 classify the intent + repo structure into the scope ladder; boundary-crossing flag) | Feed edit intents + component 03 recon/AST fixtures: a ≤3-line single-expression change classifies `token_expression`; a ≤15-line single-block change `statement`; a ≤60-line single-entity change `function_entity`; a whole-file or ≥3-top-level-entity change `file`; a >1-file change `multi_file`; a change within ≤1 line of an entity boundary or across two entities sets `at_unit_boundary`; assert **no LLM consulted** (probe the LLM client to raise). | Delete `scope.py` (pure read). | | **2. Edit-representation selection** | `represent.py` (§3.2 ladder: search/replace default, ast_transform where stdlib `ast` supports the entity, bounded_rewrite the gated exception; multi-file decomposition) | Feed classified scopes: `token_expression`/`statement` select `search_replace`; a `function_entity`/`file` change with Python-`ast` entity support selects `ast_transform`; a >40-line or structural-skeleton change selects `bounded_rewrite`; **no** localized edit ever selects `bounded_rewrite`; `multi_file` decomposes per file; the only ambiguous case left open is `at_unit_boundary: true` with two viable representations; assert **no LLM consulted**. | Delete `represent.py`. | | **3. Representation adjudication (L)** | `adjudicate.py` (§4 firing condition + ONE call + output schema + caps + parse-failure fallback) | Mock the LLM client: assert the step fires **only** on `at_unit_boundary: true` **and** a genuinely ambiguous ladder; every other case → deterministic with **no call**; the call is **one**, `temperature=0`, `max_completion_tokens=256`, output schema-constrained to one enumerated value; a malformed / out-of-set response → **deterministic fallback `search_replace`** with **no second call, no re-prompt loop**; a valid choice passes through to (d). | Delete `adjudicate.py` (no live-harness change). | | **4. Edit application** | `apply.py` (§3.4 locate + unique-match verify + transform + reparse; per representation; same representation → same change or a reported failure) | Feed representations over a fixture file: a `search_replace` block found uniquely applies; a search matching zero or >1 times → reported failure (never ambiguous apply); an `ast_transform` applies to the named entity and regenerates its source; a `bounded_rewrite` replaces the whole entity/file; a reparse failure → reported failure; application is against the pre-edit snapshot; assert determinism (same representation → same result) and **no LLM consulted**. | Delete `apply.py`. | | **5. Uniqueness / parse / lint gate** | `gate.py` (§3.5 uniqueness + deterministic parse + lint checks; apply-then-verify with atomic rollback to pre-application state) | Feed applied changes over a fixture: a zero- or multi-match target is rejected; a file that fails to parse is rejected and rolled back; an error-level hard-lint (undefined name, unused import) rejects; a warning does not; every rejection is logged with its reason; the accepted set parses by construction; assert **no LLM consulted**. | Delete `gate.py`. | | **6. Collateral-change check** | `collateral.py` (§3.6 diff-vs-declared-scope: the diff must fall strictly inside the declared entity/span; any bleed → rejected edit) | Feed applied changes: a change whose diff stays inside the declared scope passes; a whole-file rewrite that reformats unrelated lines fails; a fuzzy search/replace matching the wrong occurrence fails; a sibling-symbol touch fails; a green-but-leaking apply is scored a failure; every rejection is logged with its reason. | Delete `collateral.py`. | | **7. Commit + anchor refresh** | `commit.py` (§3.7 snapshot/commit the accepted transaction; refresh anchors so no later edit reuses stale coordinates) | Apply two edits to a fixture: after the first commit, anchors are refreshed so the second edit addresses the post-first-edit coordinates, not stale ones; exactly one commit per accepted edit; a change is committed iff it passed (e) and (e2); the edit log is diffable. | Delete `commit.py`. | | **8. A/B harness** | `ab.py` (feature-flag router + apply-success-rate capture + collateral-change-rate capture + representation-robustness capture + solve/cost/tokens guard + ablation runner + collateral-change probe + decision) | End-to-end on the §7 sample: `ab.py` runs gated-editing-vs-ungated-baseline on the §7 sample, emits apply-success rate, collateral-change rate, representation robustness, end-to-end solve/cost/tokens, the ablation report, the collateral-change probe, and the §7 decision. | Feature flag `off` → live harness untouched; `ab.py` is read-only. | The A/B runner (Phase 8) is the only phase that ever *invokes* the current harness — and only to measure it, never to modify it. --- ## 6. Dependencies & Supply-Chain Per Piper's rule: nothing is installed unless >72h old AND (≥10 stars OR deep-scrutinized). This build is **zero-install for Round 1 (Python-only, stdlib + in-repo)**: | Dependency | Why | Status / age-stars | Install needed? | |---|---|---|---| | stdlib `json`, `re`, `difflib`, `ast` | scope-ladder thresholds (§3.1), search/replace matching + unified-diff generation (§3.4, §3.6), parse gating (§3.5), AST/entity transforms for Python (§3.2, §3.4) | CPython stdlib. | **No** | | Component 03 recon graph + AST | §3.1 scope classification over the repository structure; §3.2 entity support; §3.4 AST/entity transforms | In-repo, built by spec 03 (reuse the recon graph + parsed AST; no new parser). | **No** | | Component 09 candidate change (the edit intent: target entity + span + semantic content) | §3.1 the input the write path transforms | In-repo, built by spec 09. | **No** | | Component 03 lint surface | §3.5 the deterministic lint checks | In-repo, already vetted. | **No** | | Component 05 localization (`target.span`) | §3.1 the span grounding (via 09's intent) | In-repo, built by spec 05. | **No** | | Harness weak model (`deepseek-v4-pro`) | §4 the single ambiguity-only adjudication call | The harness's own model, already vetted. | **No** | **Preferred path: zero-install.** Round 1 is Python-only and deterministic: the write path reuses component 03's recon graph/AST and lint surface, component 05's span, and component 09's candidate change as its input; the single LLM call (§4) goes to the harness's own weak model. **No new package is installed for Round 1, and nothing is flagged for a later install** — the four DEFERRED rows (§2 rows 9–12: the CODESTRUCT full structured action space, the SWE-Edit GRPO-trained adaptive editor, the BlockDiff/FuncDiff structure-aware diff formats, and the Cascaded trained applier) are training/weights on known architectures or parser surface on component 03's flagged tree-sitter, not new third-party installs; any future model download, RL-training run, GPU dependency, or multi-language parser they imply must clear the 72h + star/scrutiny + sandbox-test gate before use, and is out of Round 1 scope entirely. --- ## 7. Acceptance Criteria — the A/B Protocol **Sample (concrete).** Reuse the **same 20 held-out SWE-bench-Verified instances as spec 01** (Python-only, single-repo, no-Docker reproduction). Held out = never seen by any tuning, prompt, or generation pass. Seeded RNG. **Fallback:** `coding-cortex/piperbench` sampled to 20 tasks, held out identically. **Baseline (concrete).** The **ungated raw-edit path** — the model emits and applies raw edits with no surrounding gates: no scope classification, no representation selection, no uniqueness check, no parse/lint gate, no collateral-change check, no anchor refresh (the paper's §5 naive baseline). The gated deterministic write path must beat *this* on the honest editing metrics below, and the ablation (7.4) removes editing mechanisms against those same honest metrics. **A green apply that leaks out-of-scope is scored as a failure, not a win** — the collateral-change instrument (7.2) is not an optional extra but the co-key measure. ### 7.1 (a) Apply-success rate — the KEY measure The fraction of intended edits that apply cleanly **and parse** — a unique target matched, the transformation applied, the result parsing and linting — the direct score of the application (d) and gate (e) stages (paper §6). A failed apply is a **HARNESS failure**, not a reasoning failure: it means the channel between a correct decision and the repository bytes is failing, exactly the edit friction Lita and SWE-agent attribute to interface, not reasoning (paper §2.1, §2.2). SWE-Edit's edit-success rate (93.4% rising to 96.9% with its decomposition) is the external reference for what a mature write path should show (paper §2.7, ref 8). **The gated write path must STRICTLY IMPROVE apply-success rate** versus the ungated baseline. **This is the KEY measure — a collateral-change or solve win that does not also strictly improve apply success is not credited.** ### 7.2 (b) Collateral-change rate — the co-KEY measure The fraction of applies whose resulting diff bleeds **outside** the localized scope declared before the edit — the Edit-But-Verify metric (paper §2.6). This is the instrument that makes P6 falsifiable: it catches the whole-file rewrite that silently reformats unrelated code, the fuzzy search/replace that matches the wrong occurrence, the entity transform that touches a sibling symbol. **The gated write path must STRICTLY DROP collateral-change rate** versus the ungated baseline. A harness with a low apply-failure rate but a high collateral-change rate is landing edits "successfully" while corrupting the repository, and only this instrument sees it (paper §6). **A green apply that leaks out-of-scope is scored as a failure** — never accepted as a win, regardless of how cleanly it parsed or how many downstream tests it satisfies. ### 7.3 (c) Solve rate / cost / tokens — the end-to-end guard The end-to-end guard: does the whole pipeline — including this stage — resolve tasks at an acceptable cost? This is deliberately not the primary instrument, because it cannot distinguish an editing failure from a validation or generation failure; it is the guard that ensures the editing mechanisms are not bought at the price of the pipeline's overall competence (paper §6). The gated path must hold **solve rate ≥ current (no regression)** at **cost ≤ 1.5×** and **tokens ≤ 1.5×** end-to-end. ### 7.4 Decision rule (explicit thresholds) Run gated-write-path-vs-current on the *identical* sample with the *identical* model (the harness's own weak model, `deepseek-v4-pro`) and hardware (this box). The gated path **wins iff**: ``` Apply-success rate STRICTLY improves (the KEY measure) AND collateral-change rate STRICTLY drops (the co-KEY measure) AND solve_gated ≥ solve_current (no solve-rate regression) AND cost_gated ≤ 1.5 × cost_current AND tokens_gated ≤ 1.5 × tokens_current ``` Apply success must **strictly improve** and collateral change must **strictly drop**; solve must not regress; cost and tokens are bounded at 1.5×. A green apply that leaks out-of-scope is counted against collateral change regardless of apply success. If the gated path wins, it replaces the current path (ungated raw edit); if it loses, the build tree is kept for the record and the live harness is unchanged. ### 7.5 Ablation rule (mechanical) The instrument that enforces the "never add an editing mechanism because it sounds sensible" discipline mechanically (paper §5, §6). Two passes, everything else held fixed, the delta measured on the honest metrics (apply-success rate and collateral-change rate, plus solve/cost/tokens): 1. **Remove each mechanism in turn** — the lint gate (apply without reparse/rollback), the collateral-change check (accept any diff that applies), the deterministic representation ladder (force whole-file rewrites, or force search/replace everywhere), the ambiguity L step (never adjudicate, always take the deterministic tie-break). 2. **Re-measure with the mechanism absent.** A mechanism is **DELETED** iff its removal does **not** degrade the honest metrics beyond noise (`Δ = 0`, or `|Δ| ≤ 5%` where continuous). A mechanism whose *addition* degrades the metrics is a demonstrated harm and is removed outright. Applied first to the weakest a-priori case (the ambiguity L step — the one learned call), then to any other mechanism — regardless of how sensible it looks. **Never add an editing mechanism because it sounds sensible.** The collateral-change check and the apply–verify–revert gate are the paper's P5/P6 defaults, but they are kept only if this ablation shows they earn their place. ### 7.6 Collateral-change probe (P6 load-bearing check) The instrument that verifies the collateral-change check (e2) and the gate (e) actually catch the leakage the audit says editing tools routinely introduce (paper §2.6): 1. **Whole-file-rewrite leak:** a whole-file rewrite that silently reformats unrelated lines must be **caught by the collateral-change check** and rejected — never accepted as a green apply. 2. **Wrong-occurrence match:** a fuzzy search/replace that matches the wrong occurrence must be **caught by the collateral-change check** and rejected — never accepted as a green apply. If either probe case is accepted as a green apply, the collateral-change check or the gate fails the probe and the mechanism is re-examined against 7.4. ### 7.7 Representation robustness (P4/P7 instrument) The per-model, per-change-type apply success across search/replace versus AST/entity transform versus bounded rewrite — the Diff-XYZ / Lita instrument (paper §2.2, §2.5), and the empirical heart of P7. It reports, for this harness's model and for each change scope in §3.1, which representation applies most reliably, so that the deterministic ladder (b) and the adjudication trigger (c) are tuned to evidence rather than to convention or to research averages computed on other models. It informs, but does not itself decide, the §7.4 gate. ### 7.8 Two honest caveats Two caveats close the section, inherited from the paper's §6. **First, the strong-model-transfer caveat:** the headline numbers of the paper's review — Lita's 19.8%→45.8% for GPT-4o and its near-vanishing gap by GPT-5, CODESTRUCT's +1.2–5.0pp with its 20.8-point GPT-5-nano outlier — were all obtained with models far stronger than this harness's target, and a weak model should not be expected to reproduce them; indeed the entire architecture (P3, P4, P5) exists precisely because a weak model underperforms on raw editing, and the apply-success and representation-robustness instruments are the ones that will show whether the deterministic stages are carrying that load. **Second, the held-out and memorization caution** inherited from the earlier components: a model that has memorized a repository's shape — or a benchmark's reference patches — will score well in-benchmark without the editing mechanism having done constructive work, so all editing metrics must be reported on held-out and out-of-benchmark repositories, exactly as the generation and localization metrics were. --- ## 8. Guardrails & Reversibility **Revocability.** - The build lives entirely in `build/10-editing-mechanism/` (a separate tree); it never writes into the live harness's source. - A **feature flag** (env var `EDIT=edit10|raw`, default `raw`) routes the A/B; the flag alone switches between the new editing stage and the current ungated raw-edit path — no code deletion required to revert. - `ab.py` is a read-only driver: it *measures* the current harness, it never modifies it. Running old-vs-new is always possible in the same session, same repo, same model. **Invariants (must hold, checked at the end of every phase).** 1. **No new `.md` files outside this folder** (`projects/self-improvement/{papers,specs,build}`). Everything else is code or DB rows. 2. **No change to the live harness** until the §7 A/B gate passes. The current tool surface stays byte-identical during the entire build. 3. **The L column holds exactly ONE entry** — representation adjudication (§4), fired only on genuine ambiguity, never re-prompted, never touching repository bytes. Any phase that introduces a language model elsewhere in the write path (scope classification, representation selection, mechanical application, the uniqueness/parse/lint gate, the collateral-change check, commit/anchor refresh) is rejected outright; a learned representation chooser that runs by default is rejected outright. 4. **The model never writes repository bytes directly** — application is always deterministic software outside the LLM (locate, verify unique match, transform, reparse). The LLM decides what the change ought to mean (via 09) and, on ambiguity, which representation to apply; the harness owns the bytes. 5. **Apply–verify–revert** — every apply is followed by a parse/lint verification with atomic rollback to the pre-application snapshot on any failure; a malformed or ambiguous edit never persists and never reaches downstream validation. 6. **A collateral-change gate verifies the resulting diff stays inside the declared scope** — any bleed (lines outside the target entity/span, an unrelated symbol, a reformatting whole-file rewrite, a wrong-occurrence match) is a **rejected edit**, never a green apply. 7. **Anchors are refreshed after every mutation** — line numbers and byte offsets from before this edit are never reused by a subsequent edit; each landed edit refreshes the coordinate space the next edit will address. 8. **Editing mechanisms are ablated, not accumulated** — a mechanism that does not move the honest metrics (apply-success rate, collateral-change rate) is **removed**, never kept because it sounds sensible. Learned additions (CODESTRUCT full action space, GRPO editor, structure-aware diff formats, trained applier) must clear the §7 bar on held-out instances at a defensible cost before they may enter the write path. --- *End of spec. This document is the build contract; the next document records the A/B outcome.*