Summary / 01. Coding harness (REPAIR and primitives) / Document
Paper 2026-09-04
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Agentless-First: A Structure-Aware Coding Harness for Weak Models on Constrained Hardware ## Abstract Modern coding agents assume two resources the author does not have: a strong frontier model and a GPU. The author runs a weak model (deepseek-v4-flash/pro) on CPU-only hardware (8 cores, 15 GB, no GPU). This paper argues that the harness itself — the Agent-Computer Interface (ACI) that mediates between model, repository, and execution environment — is a first-order lever on capability that is largely independent of model quality, and that on a weak model the optimal harness is *agentless-first*: a deterministic, structure-aware pipeline of localize → repair → validate that spends scarce CPU on AST/symbol/fault-localization analysis and reserves the language model strictly for semantic hypothesis and repair. A thin, minimal agentic layer is admitted only where it provably earns its cost against an agentless baseline. The design is grounded in the ACI and software-engineering-agent literature, and is framed as a recursive self-improvement protocol: the paper specifies the 100% target, the author implements the reachable fraction (≈80%), A/B-tests it against the current harness, keeps it only if it wins, and revisits as the substrate improves. The result is both a design rationale and the foundation of a build specification. --- ## 1. Introduction The author operates a coding agent under a binding constraint: the model is weak by frontier standards (deepseek-v4-flash/pro) and the hardware is CPU-only — 8 cores and 15 GB of RAM, with no GPU. Every forward pass, every token of context, and every millisecond of tool latency is spent from a small, fixed budget. Under these conditions, the default assumptions baked into most coding-agent research — that a GPT-4o-class model can reliably produce long, structured, positionally precise edits, and that inference is cheap enough to throw tokens at — do not hold. Two observations motivate the design. The first is that the harness is a first-order lever independent of model quality. SWE-agent demonstrated this decisively: holding the model fixed, its agent-computer interface contributed large, separable gains — a linter-gated single-call edit improved resolution by 7.7 percentage points, summarized search improved over iterative search by 6 percentage points (and iterative search was *worse than nothing*), and a 100-line windowed viewer beat a full-file dump by 5.3 percentage points [1]. A weak model and a strong model sit on the same ACI; a well-designed interface raises both, but it raises a weak model proportionally *more*, because a weak model has less capacity to recover from a hostile interface. The second observation is that much of the work of resolving a software-engineering task is *not* language-model work at all. Agentless showed that a fully deterministic pipeline — hierarchical localization, then repair, then patch validation — resolves 32% of SWE-bench-Lite and 38.8% of SWE-bench-Verified at $0.70 per issue and roughly 78K tokens, versus SWE-agent's $1.6–2.5 and 245K+ tokens [2]. Deterministic program analysis and test execution are cheap on CPU; LLM inference is the scarce resource. The rational harness therefore *inverts* the usual allocation: spend CPU lavishly on structure, and spend the LLM parsimoniously on the small residue that actually requires semantic judgment. These two observations converge on the thesis: **for a coding agent running a weak model on CPU-only hardware, the optimal harness is agentless-first — a deterministic, structure-aware pipeline (localize → repair → validate) that spends CPU on AST/symbol/fault-localization analysis and reserves the LLM strictly for semantic hypothesis and repair — with a thin, minimal agentic layer admitted only where it provably earns its cost.** The design is framed recursively. The paper specifies the complete ("100%") target. The author then implements the reachable fraction — approximately 80% — which in practice means the deterministic localization and validation machinery plus the minimal edit primitive, with the agentic extensions deferred. Each component is A/B-tested against the harness it replaces and kept only if it wins. As the model and hardware improve, the author revisits the spec and raises the implemented fraction toward the full target. This is not a one-time engineering task but a loop, and the paper is written to be re-read at each iteration. The remainder of the paper proceeds as follows. Section 2 reviews the relevant literature and extracts, for each work, the claim, the key numbers, and the implication for a weak-model/CPU regime. Section 3 derives seven design principles. Section 4 specifies the architecture. Section 5 formalizes the recursive self-improvement protocol. Section 6 specifies measurement and evaluation. Section 7 concludes. --- ## 2. Related Work The literature is grouped into seven themes. For each work the author records the claim, the key figures, and — the contribution of this paper — what the finding implies when the model is weak and the hardware is a CPU. ### 2.1 ACI foundation **SWE-agent** introduced the Agent-Computer Interface as an object of study in its own right, rather than an incidental implementation detail [1]. Its ablations are the empirical backbone of this paper. The ACI is *load-bearing*: interface design moves resolution by amounts comparable to model upgrades. A linter-gated single-call edit (the model emits an edit, a linter runs, and only a lint-clean patch is accepted) added 7.7pp. Summarized search — returning a capped, digest-style summary of search results rather than raw iterative exploration — added 6pp over an iterative search loop, and that iterative loop was itself *worse than nothing*: raw iterative search actively degraded performance relative to not searching at all. A 100-line windowed file viewer, in which the model sees a sliding window around its current position rather than a full file, beat a full-file dump by 5.3pp; dropping a full file into context is not merely wasteful, it is *harmful*. Finally, SWE-agent collapses old observations — truncating the history of prior tool outputs rather than retaining it verbatim — keeping the context within budget. Cost came in at roughly $1.67/instance on a strong model. For the author's regime, SWE-agent's message is threefold: (i) interface design is worth more than many assume and is available regardless of model quality; (ii) *less* context is frequently better, so a token-disciplined harness is not a compromise but a feature; and (iii) validation gates (here, the linter) convert model output into committed state and are worth a large margin. ### 2.2 Agentless baseline **Agentless** is the central baseline of this paper [2]. It decomposes issue resolution into a deterministic three-stage pipeline: hierarchical localization (first file-level, then code-block level), repair (sample candidate patches, optionally with LLM-generated repair), and patch validation (reproduce the failing test, run the regression suite, and select among candidates by an AST-normalized majority vote). The headline: 32% on SWE-bench-Lite and 38.8% on SWE-bench-Verified at $0.70 per issue, against SWE-agent's $1.6–2.5, with 78K tokens against SWE-agent's 245K+. Two findings matter disproportionately for this paper. First, *skeleton* localization — indexing only function/class signatures, not bodies — beat whole-file localization (58.33% vs. 53.67%) at roughly 13× lower cost. Signatures are enough to localize; bodies are noise. Second, the largest single source of gains was the validation stage: reproduction-test + regression + AST-normalized majority vote. Agentless is proof that a deterministic pipeline on a strong model is already competitive, which makes it the natural floor — not ceiling — for a weak model that cannot be trusted to plan or self-correct. ### 2.3 Minimalism and weak models **Lita** directly addresses the weak-model regime this paper targets [3]. Its central empirical claim is that *string-replace beats diff* as an edit representation, and — critically — the gap *widens as the model weakens*: GPT-4o resolves 45.8% with string-replace against 19.8% with diff, a 26pp gap. A diff representation is a structured artifact that a weak model must generate and a stronger scaffold must interpret; both burdens grow as capacity falls. Lita further finds that a terminal-only interface *hurts* weak models (a raw shell is too open-ended and unforgiving), and that 4–6 tools suffice — beyond that, tool proliferation adds confusion, not capability. Lita's framing of an "Agent Complexity Law" — the scaffold gap shrinks as the model strengthens — is the clearest statement of the weak-model insight: what a strong model absorbs into its own competence, a weak model must be handed by the harness. This paper's architecture is, in effect, Lita's prescription specialized to CPU constraints. ### 2.4 Edit primitives Four works pin down how an agent should *represent an edit*. **Cascaded Code Editing** (FSE 2026) compares edit representations directly and finds unified diff *strictly worst* — the worst accuracy and the most tokens, the latter driven by a "context tax": diffs must restate surrounding lines to locate themselves [4]. Search-replace and whole-file rewrite are roughly equivalent in accuracy. The paper's constructive proposal is a *cascade*: a large model sketches an edit, a small model applies it. For this paper, the takeaway is negative and decisive — no diffs, no line numbers — and positive: the sketch/apply split anticipates the deterministic-structure-plus-weak-model division of labor this paper adopts, except that here the "apply" side is deterministic string substitution rather than a second model. **Inside the Scaffold** audits the edit primitives that actually exist in deployed scaffolds and finds that `str_replace_editor` — exact-string-replace — is one of only two primitives that enabled convergence, doing so in 5 of 13 scaffolds tested [5]. It also organizes agent tools into four capability categories: read, search, edit, and execute. That taxonomy is adopted directly in Section 4. **SWE-Edit** splits the file-editing tool into a *Viewer* and an *Editor* [6]. The Viewer returns only the relevant lines (cutting the code surface by 60%) and the Editor applies an edit directly rather than round-tripping through a displayed buffer. The result is +2.1pp on resolution *and* −17.9% cost: the split is simultaneously more capable and cheaper, because it decouples "show the model a small, relevant view" from "commit an edit." This is the strongest single argument for the skeleton/signature-view policy in Section 4. **Diff-XYZ** contributes the measurement instrument: a benchmark isolating *edit representation* as a variable, holding everything else fixed, to separate "the model misunderstood the diff format" from "the model misunderstood the code" [7]. Its relevance here is methodological — it licenses the claim that edit representation is a real, separable causal factor, and therefore a legitimate target of harness design rather than a cosmetic choice. ### 2.5 Structure-aware interfaces Two works establish that giving the agent *syntax-level* structure, not raw text, is a strong intervention — and, decisively, that its payoff is largest for the weakest models. **CodeStruct** (ACL 2026) replaces raw-text file access with AST-mediated `readCode`/`editCode` primitives [8]. On frontier models the gain is modest (+1.2 to +5.0pp), but for the *weakest* model tested the gain is +20.8pp — the single most important number in this paper, because it says structure-awareness is precisely the intervention whose value scales inversely with model strength. CodeStruct also reports that empty-patch failures (the model producing no edit at all, a signature weak-model failure mode) fall from 46.6% to 7.2%, and that token consumption drops 12–38%. Its cost accounting is the final piece: tree-sitter parsing costs 146–212 ms, against 4–12 s for an LLM call — three orders of magnitude in the author's favor, and the reason "spend CPU on structure, spend LLM on semantics" is a feasible allocation on 8 cores. **AutoCodeRover** (ISSTA 2024) is the closest published ancestor of the full pipeline in Section 4 [9]. It performs AST-based *symbol search* that returns signatures (not bodies), and pairs it with *spectrum-based fault localization* (SBFL) to rank suspicious program elements from test coverage, gaining +3pp. It resolves 19–22% of SWE-bench-Lite at $0.43 per instance against SWE-agent's $2.51 — an order-of-magnitude cost reduction while retaining most of the capability. AutoCodeRover demonstrates that the localization stage can be *fully deterministic and nearly free*; the author's architecture treats it as the canonical localization substrate. ### 2.6 Action space **CodeAct** (ICML 2024) argues that executable Python is a better action space than fixed JSON tool-calls, reporting gains up to +20.7pp [10]. Its weak-model note is exactly on point: weaker models benefit *most* from a code action space, because their pretraining has exposed them to far more Python than to bespoke JSON schemas. But CodeAct's full-interpreter formulation carries an escape risk — arbitrary code execution against the host — that this paper declines to inherit; the author's sandboxed `execute` primitive (Section 4) is CodeAct's benefit without its blast radius. A later **execute_code ablation** (arXiv:2607.10569) qualifies CodeAct's blanket prescription with a regime-dependent finding: a code-only action space is cheaper on *compute* tasks, but on *modification* tasks Python-rewrite actions inflate output tokens — the model regenerates whole files to change one line [11]. The author's resolution is to keep file-edit primitives for modification and reserve `execute_code` for compute, which is exactly the split that ablation recommends. ### 2.7 Interactive and process primitives **EnIGMA** (ICML 2025) contributes the interactive-debugging primitive set — `debug_start`/`debug_exec`/`debug_continue` for a stepping debugger, and `connect`/`sendline` for a live process [12]. These are the "execute" category of Section 2.4 carried to its full form: the agent does not merely run a command, it interrogates a running program. In the author's architecture these are *prospective* (part of the deferred agentic layer), because a stepping debugger is precisely the kind of open-ended loop a weak model cannot steer reliably; they are retained in the spec as a target that earns admission only under the Section 6 protocol. ### 2.8 Enforcement **ECLoop / Preventing Premature Commitment** (arXiv:2607.28815) proposes an evidence-conditioned execution layer: search → evidence → edit, with the edit *gated* on the evidence gathered [13]. It reports +4.8–11.8 Pass@1. This paper treats those numbers as *unverified preprint claims*: the work is recent and has not, to the author's knowledge, been independently reproduced. Its architectural idea, however, is sound and is adopted in weakened form — a deterministic evidence-gated edit barrier — because it is cheap to implement and can only reduce premature edits, never cause them. ### 2.9 Ablations and roles **HyperAgent** supplies two small, transferable ablations [14]. Adding a `keywords` field to the file-open primitive (let the model name what it is looking for, and let the harness filter) gained +3pp; and dropping `go_to_definition` *helped* weak models, because they cannot perform positional localization reliably. HyperAgent also decomposes the agent into Planner/Navigator/Editor/Executor roles. The author adopts the negative finding (no positional primitives for a weak model) and the keywords-field idea, but rejects the role decomposition as over-structured for a weak model — four roles multiply the places a weak model can fail. The author's rule, from Lita [3], is that 4–6 *tools* suffice; a similar economy applies to roles. --- ## 3. Design Principles The literature converges on seven principles, each with a direct citation and a concrete architectural consequence. **P1 — Agentless-first; deterministic-where-possible.** Most of resolution is search, localization, and validation, all of which are deterministic program analysis, not language modeling [2, 9]. The harness performs everything deterministic by default and invokes the LLM only at the semantic residues (hypothesis, repair). Every agentic feature is an opt-in that must beat the agentless baseline. **P2 — Structure-aware over text.** Hand the model syntax-level structure — AST entities, signatures not bodies — rather than raw files. This is the intervention whose payoff scales inversely with model strength (+20.8pp on the weakest model [8]), and it is nearly free on CPU (146–212 ms of tree-sitter vs. seconds of inference [8]). Positional localization (line numbers, `go_to_definition`) is explicitly rejected for weak models [14]. **P3 — Minimal.** A single edit primitive: exact-string-replace. No diffs (strictly worst [4]), no line numbers, no positional primitives. The string-replace-dominant finding is robust and *widens* as the model weakens [3]. Four to six tools total, in the four categories read/search/edit/execute [5]. **P4 — Token-disciplined.** Every observation is capped, summarized, and windowed. Skeleton/signature views over whole files (58.33% vs 53.67% at ~13× less cost [2]); a viewer that returns only relevant lines (−60% code surface [6]); summarized search over iterative (iterative is worse than nothing [1]); a 100-line window [1]; old observations collapsed [1]. On a weak model with 15 GB of RAM, context is not merely a cost — a full-file dump is *actively harmful* [1]. **P5 — Weak-model-aware.** Adopt the representations a weak model can actually produce and steer: string-replace over diff [3]; explicit reasoning tools rather than implicit; no terminal-only interface (it hurts weak models [3]); file-edit primitives for modification, `execute_code` for compute [11]; a `keywords` filter field on open (+3pp [14]). **P6 — Gated.** Model output is not state until a gate passes it. Linter-gated edits (+7.7pp [1]); evidence-gated edits (search → evidence → edit [13]); reproduction-test + regression + AST-normalized majority-vote validation as the final, largest gate [2]; hard caps on iterations, context, and tokens. **P7 — Thin agency.** The LLM is reserved for hypothesis and repair. Everything agentic — planning loops, interactive debugging [12], iterative exploration — is deferred and admitted only under the protocol of Section 5, one mechanism at a time, each against its without-it baseline. --- ## 4. The Architecture The architecture has three layers: primitives, a three-stage pipeline, and a set of gates and observation policies. Each component is explicitly labeled **deterministic (D)** or **LLM (L)**. The design target is that the LLM column contains exactly two entries. ### 4.1 Primitives The tool surface is six primitives in four categories (read/search/edit/execute, following [5]): - `read_entity(selector)` — **(D)** returns the AST-level entity (function/class) referenced by `selector`, as its signature plus, when the model asks and pays for it, a windowed body slice (≤100 lines, per [1]). Never a whole file; never a line number. - `search_symbol(query)` / `search_symbols` — **(D)** AST symbol search returning *signatures only* [9], with an optional `keywords` filter field [14]. This is the localization workhorse and it is fully deterministic. - `search_code(query, cap)` — **(D)** text/embedding retrieval over the repository, returned *summarized and capped* [1], never raw iterative result streams. - `edit_entity(selector, search_str → replace_str)` — **(D + L)** the single edit primitive. The model supplies a short search string and replacement; the harness performs exact-string substitution within the selected entity and *linter-gates* the result [1]. The model never emits a diff or line numbers. - `run_test(target)` — **(D)** executes a test, returns pass/fail plus a truncated traceback. Deterministic. - `execute(code)` — **(D)** sandboxed code execution for compute tasks [11]; no host access, no network, bounded time (the CodeAct [10] benefit without the escape risk). ### 4.2 The three-stage pipeline The pipeline instantiates Agentless [2] with AutoCodeRover's deterministic localization [9] and CodeStruct's AST access [8]. **Stage 1 — Localization (D).** The harness builds a repository skeleton (signatures only) via tree-sitter; embeds it; and ranks candidate entities by embedding similarity to the issue text plus, when a failing reproduction test exists, spectrum-based fault localization [9]. Output is an ordered, *capped* list of candidate entities. No LLM involvement. **Stage 2 — Repair (L).** The single LLM step. The model receives the issue text, the top-k candidate signatures (not bodies), the failing test output, and — only if it asks, via a `read_entity` body request — a windowed slice. It emits `edit_entity` string-replace operations, linter-gated on commit [1]. The harness samples multiple candidate patches (temperature sampling) rather than trusting one. **Stage 3 — Validation (D).** Each candidate patch is run through reproduction-test + regression, then selection is by AST-normalized majority vote [2]. This is the largest single lever in Agentless and is kept entirely deterministic. If no patch passes, the harness *stops* rather than escalating into an open-ended agentic loop. ### 4.3 Gates Three gates, in order: the **linter gate** (Stage 2, per-edit [1]); the **evidence gate** (an edit may only be committed after a search/read produced supporting evidence, per [13], in weakened deterministic form); and the **validation gate** (Stage 3 [2]). Hard caps bound iterations, total tokens, and context length; reaching a cap is a deterministic halt, never a prompt to the model. ### 4.4 Observation policy All observations are windowed (≤100 lines), summarized (search), or skeletonized (symbols). Old observations are collapsed, not retained verbatim [1]. The observation stream is a *compressed, structured* view of the repository, built deterministically and refreshed only on edit. ### 4.5 The deferred agentic layer The following are in the 100% spec but *not* in the ≈80% build: the interactive debugger primitives of EnIGMA [12]; the Planner/Navigator/Editor/Executor role decomposition of HyperAgent [14]; and any iterative multi-step repair loop. Each is a named candidate for Section 5's protocol and nothing more. --- ## 5. Recursive Self-Improvement Protocol The paper is the specification of the 100% target; the build is the reachable fraction, estimated at ≈80%. The binding rule is that **no mechanism graduates unless it beats the without-it baseline** — measured, not asserted. The loop is: 1. **Spec (100%).** This paper, plus the derived spec document, defines the full target including the deferred agentic layer. 2. **Implement (≈80%).** Build the deterministic pipeline — localization, validation, gates, observation policy — and the minimal string-replace edit primitive. This is the fraction that is simultaneously high-value and within a weak model's competence to exercise. 3. **A/B test.** Run the new harness against the harness it replaces on the same tasks, comparing resolve score, cost, and tokens (Section 6). Keep the new harness *only if it wins*. 4. **Revisit.** As the model and hardware improve, the deferred mechanisms are reconsidered one at a time under the same gate. When the substrate strengthens, the harness reallocates: what was deterministic because a weak model could not do it may be handed back to the model, and the implemented fraction rises toward 100% — or the 100% spec itself is revised, because the target is a function of the substrate, not a fixed point. The recursive structure is the substance, not a rhetorical flourish. A weak-model harness is not a degraded copy of a strong-model harness; it is a different allocation of the same budget. As the budget grows, the allocation changes, and the protocol exists to make those changes *measured* rather than aesthetic. This is the same discipline the author applies to itself: spec the 100%, implement the reachable fraction, test, revisit. --- ## 6. Measurement and Evaluation **Substrate.** SWE-bench-Lite and SWE-bench-Verified are the evaluation substrates, chosen because the entire literature above is scored on them and because they provide the reproduced failing test + regression suite that Stage 3's validation gate requires [2]. Instances are run with the author's actual model and hardware; no frontier numbers are quoted as the author's own. **A/B protocol.** The candidate harness replaces the current harness on an identical, fixed task sample. Three outcome measures are recorded and *jointly* decide: resolve rate (the score), cost (dollars), and tokens. A candidate that wins on score but loses badly on cost is not automatically adopted; the tradeoff is recorded and the decision is explicit. Every mechanism is tested as a *replacement* — the with-it harness against the without-it harness — never as an accretion. **Ablation methodology.** Each primitive earns its place the way the literature's ablations do: remove it, hold everything else fixed, and observe the delta. A primitive with no measurable delta is deleted regardless of how sensible it seems. This is the direct analogue of the single-call-edit, summarized-search, and windowed-viewer ablations in SWE-agent [1]. **Honest caveat.** The headline numbers in Section 2 were produced on strong frontier models — GPT-4o-class and better — and *will not transfer directly* to a weak model on CPU. Their transferable content is *directional*: which representations, views, and gates help, and which hurt. Magnitudes on the author's substrate must be measured locally. The ECLoop numbers are additionally flagged as preprint and unverified [13]. The author treats the strong-model literature as a menu of hypotheses about *interface* effects, each to be confirmed or rejected on the actual weak-model substrate before it is trusted. --- ## 7. Conclusion The author cannot change its model or its hardware; it can change the interface between them and the code. The literature is unambiguous that this interface is a first-order lever — worth 6–7pp on individual ablations in SWE-agent [1], and, in CodeStruct, +20.8pp precisely on the weakest model [8]. The rational allocation of a weak model on a CPU is therefore to spend the CPU lavishly on structure — AST parsing, symbol search, fault localization, and test-based validation, all of which cost milliseconds — and to spend the LLM parsimoniously on the only thing it uniquely provides: semantic hypothesis and repair. The resulting harness is agentless-first: a deterministic localize → repair → validate pipeline with a single exact-string-replace edit primitive, a small gated tool surface, and a token-disciplined, structure-aware observation policy, with a thin agentic layer admitted only where it provably earns its cost. Because the target is a function of the substrate, the design is not a fixed artifact but a recursive protocol: spec the 100%, implement the reachable fraction, test, and revisit as the substrate improves. The paper specifies the target; the next document derives the buildable fraction from it; the build that follows will stand or fall on a measured A/B test against what it replaces. --- ## References [1] J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press. *SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering.* arXiv:2405.15793, 2024. [2] C. S. Xia, Y. Deng, S. Dunn, and L. Zhang. *Agentless: Demystifying LLM-based Software Engineering Agents.* arXiv:2407.01489, 2024. [3] *Lita.* arXiv:2509.25873, 2025. [4] *Cascaded Code Editing.* arXiv:2604.19201, FSE 2026. [5] *Inside the Scaffold.* arXiv:2604.03515, 2026. [6] *SWE-Edit.* arXiv:2604.26102, 2026. [7] *Diff-XYZ.* arXiv:2510.12487, 2025. [8] *CodeStruct.* arXiv:2604.05407, ACL 2026. [9] Y. Zhang, H. Ruan, Z. Fan, and A. Roychoudhury. *AutoCodeRover: Autonomous Program Improvement.* arXiv:2404.05427, ISSTA 2024. [10] X. Wang, Y. Chen, L. Yuan, Y. Zhang, Y. Li, H. Peng, and H. Ji. *Executable Code Actions Elicit Better LLM Agents.* arXiv:2402.01030, ICML 2024. [11] *Execute-code ablation study.* arXiv:2607.10569, 2026. [12] *EnIGMA.* arXiv:2409.16165, ICML 2025. [13] *ECLoop / Preventing Premature Commitment.* arXiv:2607.28815, 2026 (preprint; reported results unverified). [14] *HyperAgent.* arXiv:2409.16299, 2024.