Summary / 19. Regression protection / Document
Paper 2026-09-06
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# The Unchanged Must Be Proved, Not Assumed: Differential, Impact-Driven, Preservation-Oriented Regression Protection for a Weak-Model Coding Harness *Component 19 of 27 — Coding-Harness Program. This document specifies the regression-protection stage, and it is precise about what it is not: it is not the certification battery of component 17, which runs the checks that already exist, and it is not the acceptance decision of component 18, which decides on the candidate using the evidence at hand. This component asks the question the other two leave open: what catches the damage that the existing checks do not cover. A patch that flips the failing tests has proven only itself — the rest of the repository is merely asserted unchanged, and that assertion must be earned by standing machinery rather than trusted. The spine position is the industrialisation of the validation row's "unbroken?" clause: component 17 ran the repository's own suite and typed each result; component 18 folded "unbroken?" into the acceptance verdict; this row turns that clause into a standing discipline that exists before, during, and after every edit. SWE-bench's PASS_TO_PASS is this row converted into a score; this row is how the score gets honestly earned. The core move is to stop treating "the tests still pass" as evidence of preservation and to start treating the pre-edit program itself as the specification of everything that must not change — pinned into characterisation tests and mined invariants before the edit, held to account through differential execution against the known-good baseline, selected to the smallest safe regression set with every skip logged as unrun, gated at the public surface where no test can see, and surfaced as an intentional-versus-accidental manifest for review rather than a boolean for the repair loop to auto-accept.* --- ## Abstract Regression protection is where a coding harness's most expensive lie hides behind its cheapest-looking evidence: a green suite after a fix. The bug test flipped, so the patch "works" — but the nine files the edit touched in passing, the callers outside the repository, the string-keyed contracts and duck-typed consumers, and the behaviors that no test ever exercised are all merely assumed unchanged, and that assumption is the regression. This paper argues that preservation is not the residue of running whatever tests happen to exist, but a first-class obligation that must be derived, selected, executed, and evidenced — and that the single most powerful oracle for it is free: the pre-patch program, whose behavior on everything the fix did not intend to change is, by definition, correct. The review establishes the case across thirty-two sources: REFINE named the failure mode of patches that fit incomplete suites rather than fix the bug [1], and SWE-bench made PASS_TO_PASS the operational metric this row exists to secure [2]; ChangeGuard [3], PATCH-SIM [4], and DiffTGen [5] built differential execution — comparing patched behavior against the known-good baseline rather than trusting absolute pass/fail; Testora [6] and PatchGuru [7] added the one genuinely learned step, reading a change's stated intent to judge which observed divergences are the fix and which are collateral, with P³ carrying the same division of labor to product-program reasoning [8]; Ekstazi [9], HyRTS [10], Rothermel and Harrold [11], and Yoo and Harman [12] supplied the safe-selection foundation, and Meta's Predictive Test Selection the risk-ranked top-slicing that selection uses when the safe set exceeds budget [13]; Daikon's dynamic invariant detection [14] and Feathers's characterisation testing [15] supplied the pre-edit executable spec of unchanged behavior, with QuickCheck's properties [16] as the standing form of that spec; Ye, Martinez, and Monperrus [17], Probe to Generate [18], CodaMosa [19], TestGen-LLM [20], SynFix [21], and Lou et al. [22] bounded how far generated and selected tests can be trusted when the oracle is weak, with Al-Bataineh's interaction-aware assessment the caution that multiple interacting faults defeat single-fault patch assessment [23]; Brito et al. [24], Jayasuriya et al. [25], SymDiff [26], and Godlin and Strichman [27] established the surface gates — API compatibility, semver, and equivalence proof — for what no test can see; and Memon et al. [28], Herzig et al. [29], SWE-ABS [30], TDAD [31], and TestPrune [32] measured the economics of what a harness skips, strengthens, and selects. From this review the paper derives eight design principles, then specifies a deterministic architecture in which the single learned entry is intent interpretation — judging which divergences are the intended fix versus collateral regression — and every other service, from selection to differential execution to invariant mining to the PreservationLedger, is deterministic software on CPU, so that the invariant "the L column contains at most one entry" holds mechanically. It closes with a recursive self-improvement protocol and a measurement plan whose instruments exist to keep the harness from mistaking an assumed green for a proved one. ## 1. Introduction The coding-harness program has, through components 01 through 18, assembled machinery that reads a repository, localizes a fault, maps its dependencies, generates a candidate patch, edits it into place, runs the certification battery, and decides whether to accept the result. Component 19 is the stage that asks the question every one of those upstream stages quietly ignored: the patch fixed the bug — what did it break? The naive harness collapses that question into a single invocation: run the full suite, and if it is green, the patch is safe. That collapse is the precise failure this component exists to prevent, because a full suite is not protection, it is sampling — and the sample is drawn by people who wrote tests for a bug that had not happened yet. A fix validated only against FAIL_TO_PASS tests says nothing about the nine files the edit touched in passing; regression risk is proportional to the diff's blast radius, not to the bug's. The thesis that governs this component is stated plainly, and it is the contract every design choice below is measured against. **Regression protection should be differential, impact-driven, and preservation-oriented: before editing, identify behavior that is expected to remain unchanged; derive preservation obligations from dependency and impact analysis; select the smallest safe set of regression tests using static and dynamic dependencies; compare patched behavior against the known-good baseline rather than relying solely on absolute pass/fail; treat unexpected execution or output divergence on previously correct behavior as regression evidence; augment incomplete suites with generated or property-based probes where reliable oracles exist; and escalate toward broader regression suites as the change's impact or uncertainty grows.** And it carries a strong rule for Piper that compresses the whole thesis into one sentence, stated verbatim: **A bug fix changes what was wrong. Regression protection proves that everything we did not intend to change stayed the same.** The first clause is the discipline of pre-editing — the behavior that must not change is identified and pinned *before* the edit, because after the edit the pre-change program is gone and the specification with it. The second clause is the discipline of derivation — preservation obligations come from impact analysis, not from whatever tests happen to exist. The third clause is the discipline of selection — the smallest *safe* set, safe first and small second, with every skipped test logged as unrun rather than silently green. The fourth clause is the discipline of the differential oracle — the pre-patch program is the free and correct oracle for every behavior the fix did not intend to touch, so preservation is a comparison against a known-good baseline, not an absolute pass. The fifth clause is the discipline of evidence — a divergence on previously correct behavior is regression evidence, to be explained or escalated. The sixth clause is the discipline of augmentation — where the suite is thin, generated and property-based probes fill the gap, but only where a reliable oracle exists. The seventh clause is the discipline of escalation — as the change's impact or uncertainty grows, the harness widens the net rather than trusting the narrow one. The weak-model problem makes regression protection the make-or-break stage for a reason that inverts the usual one. A strong model can be trusted, up to a point, to *notice* that it broke something and to diagnose the break; a weak model cannot be trusted even to notice, and it will treat its own "tests pass" as vindication. The harness therefore cannot leave preservation to the model's diligence any more than component 17 could leave certification to the model's honesty. Preservation must be machinery — deterministic, standing, and wired into every edit rather than invoked once at the end. The economics force the same conclusion from the other side: a repository-scale battery per repair round is loop-suicide, so *selection* is not an optimisation, it is the difference between having regression protection and pretending to. Ekstazi's near-zero marginal cost made per-edit selection tractable [9]; HyRTS showed dependency-aware selection can cut the regression set to 18.35% of tests versus 27.18% for file-level selection, at 31.60 seconds versus 40.06 seconds of offline time [10]; and Meta's Predictive Test Selection showed that when even the safe set is too large, a risk-ranked model can spend the budget where failures have historically been found, with an audit trail of what it chose not to run [13]. And the danger of getting selection wrong is not hypothetical: Google-scale work found that deleted tests had been catching real bugs, the minimisation caution industrialised [28]. The design implication is decisive, and it is the thesis: a weak-model harness must pin the unchanged behavior before the edit, derive obligations from impact, select the smallest safe regression set with skips logged as unrun, compare patched behavior against the known-good baseline, gate the public surface where tests cannot see, and route every observed divergence to review as intentional or accidental — because the question is never whether the tests passed, but whether everything the fix did not intend to change provably stayed the same. The paper proceeds as follows. Section 2 reviews the regression-protection literature in eight parts — the problem named, differential execution, intent as the boundary, change-scoped selection, invariant mining and the pre-change spec, suite augmentation and the oracle problem, surface gates and API compatibility, and the economics of what a harness skips. Section 3 derives eight design principles. Section 4 specifies the architecture, labeling every node deterministic (D) or learned (L) and stating the invariant that the L column holds at most one entry, with the PreservationLedger as the central data structure. Section 5 states the recursive self-improvement protocol. Section 6 gives the measurement plan. Section 7 concludes. ## 2. Related Work The literature divides into eight parts that together answer one question: what must regression protection *be* so that "everything we did not intend to change stayed the same" is a proven fact rather than a hopeful assumption? Each part supplies one design lever the preservation machinery must pull. ### 2.1 The problem named: patches that fit the suite, not the bug **REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement** (Anvith Pabba, Simin Chen, Alex Mathai, Anindya Chakraborty, and Baishakhi Ray, arXiv 2025, arXiv:2510.03588) is the anchor that names the problem this component industrialises, and it is cited here precisely and only as what it is: an October 2025 preprint with no venue, whose figures are preprint results to be locally reproduced before any magnitude is assumed to transfer [1]. REFINE's motivation is the gap between *a patch passes the visible evidence* and *a patch is actually correct*: repository-level repair agents routinely produce draft patches that are only partially correct, or that overfit incomplete test suites, because the tests that motivated the patch do not describe all previously valid behavior. The authors attempt to convert these draft patches into correct ones through better issue and code context, diverse delta-patch generation, review, and aggregation, reporting a 14.67-point improvement over AutoCodeRover on SWE-bench Lite (reaching 51.67%) and a 12.2-point improvement on SWE-bench Verified [1]. The regression-protection lesson is the one this row is built on: passing the tests that motivated the patch is insufficient evidence of preservation, because those tests may not describe the behavior that was already correct. The design transfer is the preservation obligation — the harness must maintain an explicit, impact-derived list of what must not change, and then prove every entry, rather than run whatever tests happen to exist. **SWE-bench: Can Language Models Resolve Real-World GitHub Issues?** (Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan, ICLR 2024, arXiv:2310.06770) is the paper that gave the program the two-sided vocabulary this row's entire purpose is to secure, and it belongs here because its 2,294 real GitHub issue/PR pairs across 12 Python repositories routinely require coordinated changes across multiple functions, classes, and files — the regression surface is broad by construction [2]. The benchmark scores a modified repository, not a plausible function, and its verdict is defined by two independent test sets: FAIL_TO_PASS tests must flip, and PASS_TO_PASS tests must hold. That PASS_TO_PASS set is exactly the claim this component exists to prove — "everything we did not intend to change stayed the same" — and its mechanical reapplication of protected test files is surface protection, benchmark edition: the tamper guard that stops a patch from editing the referee is the same discipline, at the surface, that this row generalises to behavior. A local edit can have nonlocal consequences — a changed function reaches direct callers, indirect callers, subclasses, serializers, config paths, and tests elsewhere — so regression obligations must be generated against all of them, even when the issue mentioned only one. ### 2.2 Differential execution: the pre-patch program as the oracle **ChangeGuard: Validating Code Changes via Pairwise Learning-Guided Execution** (Lars Gröninger, Beatriz Souza, and Michael Pradel, Proceedings of the ACM on Software Engineering, FSE 2025, DOI 10.1145/3715760, arXiv:2410.16092) is the paper that shows why running the existing suite is not enough, and its comparison number is the row's load-bearing statistic [3]. ChangeGuard executes the pre- and post-change versions of a function side by side under learning-guided execution — an LLM synthesizes the inputs and prefixes needed to run code deep inside a project — and flags any divergence in return values or raised exceptions. It detects semantics-changing edits with a precision of 77.1% and a recall of 69.5%, against the project's own existing regression tests, which catch only 7.6% of semantics-changing code changes [3]. Read alongside the selection line: choosing the right existing tests helps, but in most repositories "the tests still pass" is fully compatible with the behavior having changed. The design transfer is the differential oracle — the pre-patch program, executed on inputs that exercise the changed region, is the reference against which divergence is measured, and divergence is evidence even when the suite is green. **Identifying patch correctness in test-based program repair** (Yingfei Xiong, Xinyuan Liu, Muhan Zeng, Lu Zhang, and Gang Huang, Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, DOI 10.1145/3180155.3180182) — the PATCH-SIM paper — is the strongest statement of the behavior-preservation principle, and it starts from two observations that give the harness an elegant asymmetry: passing tests should usually behave *similarly* before and after a correct patch, while failing tests should often behave *differently* after the repair, because the failure-causing execution must change [4]. Rather than looking only at pass/fail, PATCH-SIM compares execution behavior between the buggy and patched programs, and on a dataset of 139 patches generated from five repair systems (jGenProg, Nopol, jKali, ACS, and HDRepair) it filtered 56.3% of the incorrect patches without blocking any correct patch [4]. The principle the harness inherits: for known-good behavior, prefer stability; for known-bad behavior, require change. A test that passes before and after but whose execution path changed from A→B→C→D to A→X→Y→D is still green, but something substantial changed — that is regression *suspicion*, and it is exactly the signal the differential tier of Section 4 is built to surface. PATCH-SIM must be kept distinct from two differently-named papers that share its territory and its danger of conflation: **DiffTGen** (Qi Xin and Steven P. Reiss, *Identifying test-suite-overfitted patches through test case generation*, Proceedings of the 2017 International Symposium on Software Testing and Analysis, ISSTA 2017, DOI 10.1145/3092703.3092718), the pre-LLM differential test generation for overfitting patches that ChangeGuard modernises [5]; and a third, later work under the near-identical name *DiffTestGen* (arXiv 2607.16024), which is a different paper entirely. The three are never to be merged in the ledger. ### 2.3 Intent as the boundary between the fix and the regression Differential execution finds divergences; it does not say which divergences are the point. That judgment — the one genuinely learned step in this component — is the contribution of the two papers this row promotes to the single L entry. **Testora: Using Natural Language Intent to Detect Behavioral Regressions** (Michael Pradel, IEEE/ACM International Conference on Software Engineering, ICSE 2026, arXiv:2503.18597) is the PR-level version, and the one closest to a harness gate [6]. Testora takes a pull request, reads its stated intent (title, description, linked issue), generates tests that exercise the changed code, and then uses an LLM to judge whether any observed behavioral difference between old and new is *intended* by the PR or a regression. That last step is the whole contribution, and it is the precise boundary of the L column: differential execution is deterministic — a divergence is a fact — but only intent tells which divergences are the bug fix and which are collateral. For an agent pipeline where the "PR description" is the agent's own summary of its change, Testora is a direct check of whether the agent did what it said and nothing more. The abstract records 19 regression bugs found, 13 reported to developers, 11 confirmed, and 9 fixed, at roughly 12.3 minutes and $0.003 per PR — evidence that intent-guided regression detection is cheap enough to sit in a loop. **PatchGuru: Patch Oracle Inference from Natural Language Artifacts** (Thanh Le-Cong, Bach Le, Toby Murray, Michael Pradel, and Cristian Cadar, arXiv 2026, arXiv:2602.05270) extends Testora in the direction this row needs, and it is cited here with its full author list — it is *not* a Pradel-solo paper [7]. Where Testora checks whether a PR introduces unintended behavioral changes, PatchGuru additionally validates that the intended changes are correct: it infers a patch oracle — the expected old-versus-new behavior — from the natural-language artifacts around the change and tests both halves, reporting 24 confirmed true positives from 39 warnings (precision 0.62) across 400 pull requests from four open-source Python projects, versus Testora's 24-versus-7 bug comparison and precision 0.32-to-0.62, at 8.9 minutes and $0.07 per PR [7]. The idea to steal is the deepest one in this row: regression protection and fix verification are the *same* oracle read in two directions — the change's stated intent defines both what must change and what must not, and the harness reads that oracle both ways. **P³: Reasoning about Patches via Product Programs** (Arindam Sharma, Daniel Schemmel, and Cristian Cadar, Proceedings of the ACM on Programming Languages, PACMPL/OOPSLA 2025, DOI 10.1145/3763145) is the formal-methods endpoint of the same trajectory, and the outlier of the row [8]. A product program composes the pre-patch and post-patch versions into a single program whose paths run both in lockstep, so symbolic execution can reason about where their behaviors diverge; in principle this proves behavior is preserved on all inputs outside the intended change, which no amount of test execution can. It is expensive and language-limited today, but it is the ceiling toward which the differential tier points — regression protection as an equivalence argument over the untouched input space, with the agent's stated intent defining the boundary. ### 2.4 Change-scoped selection: safe first, small second **Ekstazi: Lightweight Test Selection** (Milos Gligoric, Lamyaa Eloussi, and Darko Marinov, IEEE/ACM 37th International Conference on Software Engineering, ICSE 2015, Software Engineering in Practice track, DOI 10.1109/ICSE.2015.230) is the selection substrate this row promotes after two cameos in the impact and test-execution rows [9]. Its mechanism is the floor every other selection scheme builds on: each test's transitive file dependencies are checksummed at last run, and a test is selected if and only if any checksum changed since. Everything else is skipped — deliberately, and therefore trackably. The design insight this row inherits is that selection must be *cheaper than not selecting*, or it will not be used — by developers, and certainly not by a repair loop executing dozens of batteries per task. Ekstazi made per-edit selection tractable, and it is wired into every edit, not just the final patch, so intermediate-state regressions surface while they are still attributable. **Hybrid Regression Test Selection** (Lingming Zhang, IEEE/ACM 40th International Conference on Software Engineering, ICSE 2018, DOI 10.1145/3180155.3180198) is the deliberate reach outside the LLM-agent literature, and it is cited by its ICSE 2018 DOI with no invented identifier [10]. HyRTS combines regression-test selection at multiple granularities, choosing method-level dependency analysis where its precision pays for itself and file-level analysis where it is too expensive, and its evaluation is the numbers this component's cost-ordering is built on: 2,707 revisions across 32 Java projects totaling more than 124 million lines of code, selecting 18.35% of tests versus 27.18% for file-level selection, at 31.60 seconds versus 40.06 seconds of offline time [10]. The transfer is the ladder's middle rungs: the harness already holds, from components 02 through 06, the repository graph, dependency graph, changed symbols, and impact analysis, so "small edit, run the entire suite" becomes "changed symbol, affected tests, run those first, widen only if they pass." **A Safe, Efficient Regression Test Selection Technique** (Gregg Rothermel and Mary Jean Harrold, ACM Transactions on Software Engineering and Methodology, 1997, DOI 10.1145/248233.248262) is the safe-RTS foundation the modern selection papers are reviving, cited by its TOSEM DOI with no arXiv identifier [11]. Safety — selecting a superset of the tests whose behavior the change could affect — is the property this component demands of its own selection tier: the "smallest safe regression set" of the thesis is *safe first, small second*, and a selection tier that leaks regressions to the full suite is not a selection tier, it is a gamble with a green costume. **Regression testing minimization, selection and prioritization: a survey** (S. Yoo and M. Harman, Software Testing, Verification and Reliability, 2012, DOI 10.1002/stvr.430) is the canonical taxonomy of the three levers — minimisation (remove redundant tests), selection (run the affected subset), prioritisation (order tests to fail fast) — and its durable caution is the row's red flag [12]. Minimisation is dangerous: pruning "redundant" tests measurably reduces fault-detection capability, because redundancy is insurance against unmodelled failure modes. The implication is directive: the harness leans on selection and prioritisation, and treats minimisation — deleting tests to save compute — as a red-flagged operation that spends safety and must never be done casually. **Predictive Test Selection** (Mateusz Machalica, Alex Samylkin, Meredith Porth, and Satish Chandra, IEEE/ACM 41st International Conference on Software Engineering, Software Engineering in Practice track, ICSE-SEIP 2019, DOI 10.1109/icse-seip.2019.00018) is Meta's deployment of learned selection, and it is cited here with its corrected attribution — the authors are Machalica, Samylkin, Porth, and Chandra, and the venue is ICSE-SEIP 2019, not any 2021 variant [13]. A model trained on repository history predicts each test's failure probability given a diff, from static and dynamic features, and the harness runs the top slice — an explicit, tunable trade between compute saved and missed-failure risk, deployed at tens of thousands of test jobs daily with large compute reductions and bounded missed regressions [13]. Two lessons extend Ekstazi: selection can be *risk-ranked*, not just reachability-ranked (some reachable tests have never caught anything), and the risk model is *living infrastructure* — it retrains as the repository evolves, and its misses feed back as labels. The design transfer is the budget policy of Section 4: when the loop cannot afford the full safe set, spend the budget by predicted risk, and keep an audit trail of what the model chose not to run — because a skipped test is *unrun*, never silently green. ### 2.5 Invariant mining and the pre-change executable spec **Dynamically discovering likely program invariants to support program evolution** (Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin, Proceedings of the 21st International Conference on Software Engineering, ICSE 1999, DOI 10.1145/302405.302467; and IEEE Transactions on Software Engineering, Vol. 27, No. 2, February 2001, DOI 10.1109/32.908957) is the Daikon paper, and it is cited here with its corrected journal venue — the journal version is IEEE Transactions on Software Engineering 2001, not TOSEM 2001 [14]. Daikon instruments execution, observes variable values at program points, and infers the properties that held throughout — pre- and post-conditions, ranges, ordering constraints — as *likely* program invariants, computed with no hand-written specification. Decades of use in spec discovery, test strengthening, and anomaly detection transfer directly to this row: mine invariants around the code about to be edited, and the harness possesses an executable, named behavioral spec of the unchanged contract. Post-edit, a violation is not a mysterious failure — it is "`list` was always sorted on exit; now it isn't," and that is precisely the human- and LLM-readable summary of what behavior is at stake that the review gate consumes. **Working Effectively with Legacy Code** (Michael C. Feathers, Prentice Hall PTR, 2004, ISBN 0131177052) is the deliberate outlier of the row — not a paper but a book — because characterisation testing, the core technique of pre-edit protection, comes from here [15]. The premise is that "legacy code is code without tests," and when no spec exists, *current behavior is the specification* — bugs included. Characterisation tests pin what the code *does* (outputs, side effects, ordering) so that change can proceed safely: any behavioral difference after editing surfaces as a test failure that *names the change* rather than guessing at damage. The "cover and modify" workflow — instrument the region, pin its behavior, then touch it — is precisely the pre-change net agents lack when they refactor code no test has ever executed. The implication is the first design principle: before editing under-tested code, *generate* the characterisation suite, and post-edit its diff is a precise, reviewable manifest of what changed — with the standing honesty that pinned bugs are protected too, so the manifest routes to review, never to auto-acceptance. **QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs** (Koen Claessen and John Hughes, Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming, ICFP 2000, DOI 10.1145/351240.351266) is the property-based form of the same idea, on its second cameo in the program [16]. Properties are standing specifications — relationships that must hold over randomly generated inputs — and they are the natural durable encoding of a mined invariant or a characterisation: where Daikon observes that a property held and Feathers pins a single behavior, a QuickCheck property asserts the relation over the whole input domain. The design transfer is the augmentation tier of Section 4: where a reliable oracle exists — a mined invariant, an existing behavior, a round-trip relation — property-based probes are the cheapest way to widen the net beyond the sampled suite. ### 2.6 Suite augmentation and the oracle problem **Automated patch assessment for program repair at scale** (He Ye, Matias Martinez, and Martin Monperrus, Empirical Software Engineering, Vol. 26, 2021, DOI 10.1007/s10664-020-09920-w) scales the overfitting problem far beyond hand-inspected repairs, and it is cited here with the discipline its provenance demands [17]. The abstract-level claim — a curated dataset of 638 patches for Defects4J generated by 14 repair systems, and Random testing with Ground Truth (RGT) improving automatic patch assessment by 190% through a better oracle — is confirmed. The finer body-level figures (4,477,707 generated tests; RGT identifying about 72% of overfitting patches versus 56% for PATCH-SIM; false-positive rate 8.25% to 2.3%) are not present in the abstract and could not be independently re-confirmed from primary metadata alone, so they are carried here with that caveat as plausible but unverified-at-metadata-level, not contradicted [17]. The lesson stands regardless of the exact figures: the original test suite is only a sample of intended behavior, and generated inputs can expose behavior that used to work and now does not. The caveat the same literature exposes is the oracle problem — generating an input is easy, knowing the correct output is hard — which is why this component reserves generated and property-based probes for the cases where the pre-patch program supplies the oracle for behavior that is supposed to remain unchanged: `baseline(input) == candidate(input)` over the unaffected domain. **Probe to Generate: Program Variant-Guided Test Augmentation for Repository-Level Repair Benchmarks** (Chenglin Li, Yisen Xu, Zehao Wang, Shin Hwei Tan, and Tse-Hsun (Peter) Chen, arXiv 2026, arXiv:2604.01518) is cited under its current title and identifier — the tool was named "STING" in an early version and renamed "Probe" in the arXiv revision, and the stale "STING: Mutation-Guided Test Augmentation" title resolves nowhere today [18]. The paper attacks the root cause of regression escape — weak suites that fail to detect broken unrelated behavior — by generating program variants that survive the existing suite yet are behaviorally incorrect, then synthesizing targeted tests that eliminate those gaps. Its verified figures are the row's benchmark-scale confirmation: 77% of SWE-bench Verified instances admit at least one surviving variant, and re-evaluating the top-10 repair agents against the augmented tests reduces resolved rates by 4.2% to 9.0% [18]. Read as: the suite the harness trusts is thinner exactly where agents make mistakes, and augmentation is how the net is widened without pretending the original suite was sufficient. **CodaMosa: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models** (Caroline Lemieux, Jeevana Priya Inala, Shuvendu K. Lahiri, and Siddhartha Sen, IEEE/ACM 45th International Conference on Software Engineering, ICSE 2023, DOI 10.1109/icse48619.2023.00085) is the search-plus-LLM test generator that closes coverage plateaus, cited by its DOI with no arXiv identifier [19]. Its place is the augmentation tier's search half: where a coverage plateau hides an untested behavior, an LLM-guided search reaches inputs that random generation would not, strengthening the net where it is thin. **Automated Unit Test Improvement using Large Language Models at Meta** (Nadia Alshahwan, Jubin Chheda, Anastasia Finogenova, Beliz Gokkaya, Mark Harman, Inna Harper, Alexandru Marginean, Shubho Sengupta, and Eddy Wang, Proceedings of the ACM on Software Engineering, FSE 2024, DOI 10.1145/3663529.3663839, arXiv:2402.09171) is the industrial proof that generated tests harden existing coverage before an edit lands [20]. **SynFix: Dependency-Aware Program Repair via RelationGraph Analysis** (Xunzhu Tang, Jiechao Gao, Jin Xu, Tiezhu Sun, Yewei Song, Saad Ezzini, Wendkûuni C. Ouédraogo, Jacques Klein, and Tegawendé F. Bissyandé, Findings of the Association for Computational Linguistics: ACL 2025, DOI 10.18653/v1/2025.findings-acl.252) addresses regression protection at the architectural level, ensuring repairs account for cross-component dependencies via RelationGraph analysis over classes, functions, variables, and their interactions, with patch validation against SWE-bench regression tests [21]; its reported 52.33%-of-SWE-bench-lite figure could not be re-spotted from the anthology abstract and is carried cautiously pending a PDF re-check. **When Automated Program Repair Meets Regression Testing — An Extensive Study on Two Million Patches** (Yiling Lou, Jun Yang, Samuel Benton, Dan Hao, Lin Tan, Zhenpeng Chen, Lu Zhang, and Lingming Zhang, ACM Transactions on Software Engineering and Methodology, 2024, DOI 10.1145/3672450) is the comprehensive study of regression test selection for automated program repair, cited by its real title and DOI rather than the earlier paraphrase [22]. Across more than two million patches, it establishes that method- and statement-level RTS substantially outperform class-level RTS and should be preferred, and it supplies the practical guidance for bridging APR and regression testing — the cost-order arithmetic that Section 4's selection tier is built on. **Interaction-Aware Patch Assessment for Multi-Fault Automated Program Repair** (Omar I. Al-Bataineh, IEEE/ACM 40th International Conference on Automated Software Engineering, ASE 2025, New Ideas and Emerging Results, DOI 10.1109/ase63991.2025.00319) is the caution that the regression-protection problem is exacerbated by multiple interacting faults, with a taxonomy of five fault-interaction-aware patch assessment strategies and the first formal treatment of masking and regression risk in multi-fault settings [23]. Its lesson: single-fault patch assessment can certify a patch that a second fault silently invalidates, so the harness's preservation obligations must survive interaction, not just isolation. ### 2.7 Surface gates: what no test can see **Why and how Java developers break APIs** (Aline Brito, Laerte Xavier, André Hora, and Marco Tulio Valente, IEEE 25th International Conference on Software Analysis, Evolution and Reengineering, SANER 2018, DOI 10.1109/saner.2018.8330214) is cited with its corrected venue — SANER 2018, not ICSME 2018 — and its corrected author order: Brito, Xavier, Hora, Valente [24]. It is the breaking-change anatomy behind the surface gates: the dangerous regressions are the ones with unchanged signatures — behavioral changes invisible to a compiler and to any test that only exercises the changed module. **Understanding the Impact of APIs Behavioral Breaking Changes on Client Applications** (Dhanushka Jayasuriya, Valerio Terragni, Jens Dietrich, and Kelly Blincoe, Proceedings of the ACM on Software Engineering, FSE 2024, DOI 10.1145/3643782) is the empirical confirmation that the dangerous behavioral breaking changes are the ones with unchanged signatures, propagating to client applications that no repository test can see [25]. Together they warrant the surface tier of Section 4: duck-typed call sites, string-keyed contracts, and public API consumers outside the repository are invisible to any suite, so the harness gates them by signature — type checks, semver, and API-diff — rather than by test. **SymDiff: A Language-Agnostic Semantic Diff Tool for Imperative Programs** (Shuvendu K. Lahiri, Chris Hawblitzel, Ming Kawaguchi, and Henrique Rebêlo, Computer Aided Verification, CAV 2012, LNCS 7358, DOI 10.1007/978-3-642-31424-7_54) and **Regression Verification: Proving the Equivalence of Similar Programs** (Benny Godlin and Ofer Strichman, Proceedings of the 46th Design Automation Conference, DAC 2009) are the proof-tier ceiling that the differential tier points toward, carried forward from the validation row [26][27]. SymDiff is cited with its corrected year — CAV 2012, not 2013 — and both establish that when two program versions are structurally similar, their equivalence on the unaffected input space can in principle be *proved* rather than sampled, which is the endpoint of this row's trajectory: regression protection as an equivalence argument over the untouched input space, with the stated intent defining the boundary. ### 2.8 The economics of what a harness skips, strengthens, and selects **Taming Google-scale continuous testing** (Atif Memon, Zebao Gao, Bao Nguyen, Sanjeev Dhanda, Eric Nickell, Rob Siemborski, and John Micco, IEEE/ACM 39th International Conference on Software Engineering, Software Engineering in Practice track, ICSE-SEIP 2017, DOI 10.1109/icse-seip.2017.16) is the industrial caution that makes the skip-log a safety feature rather than bookkeeping [28]. At Google scale, deleted tests had been catching real bugs — the minimisation warning of Yoo and Harman [12], measured at the largest scale that exists. **The Art of Testing Less without Sacrificing Quality** (Kim Herzig, Michaela Greiler, Jacek Czerwonka, and Brendan Murphy, IEEE/ACM 37th International Conference on Software Engineering, ICSE 2015, DOI 10.1109/ICSE.2015.66) is the test-impact-analysis counterpart — how to run less and still catch the same regressions [29]. Together they bound the selection tier from both sides: skip deliberately and track it, or the suite quietly shrinks while the report stays green. **SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark** (Boxi Yu, Yang Cao, Yuzhong Zhang, Liting Lin, Junjielong Xu, Zhiqing Zhong, Qinghua Xu, Guancheng Wang, Jialun Cao, Shing-Chi Cheung, Pinjia He, and Lionel Briand, arXiv 2026, arXiv:2603.00520) is the benchmark-scale confirmation that a weak suite greenlights patches that a stronger one rejects — the stronger suites reject 19.71% of previously passing patches, knocking the top agent's score from 78.8% to 62.2% [30]. **TDAD: Test-Driven Agentic Development — Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis** (Pepe Alonso, Sergio Yovine, and Victor A. Braberman, arXiv 2026, arXiv:2603.17973) is the paper that put a number on the problem this row exists to solve — a vanilla agent caused 562 pass-to-pass test failures across 100 instances, roughly 6.5 broken tests per patch — and its mechanism is the cheapest regression protection available: dependency-graph regression test selection run *before* submission, so indirect dependents get exercised without running the whole suite [31]. It is the proof that the selection machinery of Ekstazi [9] and HyRTS [10] composes directly with an LLM repair loop — the harness computes the impact graph, and the checks run because the harness runs them, not because the model chose to. **Can Old Tests Do New Tricks for Resolving SWE Issues?** (Yang Chen, Toufique Ahmed, Reyhaneh Jabbarvand, and Martin Hirzel, Proceedings of the Symposium on the Foundations of Software Engineering, FSE 2026, arXiv:2510.18270) is the TestPrune paper — coverage-based selection plus systematic minimization — the regression-test-selection paper written specifically for agents, yielding a 9.4%–12.9% relative increase in issue-resolution rate within the Agentless framework [32]. Its two integration modes matter directly: feeding selected tests as validation feedback inside an agent loop, and using them to pick one patch from tens of candidates. ## 3. Design Principles Eight principles follow from the review, each tied to the evidence above and each a clause of the strong Piper rule made mechanical. **P1 — The unchanged behavior is the specification, and it is pinned before the edit.** Before editing, the harness generates characterisation tests over the blast radius and mines invariants where instrumentation is cheap, producing an executable, named spec of everything that must not change. This is Feathers's cover-and-modify workflow [15] fused with Daikon's dynamic invariant detection [14]: pin outputs, side effects, and ordering before the edit, so that after the edit the pre-change program — the free oracle — is not gone but recorded. The pre-edit behavior net pre-dates the patch, which is what makes it trustworthy: it was built from behavior that already worked. **P2 — Preservation is proven by obligations, not by whatever tests exist.** The harness derives preservation obligations from impact analysis (component 06), and the discipline is "prove every identified preservation obligation" rather than "run whatever tests happen to exist." The regression surface is proportional to the diff's blast radius, not to the bug's — a changed function reaches direct callers, indirect callers, subclasses, serializers, config paths, and tests elsewhere, and every one of those is an obligation with evidence attached or a gap recorded as untested [2][31]. A patch whose untested obligations are non-empty is not "fully protected"; it is "partially evidenced," and the acceptance stage must see that distinction. **P3 — Select the smallest safe regression set, and log skips as unrun.** Checksum-based reachability (Ekstazi [9]) is the floor; risk-ranked top-slicing (Predictive Test Selection [13]) is the budget policy when the safe set exceeds what the loop can afford. Safety is the first property — the set must be a superset of what the change could affect, per Rothermel and Harrold [11] — and smallness is the second, because the thesis says the smallest *safe* set. Every unselected test is recorded as *unrun* with the selection reason; harness rollups distinguish green-from-running from green-from-assumption, and there is no silent assume-pass, ever. Minimisation — deleting tests to save compute — is red-flagged, per Yoo and Harman [12] and the Google-scale confirmation that deleted tests were catching real bugs [28]. **P4 — The pre-patch program is the oracle for unchanged behavior.** Preservation is a comparison against a known-good baseline, not an absolute pass/fail. The differential tier executes the changed region under the pre- and post-patch programs side by side (ChangeGuard [3], PATCH-SIM [4]) and flags divergence in return values, raised exceptions, execution paths, or mined invariants — divergence on previously correct behavior is regression evidence even when the suite is green. The asymmetry is PATCH-SIM's: known-good behavior prefers stability, known-bad behavior requires change [4]. **P5 — Distinguish the two overfitting directions.** A divergence that makes the bug test pass while a neighbor fails is *regression introduction* — previously correct behavior X, Y, Z now broken at Y. A patch that passes everything but still fails the bug on inputs B and C is *incomplete fix*. Regression protection targets the first, and the differential tier classifies the two deterministically: unexpected divergence on a previously-passing behavior is regression evidence, while the intended fix's own behavior change is the point. The incomplete-fix direction belongs to acceptance (component 18); the regression-introduction direction is this row's exact target. **P6 — Gate what tests cannot see.** Type checks, builds, and API/semver compatibility run alongside the behavioral tiers, because duck-typed call sites, string-keyed contracts, and public API consumers outside the repository are invisible to any suite [24][25]. A signature-unchanged behavioral break is exactly the class the surface tier exists to catch, and it is the complement to the suite, not a duplicate of it — SymDiff and Godlin and Strichman are the proof-tier ceiling of the same obligation [26][27]. **P7 — The characterisation diff is a manifest for review, not auto-acceptance.** Post-edit, every behavioral change is presented as an intentional-versus-accidental manifest: each entry marked *intentional* (part of the fix) or *accidental* (regression). Intentional entries update the pinned tests *through review*, never by the repair loop's own hand, because pinned bugs are protected too — the net is honest about protecting current behavior including its bugs, so the manifest routes to review rather than to auto-acceptance [15]. **P8 — The L column holds at most one entry: intent interpretation.** Selection, differential execution, invariant mining, surface gating, and the ledger are deterministic software on CPU. The single learned entry is intent interpretation — judging which observed divergences are the intended fix versus collateral regression, the step Testora and PatchGuru isolate [6][7]. That step is downstream of the deterministic divergence and cannot substitute for it: the model reads what a divergence means, never whether a divergence exists, and never authors the preservation verdict. The invariant is enforced mechanically: **the L column contains at most one entry.** ## 4. The Architecture The regression-protection stage is a pipeline of services around a single data structure — the PreservationLedger — and every node is labeled D (deterministic software) or L (a learned model). The invariant that governs the whole is: **the L column contains at most one entry, and the deterministic machinery is the default.** A pure protection run of a candidate patch has exactly one L entry — the intent-interpretation call — and everything else is deterministic software on CPU: the behavior net, the obligation derivation, the selection, the differential execution, the surface gates, the manifest, and the ledger itself. The model never decides whether a divergence is a regression; it judges, against the change's stated intent, which divergences are the fix and which are collateral, and that judgment is consumed as a hypothesis attached to the ledger, never as a verdict. The concrete input is a candidate patch (or candidate repository state), the pre-patch baseline, the repository graph and impact analysis from component 06, and the dynamic coverage from component 12; the output is a PreservationLedger whose regression_status is one of PROTECTED, DIVERGENT, or UNPROVEN — plus the evidence and the untested obligations that justify it. The deterministic services, in the order they run: 1. **pin_behavior_net [D].** Before the edit, auto-generate characterisation tests for every function in the blast radius (impact row's consumers), record current outputs, side effects, and ordering as the baseline (Feathers [15]), and mine invariants where instrumentation is cheap (Daikon [14]). The output is the pre-change executable spec — the standing claim of what must not change. 2. **derive_preservation_obligations [D].** From the impact graph, derive the named obligations: which behaviors are expected to remain unchanged, which call sites, serializers, config paths, and consumers each obligation reaches, and what evidence would discharge it. This is the "prove every identified preservation obligation" discipline of P2, replacing "run whatever tests exist" [2][31]. 3. **select_regression_set [D].** Compute the smallest safe regression set: checksum-based reachability as the floor (Ekstazi [9]), widened by method-level dependency analysis where precision pays (HyRTS [10]), and risk-ranked top-slicing when the safe set exceeds budget (Predictive Test Selection [13]). Every unselected test is logged as *unrun* with its selection reason — no silent assume-pass. Safety precedes smallness (Rothermel and Harrold [11]). 4. **run_differential [D].** Execute the changed region under the pre- and post-patch programs side by side (ChangeGuard [3], PATCH-SIM [4]), and classify each divergence: on previously-passing behavior it is regression evidence; on the bug-fix behavior it is the intended change. The pre-patch program is the oracle for everything the fix did not intend to touch. 5. **surface_gates [D].** Type checks, build, and API/semver compatibility run on the changed surface — the signature-level guard for consumers no test can see (Brito [24], Jayasuriya [25]), with SymDiff-style equivalence as the aspirational ceiling [26][27]. 6. **compute_characterization_diff [D].** Produce the intentional-versus-accidental manifest: every behavioral change after the patch, marked against the pinned net. The manifest is a diff, not a verdict — it routes to review. 7. **(L) interpret_intent [L] — the at-most-one learned entry.** The model reads each divergence in the manifest against the change's stated intent and proposes whether it is the intended fix or collateral regression (Testora [6], PatchGuru [7]). Its output is an annotation on the manifest, a hypothesis, never the verdict. The deterministic ledger computes regression_status from the evidence and the untested obligations; the model's annotations inform review, not the status. 8. **maintain_preservation_ledger [D].** Assemble obligations, evidence, unexpected_differences, untested_obligations, and regression_status into the PreservationLedger, and hand it to the acceptance stage (component 18). If untested_obligations is non-empty, acceptance sees *acceptance evidence incomplete*, not full protection — even when every known test is green. The PreservationLedger is the row's contract, and its shape makes the discipline visible: ``` PreservationLedger { patch_id obligations: [ // impact-derived, P2 { behavior, evidence: [test, baseline, candidate], impact_source: "SessionConfig -> SessionManager" } ] unexpected_differences: [] // differential tier, P4/P5 untested_obligations: [] // obligations with no evidence yet, P2 regression_status: PROTECTED | DIVERGENT | UNPROVEN } ``` Every field is written by deterministic software; the only learned input is the intent annotation on unexpected_differences. The target is explicit and checkable: selection, differential execution, invariant mining, surface gating, and the ledger are software on CPU, and the L column holds at most one entry. Any proposed addition that would put a language model elsewhere — deciding whether a divergence is a regression, silently greening a skipped test, updating a pinned characterisation test to match new behavior, or authoring the final regression_status — is a regression against P8 and the Piper rule, and must clear the same bar as in earlier components: beat the deterministic stage on the honest metrics of Section 6, on held-out tasks, at a defensible cost, or be rejected. The invariant is stated again so it can be enforced mechanically: **"the L column contains at most one entry."** ## 5. Recursive Self-Improvement Protocol Component 19 of 27 follows the program's uniform protocol: the executable specification is written to 100%, round 1 lands at approximately 80% of the specified behavior, and later rounds close the gap. Each round is A/B-gated against the honest metrics of Section 6 — above all preservation-coverage and skip-visibility — never the end-to-end resolve rate alone. The three-way split disciplines every change: intent interpretation goes to the LLM's single entry, mechanics go to deterministic software (the net, the obligations, the selection, the differential execution, the gates, the ledger), and every change to either is A/B-gated on honest metrics, never on vibes. Because preservation decides what "unchanged" even means, its improvement protocol measures the COVERAGE and HONESTY of the net rather than trusting the model's sense that "nothing broke," improving itself in four loops. First, *net calibration*: the harness measures whether the pinned net actually covers the blast radius — whether every function the diff touched has a characterisation test and every reachable consumer an obligation — and widens where it is thin, guided by the coverage-plateau work of CodaMosa [19] and the suite-strengthening lesson of Probe to Generate's 77% surviving-variant rate [18]. Second, *obligation coverage*: the harness measures what fraction of impact-derived obligations actually received evidence, and closes the gap between derived and discharged, guided by TDAD's pre-submission impact analysis [31] and Rothermel and Harrold's safety guarantee [11]; an obligation that is derived but never evidenced is the acceptance-stage's "incomplete evidence" flag, and the loop exists to make every derived obligation evidenced or explicitly recorded as untested. Third, *selection tightening*: the harness measures the false-negative rate of the regression-selection tier — how often a full-suite failure escaped the safe set — and widens coverage, guided by HyRTS's granularity arithmetic [10], Lou et al.'s method- and statement-level preference [22], and TestPrune's coverage-based minimization [32]; a selection tier that leaks regressions is not safe, and safety precedes smallness. Fourth, *net-honesty auditing*: the harness periodically re-checks what selection skipped — running dropped tests offline and tracking their historical kill record, the Google lesson [28] — so a selection policy that quietly shrinks the suite is surfaced as a falling kill record rather than a stable green. Because every run writes a PreservationLedger with untested_obligations and skip reasons, a regression in protection is visible in the coverage and honesty metrics before it surfaces as a shipped break, and each A/B change must justify itself on held-out tasks at a defensible cost before it ships. ## 6. Measurement & Evaluation Regression protection is measured *separately* and *honestly*, because a stage whose whole purpose is to prove the unchanged stayed the same can be defeated in either of two opposite ways: by certifying preservation on a net that never covered the blast radius, or by declaring safety while silently skipping the tests that would have caught the break. The instruments keep the net accountable to what each run actually covered and what it actually skipped, not to the count of green lights. **Preservation-coverage.** The primary instrument reports, for every patch, what fraction of the impact-derived obligations actually received evidence, versus what fraction remained untested. This is the Piper rule's second clause made quantitative: a harness that reports "full suite green" while half its derived obligations have no evidence scores near zero on this instrument, because it has not proved the unchanged stayed the same — it has only not-run the tests that would have said otherwise. **Skip-visibility.** The companion instrument reports the distribution of skip reasons across the regression-selection tier: how many tests were run, how many were unrun-by-reachability (Ekstazi [9]), how many unrun-by-budget (Predictive Test Selection [13]), and whether any unrun-by-budget test has a historical kill record. A harness whose report cannot distinguish green-from-running from green-from-assumption is a harness that is silently shrinking its own suite, and this instrument makes the shrinking visible. **Differential divergence rate.** The instrument reports how often the differential tier (ChangeGuard [3], PATCH-SIM [4]) surfaced a divergence that the suite missed — the direct measure of how much preservation the suite alone would have dropped. A rising divergence rate with a flat suite is the suite's blindness measured, and it is the warrant for widening the net (Probe to Generate [18]). **Intent-judgment agreement.** The single L entry is measured for what it is: the instrument reports how often the model's intent annotations on the manifest agreed with the eventual review outcome — was an annotated "intentional" divergence actually part of the fix, and an annotated "regression" actually collateral? This is the Testora/PatchGuru line measured honestly [6][7], and a weak model's intent judgment is expected to be its least reliable contribution, so it is never the verdict — it is a hypothesis scored against review. **Surface-gate catch rate.** The instrument reports how often the surface gates — type check, build, semver, API-diff — caught a break that no behavioral test could see, the Brito/Jayasuriya class of signature-unchanged behavioral regressions [24][25]. A zero rate is not proof of honesty; it is a signal that the gates are not actually installed. **Characterisation-manifest review rate.** The instrument reports what fraction of post-edit behavioral changes were surfaced as manifest entries, and of those, what fraction were marked intentional versus accidental at review. A harness that auto-accepts its own manifest — updating pinned tests to match new behavior without review — is the net-tampering failure mode, and this instrument catches it. **Untested-obligation escalation rate.** The instrument reports how often a patch reached acceptance with untested_obligations non-empty, and whether acceptance downgraded it to "evidence incomplete" as the design requires. This is the loop that keeps component 18 honest about what component 19 did not prove. A failure-mode audit closes the positive instruments by naming how a preservation net lies. *Assume-pass amnesia* is skipped tests counted green, the suite quietly shrinking while the report stays green [12][28]. *Characterisation lock-in* is pinned behavior — bugs included — treated as correct forever, the net becoming a cage that vetoes the actual fix [15]. *Net-tampering* is the loop "fixing" regressions by updating characterisation tests to match new (wrong) behavior, the tamper class wearing this row's costume. *Over-minimisation* is aggressive pruning for compute whose coverage holes are discovered by the incident [12]. *Stale invariants* is mined specs encoding obsolete behavior firing rejection storms — and getting ignored, taking real violations' credibility with them [14]. *Surface blindness* is all internal tests green while the public API is broken, downstream consumers as the regression detector of last resort [24][25]. *Intermediate-state rot* is a final diff green while rounds 2 through 4 broke and un-broke neighbors — invisible without per-edit runs, fatal if any consumer snapshots mid-task. *Incomplete-versus-introduced conflation* is the two overfitting directions collapsed: a patch that never fully fixed the bug recorded as "protected" because the net was aimed only at the regression-introduction direction [17]. Two honest caveats close the section. First, the strong-pipeline numbers of Section 2 — ChangeGuard's 77.1%/69.5%/7.6% [3], PATCH-SIM's 56.3% [4], Probe to Generate's 77% and 4.2%–9.0% [18], HyRTS's 18.35% [10], TestPrune's 9.4%–12.9% [32] — were measured on repositories and models far larger than a weak-model harness controls, and a weak model should expect materially smaller absolute effects while leaning all the harder on the cheap tiers, the pinned net, and the smallest-safe-set discipline to make its small budget count. Second, all preservation metrics must be reported on held-out and out-of-benchmark tasks, as in the earlier components, so a net that has memorized a task set's answers cannot score well on coverage without its preservation verdicts having been measured on problems it never saw. ## 7. Conclusion Regression protection is where the coding harness's most expensive lie is told in its cheapest-looking form: a green suite after a fix. This paper has argued that preservation is not the residue of running whatever tests happen to exist, but a first-class obligation that must be derived, selected, executed, and evidenced — and that the single most powerful oracle for it is free, the pre-patch program whose behavior on everything the fix did not intend to change is correct by definition. The thesis that has governed every design choice is the one with which this paper began and closes: **Regression protection should be differential, impact-driven, and preservation-oriented: before editing, identify behavior that is expected to remain unchanged; derive preservation obligations from dependency and impact analysis; select the smallest safe set of regression tests using static and dynamic dependencies; compare patched behavior against the known-good baseline rather than relying solely on absolute pass/fail; treat unexpected execution or output divergence on previously correct behavior as regression evidence; augment incomplete suites with generated or property-based probes where reliable oracles exist; and escalate toward broader regression suites as the change's impact or uncertainty grows.** And the rule that compresses that thesis into one sentence of discipline for Piper is: **A bug fix changes what was wrong. Regression protection proves that everything we did not intend to change stayed the same.** The machinery is deterministic: the behavior net pinned before the edit, the obligations derived from impact, the selection logged skip-by-skip, the differential execution against the known-good baseline, the surface gates for what tests cannot see, and the PreservationLedger that reports PROTECTED, DIVERGENT, or UNPROVEN with its untested obligations in plain sight — because a preservation verdict a weak model could flatter is not a verdict. The single learned entry is intent interpretation, reading which divergences are the fix and which are collateral, downstream of the deterministic divergence and never the verdict. The architecture holds the invariant mechanically: the L column contains at most one entry, and the deterministic machinery is the default. The measured message of the literature is unambiguous — REFINE named the patch that fits the suite rather than the bug [1], SWE-bench made PASS_TO_PASS the score this row earns [2], ChangeGuard and PATCH-SIM showed the pre-patch program is the free oracle for unchanged behavior [3][4], Testora and PatchGuru isolated intent as the one learned step [6][7], Ekstazi and HyRTS made selection cheap enough to run per-edit [9][10], Daikon and Feathers supplied the pre-edit executable spec [14][15], and Memon's Google-scale finding showed that what a harness skips had been catching real bugs [28]. For a weak model, "the tests still pass" is an assumption wearing a confident face; what a weak-model harness needs is a pinned, impact-derived, differentially-checked, skip-visible preservation net that knows exactly which unchanged behaviors it proved and which it did not — because the question is never whether the tests passed, but whether everything the fix did not intend to change provably stayed the same. --- ## References 1. Anvith Pabba, Simin Chen, Alex Mathai, Anindya Chakraborty, and Baishakhi Ray. *REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement.* arXiv 2025. arXiv:2510.03588. *(Preprint, October 2025, no venue. +14.67pp SWE-bench Lite / +12.2pp SWE-bench Verified over AutoCodeRover; preprint figures to be locally reproduced.)* 2. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. *SWE-bench: Can Language Models Resolve Real-World GitHub Issues?* ICLR 2024. arXiv:2310.06770. *(2,294 issue/PR pairs, 12 Python repos; FAIL_TO_PASS/PASS_TO_PASS; PASS_TO_PASS is this row's operational metric.)* 3. Lars Gröninger, Beatriz Souza, and Michael Pradel. *ChangeGuard: Validating Code Changes via Pairwise Learning-Guided Execution.* Proceedings of the ACM on Software Engineering, FSE 2025. DOI 10.1145/3715760. arXiv:2410.16092. *(77.1% precision, 69.5% recall; existing tests catch 7.6%.)* 4. Yingfei Xiong, Xinyuan Liu, Muhan Zeng, Lu Zhang, and Gang Huang. *Identifying patch correctness in test-based program repair.* Proceedings of the 40th International Conference on Software Engineering, ICSE 2018. DOI 10.1145/3180155.3180182. *(PATCH-SIM; 139 patches, 56.3% of incorrect filtered, none correct blocked. Distinct from DiffTGen and DiffTestGen.)* 5. Qi Xin and Steven P. Reiss. *Identifying test-suite-overfitted patches through test case generation.* Proceedings of the 2017 International Symposium on Software Testing and Analysis, ISSTA 2017. DOI 10.1145/3092703.3092718. *(DiffTGen; pre-LLM differential test generation for overfitting patches.)* 6. Michael Pradel. *Testora: Using Natural Language Intent to Detect Behavioral Regressions.* IEEE/ACM International Conference on Software Engineering, ICSE 2026. arXiv:2503.18597. *(19 bugs found, 13 reported, 11 confirmed, 9 fixed; ~12.3 min / $0.003 per PR.)* 7. Thanh Le-Cong, Bach Le, Toby Murray, Michael Pradel, and Cristian Cadar. *PatchGuru: Patch Oracle Inference from Natural Language Artifacts.* arXiv 2026. arXiv:2602.05270. *(Feb 2026 preprint; 39 warnings, precision 0.62, 24 confirmed TP; vs Testora 24-vs-7 bugs, precision 0.32→0.62.)* 8. Arindam Sharma, Daniel Schemmel, and Cristian Cadar. *P³: Reasoning about Patches via Product Programs.* Proceedings of the ACM on Programming Languages, PACMPL/OOPSLA 2025. DOI 10.1145/3763145. *(Product programs compose pre/post-patch versions for symbolic divergence reasoning.)* 9. Milos Gligoric, Lamyaa Eloussi, and Darko Marinov. *Ekstazi: Lightweight Test Selection.* IEEE/ACM 37th International Conference on Software Engineering, ICSE 2015, Software Engineering in Practice track. DOI 10.1109/ICSE.2015.230. *(Checksum-based transitive file-dependency selection; the selection floor.)* 10. Lingming Zhang. *Hybrid Regression Test Selection.* IEEE/ACM 40th International Conference on Software Engineering, ICSE 2018. DOI 10.1145/3180155.3180198. *(2,707 revisions, 32 Java projects, 124M+ LoC; 18.35% vs 27.18%; 31.60s vs 40.06s.)* 11. Gregg Rothermel and Mary Jean Harrold. *A Safe, Efficient Regression Test Selection Technique.* ACM Transactions on Software Engineering and Methodology, 1997. DOI 10.1145/248233.248262. *(Safe-RTS foundation: select a superset of affected tests.)* 12. S. Yoo and M. Harman. *Regression testing minimization, selection and prioritization: a survey.* Software Testing, Verification and Reliability, 2012. DOI 10.1002/stvr.430. *(Minimisation is dangerous; redundancy is insurance.)* 13. Mateusz Machalica, Alex Samylkin, Meredith Porth, and Satish Chandra. *Predictive Test Selection.* IEEE/ACM 41st International Conference on Software Engineering, Software Engineering in Practice track, ICSE-SEIP 2019. DOI 10.1109/icse-seip.2019.00018. *(CORRECTED: Porth/Chandra, ICSE-SEIP 2019 — not "Piscotto/Yang 2021".)* 14. Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. *Dynamically discovering likely program invariants to support program evolution.* Proceedings of the 21st International Conference on Software Engineering, ICSE 1999, DOI 10.1145/302405.302467; and IEEE Transactions on Software Engineering, Vol. 27, No. 2, February 2001, DOI 10.1109/32.908957. *(Daikon; CORRECTED journal venue: IEEE TSE 2001, not TOSEM 2001.)* 15. Michael C. Feathers. *Working Effectively with Legacy Code.* Prentice Hall PTR, 2004. ISBN 0131177052. *(Characterisation testing; current behavior is the specification, bugs included.)* 16. Koen Claessen and John Hughes. *QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs.* Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming, ICFP 2000. DOI 10.1145/351240.351266. *(Properties as standing specifications.)* 17. He Ye, Matias Martinez, and Martin Monperrus. *Automated patch assessment for program repair at scale.* Empirical Software Engineering, Vol. 26, 2021. DOI 10.1007/s10664-020-09920-w. *(638 patches / 14 systems / RGT +190% confirmed in abstract; the 4,477,707-test and 72%-vs-56% and 8.25%→2.3% figures are body-level and not re-confirmed from metadata.)* 18. Chenglin Li, Yisen Xu, Zehao Wang, Shin Hwei Tan, and Tse-Hsun (Peter) Chen. *Probe to Generate: Program Variant-Guided Test Augmentation for Repository-Level Repair Benchmarks.* arXiv 2026. arXiv:2604.01518. *(Renamed from "STING"; 77% admit a surviving variant; top-10 agents drop 4.2%–9.0%.)* 19. Caroline Lemieux, Jeevana Priya Inala, Shuvendu K. Lahiri, and Siddhartha Sen. *CodaMosa: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models.* IEEE/ACM 45th International Conference on Software Engineering, ICSE 2023. DOI 10.1109/icse48619.2023.00085. *(Not on arXiv; cite DOI.)* 20. Nadia Alshahwan, Jubin Chheda, Anastasia Finogenova, Beliz Gokkaya, Mark Harman, Inna Harper, Alexandru Marginean, Shubho Sengupta, and Eddy Wang. *Automated Unit Test Improvement using Large Language Models at Meta.* Proceedings of the ACM on Software Engineering, FSE 2024. DOI 10.1145/3663529.3663839. arXiv:2402.09171. *(Generated tests hardening coverage before an edit lands.)* 21. Xunzhu Tang, Jiechao Gao, Jin Xu, Tiezhu Sun, Yewei Song, Saad Ezzini, Wendkûuni C. Ouédraogo, Jacques Klein, and Tegawendé F. Bissyandé. *SynFix: Dependency-Aware Program Repair via RelationGraph Analysis.* Findings of the Association for Computational Linguistics: ACL 2025. DOI 10.18653/v1/2025.findings-acl.252. *(The 52.33%-of-SWE-bench-lite figure is carried cautiously pending a PDF re-check.)* 22. Yiling Lou, Jun Yang, Samuel Benton, Dan Hao, Lin Tan, Zhenpeng Chen, Lu Zhang, and Lingming Zhang. *When Automated Program Repair Meets Regression Testing — An Extensive Study on Two Million Patches.* ACM Transactions on Software Engineering and Methodology, 2024. DOI 10.1145/3672450. *(Method- and statement-level RTS preferred over class-level.)* 23. Omar I. Al-Bataineh. *Interaction-Aware Patch Assessment for Multi-Fault Automated Program Repair.* IEEE/ACM 40th International Conference on Automated Software Engineering, ASE 2025, New Ideas and Emerging Results. DOI 10.1109/ase63991.2025.00319. *(Single-fault assessment can miss interaction masking and regression risk.)* 24. Aline Brito, Laerte Xavier, André Hora, and Marco Tulio Valente. *Why and how Java developers break APIs.* IEEE 25th International Conference on Software Analysis, Evolution and Reengineering, SANER 2018. DOI 10.1109/saner.2018.8330214. *(CORRECTED: SANER 2018, author order Brito, Xavier, Hora, Valente.)* 25. Dhanushka Jayasuriya, Valerio Terragni, Jens Dietrich, and Kelly Blincoe. *Understanding the Impact of APIs Behavioral Breaking Changes on Client Applications.* Proceedings of the ACM on Software Engineering, FSE 2024. DOI 10.1145/3643782. *(Behavioral breaking changes with unchanged signatures.)* 26. Shuvendu K. Lahiri, Chris Hawblitzel, Ming Kawaguchi, and Henrique Rebêlo. *SymDiff: A Language-Agnostic Semantic Diff Tool for Imperative Programs.* Computer Aided Verification, CAV 2012, LNCS 7358. DOI 10.1007/978-3-642-31424-7_54. *(CORRECTED: CAV 2012, not 2013.)* 27. Benny Godlin and Ofer Strichman. *Regression Verification: Proving the Equivalence of Similar Programs.* Proceedings of the 46th Design Automation Conference, DAC 2009. *(Equivalence proof over the unaffected input space.)* 28. Atif Memon, Zebao Gao, Bao Nguyen, Sanjeev Dhanda, Eric Nickell, Rob Siemborski, and John Micco. *Taming Google-scale continuous testing.* IEEE/ACM 39th International Conference on Software Engineering, Software Engineering in Practice track, ICSE-SEIP 2017. DOI 10.1109/icse-seip.2017.16. *(Deleted tests had been catching real bugs.)* 29. Kim Herzig, Michaela Greiler, Jacek Czerwonka, and Brendan Murphy. *The Art of Testing Less without Sacrificing Quality.* IEEE/ACM 37th International Conference on Software Engineering, ICSE 2015. DOI 10.1109/ICSE.2015.66. *(Test impact analysis at industrial scale.)* 30. Boxi Yu, Yang Cao, Yuzhong Zhang, Liting Lin, Junjielong Xu, Zhiqing Zhong, Qinghua Xu, Guancheng Wang, Jialun Cao, Shing-Chi Cheung, Pinjia He, and Lionel Briand. *SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark.* arXiv 2026. arXiv:2603.00520. *(19.71% of previously passing patches rejected; top agent 78.8% → 62.2%.)* 31. Pepe Alonso, Sergio Yovine, and Victor A. Braberman. *TDAD: Test-Driven Agentic Development — Reducing Code Regressions in AI Coding Agents via Graph-Based Impact Analysis.* arXiv 2026. arXiv:2603.17973. *(562 pass-to-pass failures across 100 instances; dependency-graph RTS before submission.)* 32. Yang Chen, Toufique Ahmed, Reyhaneh Jabbarvand, and Martin Hirzel. *Can Old Tests Do New Tricks for Resolving SWE Issues?* Proceedings of the Symposium on the Foundations of Software Engineering, FSE 2026. arXiv:2510.18270. *(TestPrune; coverage-based selection minimized to k = 10; 9.4%–12.9% relative gain.)*