Summary / 16. Candidate generation and competing hypotheses / Document
Paper 2026-09-06
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Never Marry Your First Sample: Hypothesis-Diverse, Evidence-Constrained Candidate Generation for a Weak-Model Coding Harness *Component 16 of 27 — Coding-Harness Program. This document specifies the candidate-generation stage, the moment a decomposed request first meets solution space and the harness commits to the set of explanations it will test. It is the stage whose product everything downstream consumes: execution falsifies these hypotheses, the verification components judge them, component 14 repairs the patch of whichever hypothesis survives, and component 15 reflects on the failures the survivors leave behind. The sharp distinction is between a SAMPLE and a HYPOTHESIS: a sample is a stochastic draw conditioned on the prompt; a hypothesis is a causally distinct claim about the correct behavior and how to produce it, carrying a behavioral prediction reality can confirm or contradict. Naive harnesses take the first plausible sample, or generate many and let brute force find a winner; this component argues that a weak-model harness must construct hypotheses deliberately, spread them across genuinely different strategies and frames rather than across temperature draws, let cheap referees and finally the expensive oracle eliminate the ones reality contradicts, record why each was eliminated so it is never silently regenerated, branch the survivors where evidence warrants, and hoard cross-candidate agreement as deterministic evidence that later guides selection and patch fusion. This paper translates the program's thesis into the candidate-generation build contract: run candidate generation as a deterministic pipeline — frame decomposition, diversity measurement, deduplication, behavioral clustering, cheap-referee screening, branching, evidence recording, falsification pruning, agreement extraction — in which the weak model's one contribution is a single learned step of SEMANTIC JUDGMENT (propose hypotheses and repair strategies from distinct frames), and every other decision is deterministic software that keeps duplicates from consuming the budget and keeps contradicted hypotheses from coming back.* --- ## Abstract Candidate generation is where a harness decides what it will test, and it is where the program's most seductive shortcut lives: take the model's first answer, or the most likely one, or the one that came from cranking the temperature up and hoping. The naive generators share one failure — they treat the model as a source of interchangeable stochastic samples rather than of distinct ideas, and never let reality do the pruning only reality can do. This paper argues that candidate generation must be hypothesis-diverse, evidence-constrained, and search-aware rather than a loop over temperature draws, and reviews the candidate-generation and diversity literature in twenty-four parts to make the case. AlphaCode showed that enormous sampling plus model-based filtering can win competitions but only at a compute scale a weak harness cannot afford [1]; Large Language Monkeys showed more samples without better selection buy ever-less and approach random [2]; Self-Consistency showed majority voting helps but silently counts identical answers [3]; Verbalized Sampling showed diversity is unlocked by prompting for distinct ideas, not temperature [4]; CodeT showed generated tests, run deterministically, are the referee that selects good code over plausible code — 65.8% pass@1 on HumanEval [5]; Tree of Thoughts [6], LATS [7], and SWE-Search [8] showed generation is improved by treating hypotheses as a search space expanded under evidence; Multiagent Debate established debate improves reasoning [9] while the Degeneration-of-Thought result established that naive harmonization can DEGRADE accuracy by collapsing onto a wrong shared mode [10], and MPSC [11], TraceRepair [12], and SWE-Debate [13] carried divergence-by-construction into software; Reflexion showed that regenerating with knowledge of what already failed beats regenerating from scratch [14]; CodeMonkeys quantified that SELECTION over a pool is worth real points (66.2% ensemble-selected versus 62.8% best member versus 60.9% random) [15] and DEI showed agent diversity is what makes the ensemble win [16]; verifier reranking supplied the learned-oracle lineage [17] and Adaptive-Consistency showed the budget itself can adapt, cutting sample cost by up to 7.9× as an upper bound [18]; PatchFusion argued no single patch suffices and candidates should be fused deterministically [19]; the syntax-guided and synthesis lineage — Recoder [20], SketchFix [21], Trio [22] — generated at the edit and structure level rather than as free text; and beneath all of it Chamberlin's method of multiple working hypotheses [23] and Platt's strong inference [24] are the original statement that progress comes from holding several hypotheses and letting evidence exclude them. From the review this paper derives seven design principles, then specifies a deterministic pipeline (D) with exactly one learned entry (L): frame decomposition and falsification intake, candidate proposal, schema binding, diversity measurement and deduplication, behavioral clustering, cheap-referee screening, branching and budget enforcement, oracle verification and evidence recording, falsification pruning and constraint compilation, and agreement extraction are deterministic software, and the single L entry is the frame-conditioned proposal of hypotheses and repair strategies. It closes with a recursive self-improvement protocol and a measurement plan whose honest instruments — candidate diversity, budget efficiency, duplicate rate, falsification precision, falsified-rebirth leak rate, and agreement-evidence yield — exist to keep the harness from mistaking five copies of the same answer for five ideas. ## 1. Introduction The coding-harness program has, through components 01 through 15, assembled machinery that reads a repository, decomposes a request, generates a candidate, runs it under deterministic execution, repairs it when it fails, and distills what the failure taught into durable memory. Component 16 stands at the first of those acts — the generation of the candidate — and it is where the whole program's leverage is won or lost, because every downstream stage can only be as good as the hypothesis set it is given: a repair loop cannot repair a hypothesis never given, a verifier cannot pass a candidate never generated, and a reflection stage cannot learn from a failure the harness never attempted because it settled for the first plausible sample. The naive answer is to let the weak model generate once and go, or to generate many times and let the law of large numbers sort it out; this paper argues that both are the wrong shape for a weak model. Single-shot generation marries the harness to its first sample — the most probable token path, which is also the most confidently-wrong one. Unguided repeated sampling treats the model as a lottery: it burns a candidate budget on near-identical draws from the same mode, spends the budget before the genuinely different idea is ever drawn, and then selects among duplicates as if it had selected among ideas. The thesis that governs this component is stated plainly. **Candidate generation should be hypothesis-diverse, evidence-constrained, and search-aware rather than repeated stochastic sampling: explicitly construct multiple causally distinct explanations and repair strategies; record the behavioral prediction that follows from each; eliminate hypotheses contradicted by execution, tests, traces, or invariants; prevent falsified ideas from being regenerated; branch promising hypotheses into multiple repair candidates where warranted; measure semantic and structural diversity so duplicates do not consume the candidate budget; and preserve cross-candidate agreement as evidence that can later guide selection or deterministic patch fusion.** And it carries a strong rule for Piper that compresses the whole thesis into one sentence, stated verbatim: **Five answers are valuable only if they represent five different ideas. Generate disagreement deliberately, then let reality collapse the search space.** The first clause is the design's north star — value is a property of the SET's diversity, not its size, so the harness must manufacture difference on purpose because the model will not volunteer it (Verbalized Sampling [4] and the divergence penalty [10] are the proof that distinct ideas must be elicited, not assumed). The second clause is the whole architecture — after the disagreement is generated, reality — execution, tests, traces, invariants — does the collapsing, deterministically, never the model's self-evaluation about which of its own ideas it likes best. The weak-model problem makes candidate generation the make-or-break stage precisely because a weak model is where the two naive strategies fail hardest: its most-probable first sample is wrong more often, so single-shot generation fails more often, and its samples are more concentrated around its few strongly-held (and strongly-wrong) modes, so repeated sampling produces duplicates faster and spends the budget on restatements of the same error sooner. The trap is real and quantified: AlphaCode's million-sample-and-filter pipeline [1] and Large Language Monkeys' demonstration that unselected sampling drifts toward random [2] show that raw volume is a weak lever once the model's few modes are exhausted. The escape is diversity-then-selection, measured on software engineering with unusual clarity: CodeMonkeys reports that selecting over a candidate ensemble reaches 66.2% while random selection from the same pool reaches only 60.9% — nearly six points that live entirely in the SELECTION step [15] — and DEI reports that diversity among constituent agents is what powers the ensemble [16]. The design implication is decisive and it is the thesis: a weak-model harness must spend its budget acquiring genuinely different hypotheses — different causal stories, repair strategies, and behavioral predictions — and then let a cheap referee and finally the real oracle eliminate the ones reality contradicts, so that the samples that remain are the ones that survived being checked, not the ones the model happened to like. The paper proceeds as follows. Section 2 reviews the candidate-generation literature in twenty-four parts — the repeated-sampling baseline and its cost, verbalized diversity against mode collapse, tests as the deterministic referee, search over hypothesis space, competing hypotheses by construction through debate and divergence, conditioned regeneration, selection over candidate sets, verifiers and oracle reranking, deterministic fusion and syntax-guided repair, and the classical roots in Chamberlin and Platt. Section 3 derives seven design principles. Section 4 specifies the architecture, labeling every node deterministic (D) or learned (L) and stating the invariant that the L column holds exactly one entry. Section 5 states the recursive self-improvement protocol. Section 6 gives the measurement plan. Section 7 concludes. ## 2. Related Work The literature divides into ten parts that together answer one question: what is candidate generation FOR, and how must it be built so a weak model's budget buys the most distinct ideas it can hold? Each part supplies one design lever the candidate stage must pull. ### 2.1 The repeated-sampling baseline and its measured cost **AlphaCode** (Li, Choi, Chung, Kushman, Schrittwieser, Leblond, Eccles, Keeling, Gimeno, Dal Lago, Hubert, Choy, de Masson d'Autume, Babuschkin, Chen, Huang, Welbl, Gowal, Cherepanov, Molloy, Mankowitz, Sutherland Robson, Kohli, de Freitas, Kavukcuoglu, and Vinyals, Science 2022, arXiv:2203.07814) proved that brute-force sampling plus model-based filtering can win competitive programming, and is the caution a weak-model harness cannot follow: it generated on the order of a million samples per problem, filtered them with a model trained to recognize which pass hidden tests, and clustered the survivors to submit diverse candidates [1]. It is the ancestor of everything this component admires — massive sampling, model-based selection, diversity-preserving clustering — and a scale it rejects, because a harness generating a million samples per request is not a harness but a supercomputer. The transferable lesson is the SELECTION and DIVERSITY machinery, not the budget. **Large Language Monkeys** (Brown, Juravsky, Ehrlich, Clark, Le, Ré, and Mirhoseini, 2024, arXiv:2407.21787) quantifies the trap at the heart of repeated sampling: the gain per sample collapses as the model's modes are exhausted, and samples drawn without a selection mechanism behave almost as badly as random [2]. That is the direct warrant for rejecting naive sampling — at the margin, generating N and taking one at random IS random selection, and CodeMonkeys [15] shows random selection from a good pool still leaves nearly six points on the table. More samples is the wrong axis; the budget must be spent on DIFFERENT samples — a diversity problem, not a volume problem. **Self-Consistency** (Wang, Wei, Schuurmans, Le, Chi, Narang, Chowdhery, and Zhou, ICLR 2023, arXiv:2203.11171) contributes the majority-vote mechanism that is simultaneously the most popular cheap aggregation and the clearest illustration of why duplicates must be measured. Sampling multiple reasoning paths and taking the majority improves reasoning across tasks [3], but the mechanism counts identical answers as agreeing votes, so a mode the model over-samples — the most likely, often wrong, answer — accumulates votes purely because it is drawn more often. Majority vote rewards REPETITION, not correctness, unless the generator first ensures that what looks like five votes is five different ideas. ### 2.2 Verbalized diversity against mode collapse **Verbalized Sampling** (Zhang, Yu, Chong, Sicilia, Tomz, Manning, and Shi, 2025, arXiv:2510.01171) reframes diversity as something elicited rather than assumed, and is the empirical anchor for refusing to diversify by temperature. Its premise is mode collapse: models asked to sample repeatedly concentrate draws in a small set of modes, and temperature alone is a weak instrument for escaping them. The mechanism is to verbalize the diversity — ask the model, in a first pass, to NAME distinct approaches, then sample conditioned on those named ideas — which unlocks diversity that temperature-scrambling does not [4]. The transfer becomes principle P2: the way to get five different answers is to ask for five different strategies or frames FIRST and generate per frame, not to sample five times from one prompt and hope the temperature knob manufactures disagreement. A weak model told "these five strategies are all live, propose from each" produces genuinely different candidates; the same model told "sample five times" produces five near-identical draws from its dominant mode. ### 2.3 Tests as the deterministic referee **CodeT** (Chen, Zhang, Nguyen, Zan, Lin, Lou, and Chen, ICLR 2023, arXiv:2207.10397) is the ancestor of this component's cheap-referee design, being among the first to show that the right way to choose among generated code candidates is not to ask the model which it likes but to run generated tests and let deterministic execution decide. CodeT generates code samples AND test cases, runs each candidate against the tests, and selects the one that passes — a filter that does not depend on the generator's self-assessment. It improves pass@1 on HumanEval to 65.8%, an absolute +18.8% over code-davinci-002 and more than 20% over the prior state of the art [5]. Two corrections belong with the citation and are not repeated from the earlier source: the correct identifier is arXiv:2207.10397 (2202.11305 resolves to an unrelated topology paper and must never be used for CodeT), the venue is ICLR 2023 and not 2022, and the headline is 65.8% pass@1 / absolute +18.8% — not a "~1.8×" framing, which does not appear in the paper. The design transfer is the referee: deterministic execution against tests is a referee that cannot be flattered, adopted wholesale in stage (e) — run the cheap referee first so the expensive oracle is spent only on candidates the referee did not already falsify. ### 2.4 Search over hypothesis space **Tree of Thoughts** (Yao, Yu, Zhao, Shafran, Griffiths, Cao, and Narasimhan, NeurIPS 2023, arXiv:2305.10601) is the seminal demonstration that a model solves harder problems by deliberately exploring multiple candidate next-steps as branches of a tree, evaluating them, and backtracking from dead ends, rather than by greedy decoding [6]. Its weakness for this component is that the branch evaluation is done by the model itself, reintroducing the self-evaluation the whole program distrusts; ToT is adopted here as the SHAPE — explicit branching over candidate continuations — not the evaluator. **LATS** (Zhou, Yan, Shlapentokh-Rothman, Wang, and Wang, ICML 2024, arXiv:2310.04406) unifies reasoning, acting, and planning in a language-agent tree search with Monte Carlo tree search, grounding branch value in ENVIRONMENT feedback rather than the model's own estimate [7] — precisely the evidence-constrained posture this component demands. **SWE-Search** (Antoniades, Örwall, Zhang, Xie, Goyal, and Wang, ICLR 2025, arXiv:2410.20285) carries the pattern to software, combining MCTS with iterative refinement and reporting a 23% relative improvement across five model backbones versus standard open-source agents without MCTS [8]. Together the three warrant the thesis's "search-aware" clause: expand the most promising branches and abandon contradicted ones on the evidence, rather than sampling a fixed number of i.i.d. guesses. The candidate stage inherits the discipline as the branching stage (f). ### 2.5 Competing hypotheses by construction: debate, divergence, and multi-perspective sampling The debate lineage is where the thesis's demand for MANUFACTURED disagreement finds its strongest coding-side evidence, and it must be cited with an attribution the earlier source got wrong. **Multiagent Debate** (Du, Li, Torralba, Tenenbaum, and Mordatch, ICML 2024, arXiv:2305.14325) is the original result that multiple agents reasoning independently then debating improves factuality and reasoning [9]. Du et al. argue debate IMPROVES reasoning; they do not report that naive harmonization degrades it. That finding — that naive consensus-seeking, in which one agent's answer becomes another's prompt in a tit-for-tat harmonization, can DEGRADE accuracy by collapsing the set onto the same wrong mode — belongs to a different paper. **Encouraging Divergent Thinking** (Liang, He, Jiao, Wang, Wang, Wang, Yang, Shi, and Tu, 2023, arXiv:2305.19118) introduces the Degeneration-of-Thought problem and shows that when debate degenerates into agreement the agents converge on a shared incorrect answer; its remedy is a divergence penalty that explicitly forces DIFFERENT answers, and it reports that manufactured divergence beats both naive consensus and single-agent reasoning [10]. This is the single most important citation for the Piper rule: Degeneration-of-Thought is the empirical proof that agreement among agents is not evidence of correctness but evidence of mode collapse, and that the diversity the thesis demands must be manufactured against the agents' own inclination to harmonize. The coding-side descendants carry the pattern into software. **MPSC** (Huang, Lu, Wan, and Duan, ACL 2024, 2024.acl-long.78, arXiv:2309.17272) reports that prompting a coding model from MULTIPLE PERSPECTIVES and taking a consistent answer improves coding substantially — +15.91% on HumanEval, +6.43% on MBPP, and +9.37% on CodeContests [11]; the author list here is the ACL-published four-author list, and this paper stays consistent with it (arXiv v3 carries a fifth author, Weizhu Chen). **TraceRepair** (Wu, Wu, Zhang, Dong, and Shen, 2026, arXiv:2604.02647) brings multi-strategy debate to program repair and grounds it in the repair trace — "From Guessing to Seeing" — reporting that trace-guided debate correctly fixes 392 defects in Defects4J [12]; the "ArcXiv" spelling in the earlier source was a typo, the identifier resolves cleanly, and the 392-Defects4J figure is confirmed verbatim in the abstract. **SWE-Debate** (Li, Shi, Lin, Gu, Lian, Wang, Jia, Huang, and Wang, 2025, arXiv:2507.23348) is the competitive-debate framing for software issue resolution, in which agents propose competing resolutions and debate to converge on or refine the winner [13]. Together the four establish that for coding, as for reasoning, the reliable route to a good answer is to construct several genuinely different candidate explanations or repair strategies and let debate, tests, or consistency select. ### 2.6 Conditioned regeneration: what already failed must shape what comes next **Reflexion** (Shinn, Cassano, Berman, Gopinath, Narasimhan, and Yao, NeurIPS 2023, arXiv:2303.11366) is the anchor of component 15, but it earns a place here because it is also the cleanest demonstration that REGENERATION should be conditioned on what already failed rather than restarted from scratch. Reflexion writes a verbal self-reflection of its past failure into episodic memory and carries it into the next attempt, achieving 91% pass@1 on HumanEval from roughly an 80% baseline with no weight updates [14]. The transfer is the seed of principle P3 and the stage that compiles prune-reasons into regeneration constraints: if the next proposal is seeded with the knowledge that hypothesis H was already tried and contradicted, the next proposal is far less likely to restate H. This component's falsification ledger is that record made machine-readable — the model is handed a typed set of excluded hypotheses and their prune-reasons, which the generator is structurally prevented from re-proposing. ### 2.7 Selection over candidate sets: the measured value of choosing well and of diversity **CodeMonkeys** (Ehrlich, Brown, Juravsky, Clark, Ré, and Mirhoseini, 2025, arXiv:2501.14723) scales test-time compute for software engineering by building an ensemble of agents and reports the selection arithmetic precisely: selecting over the ensemble reaches 66.2%, the best single member (the Blackbox AI Agent) reaches 62.8%, and RANDOMLY selecting from the same pool reaches 60.9% [15]. The three numbers are the whole argument in miniature — the pool contains a 62.8% best member, but a harness that generates a pool and does not select well loses 3.4 points to the best member and ends only 1.9 above random, while selecting over it with evidence recovers nearly all of it. Random selection from a good pool is barely better than the pool's floor; selection is where the value is, and selection is only as good as the diversity it chooses from. **DEI** (Zhang, Yao, Liu, Feng, Liu, Murthy, Lan, Li, Lou, Xu, Pang, Zhou, Heinecke, Savarese, Wang, and Xiong, 2024, arXiv:2408.07060) — Diversity Empowers Intelligence — is the direct claim that constituent diversity is what makes an ensemble of software-engineering agents resolve more issues than any member alone [16]. DEI is the sixteen-author statement of this component's title: never marry your first sample, and do not even marry your best single agent — integrate a diverse set, because the diversity, not the individual strength, is the source of the ensemble's power. ### 2.8 Verifiers and oracle reranking The verifier lineage supplies the expensive-oracle half of the cheap-referee-before-oracle design, and it must be cited with a corrected identifier. **Training Verifiers to Solve Math Word Problems** (Cobbe, Kosaraju, Bavarian, Chen, Jun, Kaiser, Plappert, Tworek, Hilton, Nakano, Hesse, and Schulman, 2021, arXiv:2110.14168) is the canonical result that a separately trained verifier scoring a model's candidate solutions, used for best-of-N reranking, substantially improves over greedy decoding and majority vote [17]. The correction matters and is not carried over from the earlier source: this verifier work is arXiv:2110.14168, NOT 2107.03374, which resolves to the Codex code-training paper and must not be cited as the verifier. The design transfer is the ORACLE: a verifier (or the test suite) is an expensive but trustworthy arbiter of which candidates survive, so the candidate stage must screen cheaply first so the oracle's budget is spent on candidates the cheap referee could not eliminate. **Adaptive-Consistency** (Aggarwal, Madaan, Yang, and Mausam, EMNLP 2023, arXiv:2305.11860) — "Let's Sample Step by Step" — adds the budget-efficiency half: rather than sampling a fixed large N for every problem, it samples adaptively until the answer-consistency crosses a threshold, reporting over 17 reasoning and coding datasets and three LLMs that it reduces the sample budget by UP TO 7.9× with an average accuracy drop of less than 0.1% [18]. The 7.9× is an upper bound over datasets and models, not a constant, and this paper states it as such; the transfer is that not all problems deserve the same budget — which becomes the budget-allocation clause of stage (a) and the budget-efficiency metric of Section 6. ### 2.9 Deterministic fusion and the syntax-guided repair lineage **PatchFusion** (Yang, Hu, Ren, Chen, Le, Bissyandé, and Tian, July 2026 preprint, arXiv:2607.01597) — "A Single Patch Is Not Enough" — argues that no single repair candidate suffices and proposes deterministic fusion of repair candidates, reporting that it solves 426/500 bugs on SWE-bench Verified and 236/300 on SWE-bench Multilingual [19]. The numbers must be labeled exactly as the verification established: they are SELF-REPORTED in the abstract of a roughly two-month-old July-2026 preprint and require independent confirmation before they anchor any benchmark claim; this paper cites them as a promising directional result, not a settled benchmark. The design transfer is the fusion clause of the thesis: when several distinct hypotheses survive verification, their AGREEMENT is itself deterministic evidence that can be fused into a single patch, which is why the candidate stage must preserve cross-candidate agreement rather than throwing away every loser the moment a winner is found. **Recoder** (Zhu, Sun, Xiao, Zhang, Yuan, Xiong, and Zhang, ESEC/FSE 2021, arXiv:2106.08253) is the syntax-guided edit decoder that generates repairs as EDIT sequences over the abstract syntax tree rather than as free-form replacement text, keeping the candidate inside the space of syntactically valid programs [20]; the modern "syntax-guided provider/decider" framing is a component concept descended from this lineage, treated as such rather than as a separate canonical paper. **SketchFix** (Towards Practical Program Repair with On-Demand Candidate Generation, ICSE 2018, ACM 10.1145/3180155.3180245) is the classical on-demand candidate-generation repair tool; it is NOT on arXiv, and this paper cites it by its ACM DOI as the verification requires — no arXiv identifier exists or should be invented. **Trio** (Inductive Synthesis of Structurally Recursive Functional Programs from Non-Recursive Expressions, Journal of Functional Programming 2025, public artifact at Zenodo doi:10.5281/zenodo.15690878) is likewise not on arXiv and is cited by its JFP/Zenodo record [22]. SketchFix and Trio stand for the granularity principle: generation at the edit or structure level yields hypotheses that are at least syntactically coherent, the precondition for behavioral clustering to mean anything. ### 2.10 Classical roots: the method of multiple working hypotheses and strong inference Beneath all of the machine-learned diversity sits a century-old scientific method, and this component's posture — hold several causally distinct hypotheses at once and let evidence exclude them — is that method made into a build contract. **Chamberlin's Method of Multiple Working Hypotheses** (Thomas Chrowder Chamberlin, Science, 1890) is the original argument that the investigator who holds a single working hypothesis falls in love with it and reads all evidence through it, while one who holds SEVERAL competing hypotheses at once is led to seek the evidence that will DISCRIMINATE among them — the very posture this component enforces by construction, because a harness that generates several causally distinct hypotheses is forced to record what would distinguish them [23]. Chamberlin is not on arXiv (it predates it by a century). **Platt's Strong Inference** (John R. Platt, Science 146(3642):347–353, October 16, 1964) is the sharper statement: devise several alternative hypotheses, then devise a crucial experiment (or experiments) that will exclude one or more of them, then carry it out cleanly and let it prune the space [24]. Platt's crucial experiment is this component's oracle; his "exclude one or more of them" is its falsification pruning; his insistence that the hypotheses be devised so a clean experiment CAN discriminate among them is the demand that every hypothesis carry a behavioral prediction reality can contradict (P1). Neither assumed the hypotheses would come from a weak language model; but the method they specified — generate disagreement, then let reality collapse the search space — is the intellectual charter of the Piper rule, stated a century and a half before the harness was built to obey it. ## 3. Design Principles Seven principles follow from the review, each tied to the evidence above and each a clause of the strong Piper rule made mechanical. **P1 — Hypotheses and patches are separate objects.** The generator's output is a typed HYPOTHESIS — a claim about the correct behavior and a strategy for producing it — distinct from the PATCH that later materializes if it survives. A hypothesis carries `{ frame, claim, repair_strategy, behavioral_prediction, status }`; a patch is a later, separate artifact that instantiates the winner. The separation lets the harness falsify an IDEA before it spends effort building the patch, and lets it keep a falsified hypothesis's lesson (Chamberlin's discrimination, Platt's crucial experiment [23][24]) rather than discarding the whole line of attack. AlphaCode's filtering [1] and CodeT's test-based selection [5] implicitly separate candidate-from-winner; this principle makes it structural. **P2 — Diversify by strategy and frame, not temperature.** Diversity is manufactured by generating from DISTINCT strategies and frames — this repair approach, that causal story, the other instrumentation route — sampling within each frame only as needed, never by cranking temperature on one prompt. Verbalized Sampling showed named-then-sampled diversity beats temperature scrambling [4]; Degeneration-of-Thought showed unmanufactured agreement collapses onto a wrong mode [10]; Self-Consistency showed repetition masquerading as agreement inflates majority votes [3]. The mechanism is stage (a)'s frame decomposition: spend the budget on the NUMBER of distinct frames first, because five frames with one candidate each is worth more than one frame with five temperature draws. **P3 — A contradicted hypothesis becomes a constraint on regeneration.** When execution, tests, traces, or invariants contradict a hypothesis, the contradiction is not only a record of its death — it becomes a typed constraint fed forward to the generator, so the next proposal is structurally prevented from re-proposing the falsified idea. Reflexion proves conditioning the next draw on what failed improves it [14]; this principle makes that conditioning deterministic and exhaustive, because a weak model cannot be relied on to read its own failure prose and remember it. The falsified hypothesis and its prune-reason go onto the exclusion list seeding every future proposal call (stage h). **P4 — Cluster behaviorally before verifying.** Hypotheses are clustered by the behavior they PREDICT before any is sent to the oracle, so one executed test or trace can falsify an entire cluster predicting the same observable, rather than spending one oracle call per near-equivalent hypothesis. AlphaCode's clustering-before-submission [1] and TraceRepair's trace-guided discrimination [12] are the ancestors; the move is to define each hypothesis by its predicted behavior and treat hypotheses predicting the same observable as one cluster for verification, while recording that they may differ in repair strategy (which matters for fusion). **P5 — Run a cheap synthesized referee before the expensive oracle.** Before any hypothesis is judged by the true test suite or verifier — the expensive oracle — it is screened by a cheap deterministic referee: generated tests, type checks, structure checks, static invariants, or a fast synthesized probe exercising the hypothesis's behavioral prediction. CodeT is the ancestor that tests are the referee [5]; the cheap referee runs on every candidate at pocket change and eliminates the obviously-falsified ones, so the oracle — the verifier of Cobbe [17], the real suite — is spent only on candidates the referee could not kill. The synthesized referee is itself generated deterministically from the behavioral predictions (P1), so a hypothesis with an untestable prediction is caught as unfalsifiable before it costs anything. **P6 — Cross-candidate agreement is deterministic edit-atom evidence.** When two or more distinct hypotheses survive verification, their agreement is preserved as evidence — not because agreement proves correctness (Degeneration-of-Thought forbids that inference [10]), but because agreement over INDEPENDENT hypotheses that survived independent falsification is a deterministic signal that can later guide selection or be fused into a single patch at the edit-atom level. PatchFusion is the demonstration that single patches are not enough and fusion is the endpoint [19]; this principle ensures the candidate stage produces the raw material fusion needs — a record of which edit atoms the survivors share and which they dispute. **P7 — Record prune-reasons so eliminated hypotheses are never silently regenerated.** Every elimination records WHY — contradicted by which execution, test, trace, invariant — in a falsification ledger, and the regeneration step is seeded with that ledger, so a hypothesis eliminated by evidence cannot reappear next round simply because the model forgot it died. Reflexion's verbal conditioning [14] is the informal version; the ledger is the machine-checked version, and it is what makes the thesis's "prevent falsified ideas from being regenerated" clause hold even when the weak model, on a fresh call, would happily re-propose the idea it lost a round ago. P7 turns a candidate generator from a memoryless sampler into a search that actually shrinks (Platt's pruning [24]). ## 4. The Architecture The candidate-generation stage is a sequence of stages; every node is labeled D (deterministic software) or L (a learned model). The invariant that governs the whole is: the L column contains exactly one entry, and every upstream and downstream step is deterministic software on CPU — a single weak-LLM frame-conditioned proposal call, with frame decomposition, schema binding, diversity measurement, deduplication, behavioral clustering, cheap-referee screening, branching, oracle verification, evidence recording, falsification pruning, constraint compilation, and agreement extraction all deterministic. The concrete inputs are the decomposed request, the repository context, and the falsification ledger from prior attempts on this and similar tasks; the output is a candidate HYPOTHESIS SET — causally distinct, deduplicated, behaviorally clustered, each with a behavioral prediction and evidence record, the survivors carrying agreement evidence, the eliminated carrying prune-reasons — plus the regeneration constraints compiled from those prune-reasons. The flow, in the deterministic stages (a) through (i) plus the one learned entry (L): 1. **(a) Frame decomposition, budget allocation, and falsification intake [D].** The decomposed request is split deterministically into DISTINCT frames — distinct strategies and causal framings the candidate set should span: a contract-based repair frame, a minimal-edit frame, a reimplementation frame, a root-cause hypothesis. The budget is allocated so a minimum number of causally distinct frames is represented before any frame gets more than one candidate (P2; the budget-efficiency lesson of Adaptive-Consistency [18]). Simultaneously the falsification ledger (P7) is parsed into an exclusion list seeding the generator with what it must NOT propose. The model decides none of it. 2. **(L) Frame-conditioned hypothesis-and-strategy proposal [L] — the single learned entry.** The weak model, seeded with the decomposed request, the repository context, the distinct frames from stage (a), and the ledger's exclusion list, emits hypotheses and repair strategies FROM EACH DISTINCT FRAME — one or a few per frame — each stating its claim about the correct behavior and the strategy that follows. This is the component's single LLM step, the semantic-judgment step and nothing more: propose DIFFERENT IDEAS from DIFFERENT FRAMES, and state for each the behavioral prediction reality can check. It does not select among its own proposals, judge which frame is best, decide the budget, or evaluate whether its hypotheses are good — those judgments are deterministic and downstream. The exclusion list makes this call structurally unable to regenerate a falsified idea (P3/P7). 3. **(b) Normalization and schema binding [D].** Each proposal is bound deterministically into a typed `Hypothesis` object `{ frame, claim, repair_strategy, behavioral_prediction, status }`, kept structurally separate from any `Patch` object (P1). A proposal that fails to yield a parseable behavioral prediction — a claim about what observable would change if the hypothesis were true — is rejected here as unfalsifiable (Platt's requirement that a hypothesis admit a discriminating experiment [24]), before it costs an oracle call. 4. **(c) Diversity measurement and deduplication [D].** The candidate set is measured for SEMANTIC diversity (are the claims different ideas?) and STRUCTURAL diversity (do the repairs touch different code / use different edit shapes?), and near-duplicates — restatements of the same idea, by embedding similarity or structural edit overlap — are merged or dropped deterministically (Self-Consistency's warning that duplicates masquerade as votes [3], Large Language Monkeys' warning that unselected duplicates approach random [2]). The measurement enforces the thesis's clause that duplicates must not consume the candidate budget; it runs BEFORE verification spend, so the budget buys distinct hypotheses. 5. **(d) Behavioral clustering [D].** The deduplicated hypotheses are clustered deterministically by the behavior they PREDICT — hypotheses whose predicted observables overlap are grouped so one executed test or trace can falsify the cluster jointly (P4; AlphaCode's clustering [1], TraceRepair's trace discrimination [12]). Clustering is about verification economy and about fusion later (P6): hypotheses in one behavioral cluster share the fate of a single falsifying probe, yet remain recorded as distinct repair strategies so a surviving cluster can still feed multiple repair branches. 6. **(e) Cheap synthesized referee screening [D].** Each behavioral cluster is screened by a cheap deterministic referee before any expensive oracle call (P5): generated tests and type/structure/invariant probes, synthesized from the cluster's behavioral predictions, run at pocket change. A cluster whose predictions the cheap referee contradicts is falsified here and routed to stage (h) with its prune-reason, never reaching the oracle. This is CodeT's tests-as-referee [5] as a budget-saving first pass; the oracle (verifier [17], real suite) is reserved for the survivors. 7. **(f) Candidate branching and budget enforcement [D].** Promising hypotheses — those that survived the cheap referee — are branched into MULTIPLE repair candidates where the evidence warrants (the thesis's "branch promising hypotheses" clause; ToT/LATS/SWE-Search's search-expansion [6][7][8]): a surviving hypothesis may yield several concrete repair edits, each a candidate. The total is enforced against the budget allocated in stage (a), deterministically — no frame or branch exceeds its share, and the minimum-distinct-frame guarantee is checked again so the final set still spans causally distinct ideas. 8. **(g) Oracle verification and evidence recording [D].** Each surviving candidate runs against the expensive oracle — the real test suite, traces, invariants, or the verifier — and the outcome is recorded deterministically as EVIDENCE against its hypothesis: which tests it passed, which it failed, which traces contradicted its behavioral prediction. This is the "let reality collapse the search space" step: reality, not the model, decides which hypotheses survive (Platt's crucial experiment [24]). Every outcome is written to the evidence record so survival is traceable to the observations that supported it. 9. **(h) Falsification pruning and constraint compilation [D].** Hypotheses contradicted by the oracle are eliminated deterministically, and each elimination writes a prune-reason to the falsification ledger — contradicted by test X, trace Y, invariant Z (P3/P7). The ledger is compiled into the regeneration-constraint set seeding future proposal calls, so the eliminated hypothesis and the reason it died are carried forward and the generator is structurally prevented from re-proposing it. This is the search-shrinking step (Platt's "exclude one or more of them" [24]) and the mechanism that keeps the harness from being a memoryless sampler that rediscovers its own graveyard every round. 10. **(i) Agreement extraction and edit-atom assembly [D].** The surviving hypotheses are compared deterministically to extract cross-candidate agreement (P6): which edit atoms the survivors share, which they dispute, and where independent survival over independent falsification corroborates a change. The agreement structure is preserved for two downstream consumers — it later guides SELECTION among survivors, and it feeds the deterministic patch-fusion step (the PatchFusion endpoint [19]) that merges agreeing survivors into one patch. No hypothesis that lost is thrown away silently: its prune-reason is in the ledger (h) and its edit atoms are accounted for in the fusion record. The primitive set is deterministic except where noted: `decompose_frames` (split the request into frames and allocate the budget), `intake_falsification` (parse the ledger into an exclusion list), `propose_from_frames` (the single learned call emitting frame-conditioned hypotheses and strategies), `bind_hypothesis` (normalize into the typed Hypothesis object, reject the unfalsifiable), `measure_diversity` and `dedupe_hypotheses` (semantic and structural diversity; merge or drop near-duplicates), `cluster_by_behavior` (group by predicted observable), `synthesize_referee` and `run_cheap_referee` (generate and run the cheap deterministic probes), `branch_candidates` (expand survivors under the budget), `verify_oracle` and `record_evidence` (run the expensive oracle and log outcomes), `prune_falsified` and `compile_constraints` (eliminate contradicted hypotheses and write prune-reasons into the regeneration constraint set), and `extract_agreement` (compare survivors for shared and disputed edit atoms). The target is explicit and checkable: stages (a)–(i) are software on CPU, and the L column holds exactly one entry — the frame-conditioned proposal call that contributes the semantic judgment of what different ideas to try. Any proposed addition that would put a language model elsewhere in the pipeline — having the model choose which of its own frames is best, judge which candidate it likes, set its own budget, decide its own diversity, or evaluate whether a hypothesis survived — is a regression against P1–P7 and must clear the same bar as in earlier components: beat the deterministic stage on the honest metrics of Section 6, on held-out tasks, at a defensible cost, or be rejected. The invariant is stated again so it can be enforced mechanically: **"the L column contains exactly one entry."** ## 5. Recursive Self-Improvement Protocol Component 16 of 27 follows the program's uniform protocol: the executable specification is written to 100%, round 1 lands at approximately 80% of the specified behavior, and later rounds close the gap. Each round is A/B-gated against the honest metrics of Section 6 — above all candidate diversity and budget efficiency — never the end-to-end resolve rate alone. The three-way split disciplines every change: semantic judgment goes to the LLM's single entry (propose hypotheses and repair strategies from distinct frames), mechanics go to deterministic software, and every change to either is A/B-gated on honest metrics, never on vibes. Because candidate generation decides what ideas the harness even gets to try, its improvement protocol measures the DIVERSITY and EVIDENCE-DISCIPLINE of the candidate set rather than trusting the model's sense of whether it generated well, improving itself in four loops. First, *frame-decomposition tuning*: the harness measures whether its frames produce distinct hypotheses and tunes the frames and budget accordingly, guided by Verbalized Sampling [4], Degeneration-of-Thought [10], and Adaptive-Consistency's budget evidence [18]; a decomposition yielding five near-identical hypotheses no matter how many frames it declared lied about its diversity. Second, *referee calibration*: the harness measures whether the cheap referee agrees with the expensive oracle — whether candidates it kills would have been killed by the oracle anyway, and whether it ever kills one the oracle would have passed (a false-kill that costs a good hypothesis) — and calibrates strictness, guided by CodeT's demonstration that tests are a trustworthy referee [5]; a cheap referee that false-kills good hypotheses is worse than none, because it prunes the search on wrong evidence. Third, *falsification-ledger tightening*: the harness measures the falsified-rebirth leak rate — how often a hypothesis eliminated with a prune-reason reappears in a later proposal — and tightens the constraint compilation so the exclusion list is exhaustive and the generator is structurally unable to regenerate a falsified idea, guided by Reflexion's proof that conditioning regeneration on failure is a real and large gain [14]; a ledger that leaks is a search that walks back over its own graveyard. Fourth, *agreement and fusion yield*: the harness measures whether preserved cross-candidate agreement actually guides better selection and fusion — whether survivors that agreed produced a better fused patch than any single survivor (PatchFusion's premise [19]) — and tunes what agreement evidence is preserved and how it is weighted, closing the loop between candidate generation and the patch that lands. Because every candidate and every elimination carries an evidence record, a regression in generation is visible in the diversity and falsification metrics before it surfaces as a wrong patch, and each A/B change must justify itself on held-out tasks at a defensible cost before it ships. ## 6. Measurement & Evaluation Candidate generation is measured *separately* and *honestly*, because a stage whose whole purpose is to produce distinct, evidence-constrained hypotheses can be defeated in either of two opposite ways: by generating five copies of the same idea and calling it diversity, or by generating five different ideas and wasting them all by choosing among them without evidence. The instruments keep the candidate set accountable to actual diversity and falsification, not to the model's confidence that it did well. **Candidate diversity (semantic and structural).** The primary instrument reports, for every candidate set, both the semantic diversity of its claims (are these different IDEAS?) and the structural diversity of its repairs (do they touch different code, use different edit shapes?), measured deterministically before any verification spend. This is the Piper rule's first clause — "five answers are valuable only if they represent five different ideas" — made quantitative: a generator returning five near-identical hypotheses scores near zero no matter how plausible each is, and the frame-decomposition-tuning loop (Section 5) optimizes against it. The measurement must be deterministic and outside the model because the model cannot be trusted to report its own diversity — Self-Consistency shows duplicates silently inflate votes [3] and Verbalized Sampling shows uncoaxed sampling collapses onto modes [4]. **Budget efficiency.** The companion instrument reports how much verification and oracle budget the harness spent per distinct idea carried forward: candidates generated per distinct hypothesis that survived, cheap-referee calls per candidate eliminated, oracle calls per survivor. This measures the thesis's clauses that duplicates must not consume the candidate budget and that the cheap referee must precede the expensive oracle — guided by CodeT's cheap-tests-then-select [5] and Adaptive-Consistency's evidence that sample cost can fall by up to 7.9× at under 0.1% accuracy cost [18]. A harness that spends ten oracle calls to learn what one cheap test would have shown is paying for diversity it never bought. **Duplicate rate before deduplication.** The instrument reports what fraction of raw proposals were near-duplicates of an idea already in the set, measured at stage (c) before dedup merges them. A high raw duplicate rate is not a failure of the dedup stage — it is a signal about the model and the frames: the frames are not eliciting distinct ideas (a frame-decomposition failure per P2 and the Verbalized-Sampling/divergence results [4][10]) or the model is over-sampling its own dominant mode. The instrument separates the two: a dedup rate that stays high no matter how the frames are split points at the generator's mode-collapse, not the frame list. **Falsification precision.** The instrument audits whether eliminations are correct: for hypotheses the cheap referee killed and for those the oracle killed, was the kill justified — did the evidence actually contradict the hypothesis's behavioral prediction? This guards against over-aggressive pruning that eliminates a good hypothesis on a weak or misattributed signal (the false-kill the referee-calibration loop must minimize), and keeps the falsification ledger honest so the constraints compiled in stage (h) exclude only hypotheses reality genuinely contradicted (Platt's clean crucial experiment [24]). **Falsified-rebirth leak rate.** The instrument measures the thesis's most important negative guarantee: of the hypotheses eliminated with a prune-reason, what fraction are nonetheless proposed again in a later round — a regeneration the constraint set (P3/P7) should have made impossible? A nonzero leak rate means the exclusion list is incomplete or the generator is bypassing it, and it is the clearest sign that the candidate stage has quietly become the memoryless sampler the thesis forbids. Reflexion's demonstration that conditioning regeneration on failure is where large gains live [14] is the warrant that getting this to zero is worth real engineering. **Agreement-evidence yield.** The final instrument measures what the preserved cross-candidate agreement (P6) actually buys downstream: when two survivors agreed on an edit atom, did fusing on that agreement beat picking a single survivor — and when they disagreed, did preserving the dispute lead to a better decision than suppressing it? PatchFusion's premise that single patches are not enough and fusion is the endpoint [19] is the warrant; the yield instrument verifies the harness is not preserving agreement records nobody reads, closing the loop between candidate generation and the fused patch that lands. A failure-mode audit closes the positive instruments by naming how a candidate generator lies. *Temperature-as-diversity* is raising the temperature and calling the scrambles distinct ideas — the failure the whole thesis forbids, caught by near-zero semantic diversity; *mode-collapse* is returning five restatements of the dominant (usually wrong) mode, caught by a persistently high duplicate rate [3][10]; *duplicate-budget-burn* is spending the budget verifying copies before any distinct idea is generated, caught by collapsing budget efficiency; *falsified-rebirth* is an eliminated hypothesis reappearing because the ledger leaked, caught by the leak-rate instrument; *oracle-cascade* is every candidate going straight to the expensive oracle because the cheap referee was never run or was too strict, caught by budget efficiency and falsification precision together; *selection-by-vibes* is choosing survivors by the model's self-assessment rather than by referee and oracle evidence, the failure that CodeMonkeys's 66.2-versus-60.9 gap shows costs nearly six points [15]; and *agreement-as-truth* is mistaking cross-candidate agreement for proof of correctness — the Degeneration-of-Thought error [10] — caught by remembering that agreement is evidence for fusion, never a verdict, until reality has confirmed it. Two honest caveats close the section. First, the strong-model numbers of Section 2 — CodeT's 65.8% [5], CodeMonkeys's 66.2% [15], SWE-Search's 23% relative [8], PatchFusion's self-reported 426/500 and 236/300 [19] — will NOT reproduce on a weak model, and PatchFusion's in particular are self-reported from a fresh preprint and must not be treated as an anchor; a weak-model harness should expect materially smaller absolute effects and lean all the harder on cheap referees, deterministic dedup, and falsification pruning to make its small budget count. Second, all candidate-generation metrics must be reported on held-out and out-of-benchmark tasks, as in the earlier components, so a generator that has memorized a task set's answers cannot score well on diversity without its hypotheses having been measured on problems it never saw. ## 7. Conclusion Candidate generation is the stage where the coding harness decides what it will test, and where the program's most expensive mistake is decided in the cheapest-looking way — by marrying the first sample, or by sampling a hundred times and calling the duplicates diversity. This paper has argued that the mistake is structural, not incidental: a single-shot generator is locked to the weak model's most probable and most confidently-wrong path, and an unguided repeated sampler burns its budget on restatements of the same few modes before the genuinely different idea is ever drawn. The harness must therefore never treat its model as a source of interchangeable stochastic draws; candidate generation must be a deterministic pipeline that manufactures distinct hypotheses, screens them cheaply, lets reality prune them, remembers why each died, branches the survivors, and hoards their agreement as evidence. The thesis that has governed every design choice is the one with which this paper began and closes: **Candidate generation should be hypothesis-diverse, evidence-constrained, and search-aware rather than repeated stochastic sampling: explicitly construct multiple causally distinct explanations and repair strategies; record the behavioral prediction that follows from each; eliminate hypotheses contradicted by execution, tests, traces, or invariants; prevent falsified ideas from being regenerated; branch promising hypotheses into multiple repair candidates where warranted; measure semantic and structural diversity so duplicates do not consume the candidate budget; and preserve cross-candidate agreement as evidence that can later guide selection or deterministic patch fusion.** And the rule that compresses that thesis into one sentence of discipline for Piper is: **Five answers are valuable only if they represent five different ideas. Generate disagreement deliberately, then let reality collapse the search space.** The model generates the disagreement — its single learned contribution is to propose, from each distinct frame, hypotheses and repair strategies that are actually different, each carrying a behavioral prediction reality can check. Reality does the collapsing — the cheap synthesized referee screens, the oracle verifies, deterministic pruning eliminates what execution, tests, traces, and invariants contradict, the falsification ledger records each death so nothing is silently reborn, branching expands the survivors, and agreement extraction assembles the evidence that will later fuse the winners into one patch. The architecture is deterministic end to end with exactly one learned entry, and every clause of the thesis is a deterministic stage: diversity is measured and deduplicated so the budget buys ideas; contradiction is recorded and compiled into constraints so the search actually shrinks; and agreement is preserved so the harness that generated five different ideas and kept the ones reality favored does not throw away what the survivors had to say to one another. The measured message of the literature is unambiguous — CodeMonkeys shows that selecting over a candidate pool is worth nearly six points over random [15], DEI shows the diversity is what powers the ensemble [16], the Degeneration-of-Thought result shows that unmanufactured agreement is mode collapse and not correctness [10], and Chamberlin and Platt wrote the whole method down a century before any of it ran on silicon [23][24]. For a weak model, a single sample is a lottery ticket and five copies of it are five lottery tickets with the same number; what a weak-model harness needs is five DIFFERENT ideas, checked against reality, with the losers' deaths remembered and the survivors' agreement preserved — because never marrying your first sample is only the first half of the rule, and the second half is marrying the idea that reality, not the model, allowed to survive. --- ## References 1. Li, Choi, Chung, Kushman, Schrittwieser, Leblond, Eccles, Keeling, Gimeno, Dal Lago, Hubert, Choy, de Masson d'Autume, Babuschkin, Chen, Huang, Welbl, Gowal, Cherepanov, Molloy, Mankowitz, Sutherland Robson, Kohli, de Freitas, Kavukcuoglu, and Vinyals. *Competition-Level Code Generation with AlphaCode.* Science 378(6624):1092–1097, 2022. arXiv:2203.07814. *(Massive repeated sampling + model-based filtering + clustering before submission. The scale this component admires and rejects.)* 2. Brown, Juravsky, Ehrlich, Clark, Le, Ré, and Mirhoseini. *Large Language Monkeys: Scaling Inference Compute with Repeated Sampling.* 2024. arXiv:2407.21787. *(Repeated sampling scales inference compute; unselected sampling approaches random as modes exhaust — more samples is the wrong axis, diversity is the right one.)* 3. Wang, Wei, Schuurmans, Le, Chi, Narang, Chowdhery, and Zhou. *Self-Consistency Improves Chain of Thought Reasoning in Language Models.* ICLR 2023. arXiv:2203.11171. *(Majority vote over sampled paths improves reasoning but counts identical answers as votes — dedup must precede aggregation.)* 4. Zhang, Yu, Chong, Sicilia, Tomz, Manning, and Shi. *Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity.* 2025. arXiv:2510.01171. *(Mode collapse is escaped by naming distinct approaches then sampling, not by temperature — diversity is elicited, not assumed.)* 5. Chen, Zhang, Nguyen, Zan, Lin, Lou, and Chen. *CodeT: Code Generation with Generated Tests.* ICLR 2023. arXiv:2207.10397. *(Tests as deterministic referee; 65.8% pass@1 HumanEval, absolute +18.8% over code-davinci-002, >20% over prior SOTA. CORRECTED: ID is 2207.10397, NOT 2202.11305 (a topology paper); venue ICLR 2023 not 2022; no "~1.8×" phrasing exists.)* 6. Yao, Yu, Zhao, Shafran, Griffiths, Cao, and Narasimhan. *Tree of Thoughts: Deliberate Problem Solving with Large Language Models.* NeurIPS 2023. arXiv:2305.10601. *(Explicit branching over candidate continuations with backtracking; branch shape adopted, model self-evaluation rejected.)* 7. Zhou, Yan, Shlapentokh-Rothman, Wang, and Wang. *Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models.* ICML 2024. arXiv:2310.04406. *(LATS; MCTS with branch value grounded in ENVIRONMENT feedback, not the model's estimate.)* 8. Antoniades, Örwall, Zhang, Xie, Goyal, and Wang. *SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement.* ICLR 2025. arXiv:2410.20285. *(MCTS + iterative refinement for software; 23% relative improvement across five model backbones.)* 9. Du, Li, Torralba, Tenenbaum, and Mordatch. *Improving Factuality and Reasoning in Language Models through Multiagent Debate.* ICML 2024. arXiv:2305.14325. *(Multi-agent debate IMPROVES reasoning. Du et al. do NOT report that harmonization degrades accuracy — that finding is [10].)* 10. Liang, He, Jiao, Wang, Wang, Wang, Yang, Shi, and Tu. *Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate.* 2023. arXiv:2305.19118. *(Source of the Degeneration-of-Thought result: naive consensus/harmonization can DEGRADE accuracy by collapsing onto a wrong shared mode; a divergence penalty that forces different answers beats both consensus and single-agent reasoning. MISATTRIBUTION CORRECTED — earlier source credited this to [9].)* 11. Huang, Lu, Wan, and Duan. *Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency.* ACL 2024 (2024.acl-long.78). arXiv:2309.17272. *(+15.91 HumanEval / +6.43 MBPP / +9.37 CodeContests. Uses the ACL-published 4-author list; arXiv v3 carries a 5th author (Weizhu Chen) — kept consistent here.)* 12. Wu, Wu, Zhang, Dong, and Shen. *From Guessing to Seeing: Enhancing LLM-Based Program Repair via Trace-Guided Multi-strategy Debate.* 2026. arXiv:2604.02647. *(Trace-guided multi-strategy repair debate; correctly fixes 392 defects on Defects4J. "ArcXiv" was a source typo — the ID resolves.)* 13. Li, Shi, Lin, Gu, Lian, Wang, Jia, Huang, and Wang. *SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution.* 2025. arXiv:2507.23348. *(Competitive multi-agent debate carried to software issue resolution.)* 14. Shinn, Cassano, Berman, Gopinath, Narasimhan, and Yao. *Reflexion: Language Agents with Verbal Reinforcement Learning.* NeurIPS 2023. arXiv:2303.11366. *(Verbal reflection stored and used to condition the next attempt; 91% pass@1 HumanEval from ~80% baseline. Anchor that regeneration conditioned on failure beats regeneration from scratch.)* 15. Ehrlich, Brown, Juravsky, Clark, Ré, and Mirhoseini. *CodeMonkeys: Scaling Test-Time Compute for Software Engineering.* 2025. arXiv:2501.14723. *(66.2% ensemble-selected vs 62.8% best member vs 60.9% random selection — the value lives in SELECTION over a pool, nearly six points over random.)* 16. Zhang, Yao, Liu, Feng, Liu, Murthy, Lan, Li, Lou, Xu, Pang, Zhou, Heinecke, Savarese, Wang, and Xiong. *Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents.* 2024. arXiv:2408.07060. *(DEI; diversity among constituent agents is what powers the ensemble's resolution over any single member.)* 17. Cobbe, Kosaraju, Bavarian, Chen, Jun, Kaiser, Plappert, Tworek, Hilton, Nakano, Hesse, and Schulman. *Training Verifiers to Solve Math Word Problems.* 2021. arXiv:2110.14168. *(Separately trained verifier scoring candidates for best-of-N reranking. CORRECTED ID: 2110.14168, NOT 2107.03374, which is the Codex code-training paper.)* 18. Aggarwal, Madaan, Yang, and Mausam. *Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs.* EMNLP 2023. arXiv:2305.11860. *(Adaptive sample budget to difficulty; reduces sample budget by UP TO 7.9× — an upper bound, not a constant — at <0.1% average accuracy drop.)* 19. Yang, Hu, Ren, Chen, Le, Bissyandé, and Tian. *A Single Patch Is Not Enough: Deterministic Fusion of Repair Candidates.* July 2026 preprint. arXiv:2607.01597. *(Deterministic fusion of candidates; reports 426/500 SWE-bench Verified and 236/300 Multilingual. SELF-REPORTED in the abstract of a fresh preprint — requires independent confirmation; not a benchmark anchor.)* 20. Zhu, Sun, Xiao, Zhang, Yuan, Xiong, and Zhang. *A Syntax-Guided Edit Decoder for Neural Program Repair.* ESEC/FSE 2021. arXiv:2106.08253. *(The "Recoder" tool/repo; repairs as AST edit sequences. The modern "syntax-guided provider/decider" is a component concept descended from this lineage, not a separate canonical paper.)* 21. Towards Practical Program Repair with On-Demand Candidate Generation.* ICSE 2018. ACM 10.1145/3180155.3180245. *(The "SketchFix" tool. NOT on arXiv — cited by ACM DOI, no arXiv ID invented.)* 22. *Trio: Inductive Synthesis of Structurally Recursive Functional Programs from Non-Recursive Expressions.* Journal of Functional Programming, 2025. Public artifact: Zenodo doi:10.5281/zenodo.15690878. *(NOT on arXiv — cited by JFP/Zenodo record.)* 23. Chamberlin. *The Method of Multiple Working Hypotheses.* Science, 1890. *(Classical; holding several competing hypotheses at once leads the investigator to seek discriminating evidence rather than falling in love with one. NOT on arXiv — a 1890 text.)* 24. Platt. *Strong Inference: Certain systematic methods of scientific thinking may produce much more rapid progress than others.* Science 146(3642):347–353, October 16, 1964. DOI 10.1126/science.146.3642.347. *(The method: devise several hypotheses, devise a crucial experiment to exclude some, run it cleanly, let it prune the space.)*