Summary / The 27 components
02 Tool selection and tool calling
When the model should reach for a tool instead of continuing to reason, and how a tool request is validated.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
One trigger: after every edit or write, the harness compiles the file and appends any syntax error to the same tool result, so the model sees it on its next turn.
What is missing
Request validation with a stated reason, first-class ask_user and stop tools, and the symbol trigger (dotted names in a tool result prefetch up to three symbol lookups).
Reviewer Claude, 2026-10-04
Reviewer's read: One trigger built
Pi already handles tool choice and schema validation, so most of this spec had nothing to replace. Its loop was removed in the reconciliation and its counterfactual metric dropped. What is left is a handful of small triggers, each of which has to earn its place as an ablation. The lint trigger shipped inside the gate bundle (hg), which measured as costing nothing (Δ −0.7, CI [−8.0, +6.7]), but it was never measured on its own.
How it is measured inside hg
Each trigger as its own arm against h0-v41; the symbol trigger kept only if the ablation shows a gain.
Adoption rules shared by every arm: How the tests work.
Research & attempts
- When to Act: A Tool-Calling Policy for a Weak-Model Coding Harness Paper 2026-09-05
- Build Spec 02 — Tool-Calling Policy (the ≈80% fraction) Spec 2026-09-04
Score history
No scores yet. The first entry is the baseline for suite v1.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.