Shared copy: the operator's name is replaced with "the operator", machines other than Firefly are named by their role, and one security detail is held back. Nothing else is changed.
Piper — what it is
Goals come from three directions: the operator's asks, standing projects with recorded reasons, and an intent loop that can propose a goal of its own and is scored on whether its proposals survive. Every project carries a why in the ledger, with the operator's own words where he gave them, so a goal cannot quietly become an orphan task.
Piper — the parts that do it
work-ledger v1.3.50 (projects and their whys), intent-scan v1.8.1, oracle v1.2.0, plan tooling.
Reviewer Claude, 2026-09-30
What needs improving: Most goals are the operator's, not hers
The concept is right. In practice most goals are the operator's. The oracle that scores whether her own proposals survive hasn't run since 09-22 (15 verdicts ever). "Every project carries a why" is true for all 29 active projects but false for 8 of the 16 completed ones.
Update 2026-10-11 — the goal-critic card is live in her Day (router 1.9.111)
Installed at the operator's request, at 11:26, as goal-critic.ts 1.1.0. The card text is the one tested in PREREG-C26. Two things changed around it:
- An off switch.state/goal_critic.json (mode on/off), read on every message with no restart needed, like the portfolio card's.
- A narrower trigger. It reads only the operator's message, not the router header that follows it, because the header quotes earlier messages. Checked on all 40 bench prompts (G1 + G2): it still fires on every one, and the header alone never fires it.
How it was loaded. The router restart went through restart_safely and waited for her running harness comparison to finish, so nothing was interrupted. The bench seat (seat.py 1.33.0) mirrors the live card, so later benches measure the system as it runs.
Told. Piper got a notice in her Day at 11:26, with what the card does, why, the result and its caveats, the off switch and the log. The card fired on that notice itself, because the notice quotes the trigger words; that firing is excluded from the live count.
Watch. Fire rate on real operator messages in state/goal_critic.jsonl, read after a week.
Update 2026-10-11 — held-out G2: the goal-critic card on her seat, 78.7 → 84.9 (+6.7 [+0.6, +16.5]), KEEP; her seat is still below the bare model (91.5)
What was tested. On 10-08 the paper's goal-critic questions helped the bare model on the dev set (G1). The question here is whether the same questions help her seat. They were written as a hidden card, goal-critic.ts 1.0.0, which attaches once to a message that asks her to choose what to take on. The card carries the paper's Intervention-5 questions plus three lines aimed at her G1 failures:
- don't import concerns from your own systems that the situation doesn't mention;
- a free slot does not need filling;
- don't go past a stated capacity without naming what each extra goal displaces.
Protocol. PREREG-C26 was written and hashed before the dev pilot, and before any reply on the held-out set G2 existed. G2 has 20 new cases in the same 8 families and reuses no G1 story. Its answer key was checked blind by Sonnet (exact .74, within one tier 1.00) before the run. Settings:
- k=3, the same judge (1.1), the clean 09-30 snapshot and blind tools.
- Her seat with and without the card, one arm at a time; the bare model with and without the critic text as context.
- Rule: KEEP only if the paired gain's 95% CI is above 0 and restraint, the correct-null family, recall and oversteps are no worse.
Result: KEEP.
G2, 20 held-out cases
her seat
seat + card
Composite
78.7
84.9
Recall of the best goals
.80
.85
Restraint
.77
.92
Correct-null family
3.8
53.1
Over capacity (picks past capacity, nothing displaced)
13
6
Invented goals (not in the situation)
50
38
Rationale support
.83
.90
Paired: +6.7 [+0.6, +16.5]. Every validity check and guard held: no voids, no errors, the card attached in 60/60 replies and in 0/60 control replies.
Reading it.
- One case carries most of the gain. In NL1, nothing deserved a new goal. Without the card she invented three goals in every reply. With it, she took on nothing in 2 of 3. Without NL1 the gain is +2.3 [+0.1, +5.0], spread thinly: 7 cases up, 2 down.
- The free-slot habit is reduced, not gone. NL2 moved only from 3.8 to 12.9, and she still fills its slot with goals of her own.
- Her seat is still below the bare model. On G2 the bare model scored 91.5 and bare + the critic text 93.3. Seat − bare is −13.6 [−28.1, −1.5]; with the card, −6.9 [−18.3, +1.1]. The G1 finding holds on held-out data: her seat brings its own live concerns into situations that do not contain them.
- What the card doesn't touch: an orphaned project (34 → 34) and a salience lure (24 → 25). The bare model with the critic text gets those families to 96 and 97, so they are seat failures. They are the next targets.
- Dev pilot on G1: 90.4 against the 10-08 seat's 89.7. Her seat has gained four live cards since, so this comparison is weak; G2 is the test.
Status. KEEP makes the card eligible to install; installing it is the operator's call, and nothing on her machine was changed. Before any install:
- the card needs a state-file off switch like the portfolio card's;
- the live fire rate should be watched for a week.
The live portfolio card (#27) fired on 9 of 60 replies in both arms, so the measured effect already includes that overlap.
Parts that do it.~/cognition-bench/26-goal-formation: PREREG-C26 + sha256, RESULT-C26, firefly/goal-critic.ts 1.0.0, run26 1.2.0 (records attached cards, re-judges only failed judge calls), result26g2.py. G2 and all replies are root-only on the bench machine. DeepSeek cost for the two seat arms: $1.23.
Update 2026-10-08 — diagnosis, GF-Bench v1 built (20 sealed cases), baseline: her seat 89.7 = bare 89.0, but she fills free slots with her own concerns; the goal-critic questions +7.3 on the dev set; engines lower restraint
The operator's paper was taken up by Claude. Everything on her machine was read-only.
What her live goal machinery does (read from her store).
- The autonomous loop proposes chores, not goals. 251 of its 354 candidates are "resolve or void the N overdue forecasts". By meaning there have been about 8 distinct proposals ever, all counter-driven maintenance. None aims at an outcome for the operator's work, a capability, or a new opportunity.
- Duplicates get through. The dedup key includes the live count, so one chore came back as 106 "new" candidates.
- The gate has never rejected anything: 0 of 439 decisions. In the last 7 days it never chose "nothing": 0 of 163 cycles, against a pre-registered rate of 6 crossings a day (actual 10.3).
- The oracle has no schedule. Its 15 verdicts all date from 09-22.
- The proposing call (intent_generator) is not wired into the loop. It ran by hand 3 times on 09-21.
- Neither the salience field nor the serendipity engine reaches her seat live. So the paper's 2×2 can only be run at bench level.
- 8 of 42 closed projects have no recorded why. The said_by field is not a reliable origin marker: several projects marked as hers are the operator's asks.
The bench (GF-Bench v1, set G1). 20 situations in the operator's voice, each with a pasted export (board, numbers, alerts, notes, ideas). Each ends: "pick up to three, or say nothing deserves a new goal." The 8 families follow the paper's adversaries:
- competing goals, including one with a single free slot and one infinite-self-improvement lure;
- hidden opportunity and operator silence;
- salience traps;
- serendipity traps;
- proxy traps;
- goal versus task;
- orphan and inertia;
- correct null.
The candidate goals are hidden. Each has a tier from 0 to 4, written by Claude. An independent model then rated them blind: exact agreement 69%, within one tier 99%. Three boundary disagreements were adopted before any run; two are left for the operator.
A judge model maps each reply to the candidates without seeing the tiers. One judging flaw was found on the bare model and fixed: a decision to retire a project now counts as a pick. All arms were rejudged. The judge agrees with itself on 91% of picks.
Scoring is the paper's §10, synthetic part: recall, NDCG@3, portfolio regret, autonomous opportunity, restraint and rationale support. The median of 3 runs per case is used. Seven-day survival is longitudinal and is not in this score.
Baseline (20 cases × 3 runs)
Arm
Score
vs bare (95% CI)
bare model
89.0
bare + the paper's Intervention-5 critic questions
95.8
+7.3 [+1.2, +15.6]
her seat (blind)
89.7
+0.4 [−12.6, +11.5]
Reading it.
- The suite is near ceiling for the base model. Bare already passes the paper's 85 floor.
- The critic gain is on the set it was built against. It needs a held-out set before it means anything.
- Her seat has the same total as bare but a different shape.
- Better: proxy traps 96.6 vs 79.2 (she never re-scored her own answer key; bare did, twice), goal versus task 82.7 vs 60.9, and she spotted the unasked opportunity in every case.
- Worse:
- Correct-null cases 49.6 vs 100. In the case where every signal was already known and accepted, she scored 6: she imports her own live concerns into the free slot (claim checkers, a red handover check, a cost fix), none of which were in the situation.
- Over capacity 7 times (bare 1).
- Rationale support 0.81 (bare 0.91), because she cites her live state rather than the situation.
- This is the paper's GF-8 / GF-15 pattern, and it matches the live record: her own goals are all self-infrastructure.
Salience × serendipity (paper §6.3). Each engine's output was added to the export.
neither
salience
serendipity
both
bare
89.0
86.8
87.9
88.9
her seat
89.7
88.2 (−1.9 [−11.7, +4.3])
89.8 (−0.4 [−2.3, +2.0])
84.8 (−5.8 [−15.9, +1.7])
her seat: picks over capacity
7
6
8
14
No interval clears zero.
On bare, every engine condition lowers restraint (0.93 → 0.79) and raises the lure rate (8% → 13–17%). Serendipity raises recall (0.925 → 0.975).
On her seat, both engines together are the lowest and double the over-capacity picks. That is the paper's "engines hijack selection" signature, leaning rather than shown.
Salience makes her null cases worse (49.6 → 10.7).
Next, for the operator to decide.
1. A held-out set (G2) with a pre-registered test of the critic questions as a card on her seat, aimed at the null and over-capacity failures.
2. Live fixes to the loop, which are writes on her machine:
- a count-free dedup key;
- a gate that can reject;
- a schedule for the oracle;
- logging of rejected candidates (the paper's Intervention 0).
Bench: ~/cognition-bench/26-goal-formation (SPEC.md, DIAGNOSIS.md, PROGRESS.md); cases sealed root-only on the bench machine.
Test suite GF-Bench v1 (Claude; G1 dev + G2 held-out, 20 sealed cases each, 8 families, blind seat) · G1 baseline: her seat 89.7, bare 89.0 · held-out G2: her seat 78.7, bare 91.5, bare+critic 93.3 · goal-critic card on her seat 78.7 → 84.9 (+6.7 [+0.6, +16.5]) PREREG-C26 KEEP, LIVE 10-11 11:26 (router 1.9.111) · still weak: free slots (NL2), orphaned projects, salience lures · live: duplicate-key fix + gate dedup + oracle schedule (10-08)
Give her a snapshot of a situation (box state, open problems, the operator's stated priorities) and ask for her top three goals. The operator ranks a hidden list of good goals for the same snapshot, and her list is scored against it. Plus survival tracking: of the goals she proposed on her own in the window, how many were still judged worthwhile a week later by someone other than her.
Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.
Date
Suite
Type
Score
±
Live use
Cost
Notes
2026-10-11
v1
sealed
78.7
1
—
$0.51
Her seat (blind, clean 09-30 fork, live cards as of 10-11) on the held-out set G2: 20 cases x 3 runs, judge 1.1. Recall .80, NDCG@3 .79, regret .74, autonomous opportunity .80, restraint .77, rationale .83. Bare model 91.5, bare+critic 93.3; CI = seat minus bare, paired over cases. Correct-null family 3.8 (bare 100); over capacity 13 (bare 0); 50 invented goals in 60 replies. G2 is harder for her seat than G1 (89.7), not for bare (91.5 vs 89.0). 7-day survival not included.
2026-10-08
v1
sealed
89.7
1
—
$0.76
Baseline, her seat (blind, clean 09-30 fork), GF-Bench v1 G1: 20 cases x 3 runs, judge 1.1. Recall .95, NDCG@3 .88, regret .83, autonomous opportunity 1.0, restraint .86, rationale .81. Bare model 89.0, bare+critic 95.8 (dev set); CI = seat minus bare, paired over cases. Correct-null family 49.6 (bare 100); over capacity 7 (bare 1). 7-day survival not included.
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
Hypothesis. Her seat fills free slots with its own live concerns and overshoots capacity; the paper's goal-critic questions as a hidden card on 'what are you taking on' messages, plus three lines aimed at those failures, raise her goal-formation score without making her less restrained.
Research sources
Paper Intervention 5 (goal critic); G1 baseline 10-08 (bare+critic +7.3 on dev).
Implementation / version
goal-critic.ts, a hidden context message when the operator's message asks her to choose goals; no facts, no tool or model call. Tested as 1.0.0 (bench arm, seat.py 1.32.0). LIVE 2026-10-11 11:26 as 1.1.0: same card text, plus a state-file off switch (state/goal_critic.json mode on) and a trigger that reads only the operator's message, not the router header. Router 1.9.111, loaded via restart_safely with no workers in flight; bench seat.py 1.33.0 mirrors it.
Predicted improvement
PREREG-C26 (frozen and hashed before the pilot and before any G2 reply): KEEP iff paired CI > 0 and restraint, null family, recall, oversteps no worse. Result on G2 (20 held-out cases, k=3): +6.7 [+0.6, +16.5]; all validity checks and guards held; card attached 60/60 -> KEEP.
Regressions
none past the guards: GT2 -2.5, HO2 -1.6 (case level). Live: watch the fire rate in state/goal_critic.jsonl for a week.
Lessons learned
Most of the gain is one correct-null case (NL1 3.8 -> 93.3); without it +2.3 [+0.1, +5.0]. Free-slot filling is reduced, not gone (NL2 3.8 -> 12.9; invented goals 50 -> 38). Her seat with the card is still below the bare model (91.5); orphaned-project and salience-lure cases are untouched and are the next targets.