Piper · Cognition
Log in

Summary / Component 13 of 38

13 Planning

Constructing sequences of actions toward a goal.

Latest99.6v1
Baseline99.6Oct 6
Live use—last 7 days

Shared copy: the operator's name is replaced with "the operator", machines other than Firefly are named by their role, and one security detail is held back. Nothing else is changed.

Piper — what it is

Plans live in the work ledger with dependencies and blocked reasons, and long work is launched as a detached unit with a landing marker rather than held in a session. Honest state: planning is on my list of unmeasured capabilities — I have the machinery but no instrument that scores whether my plans are good, which is exactly the blank I noted in the capability map.

Piper — the parts that do it

work-ledger v1.3.50, launch-detached v1.1.0, systemd user units and crons, idle-wake v1.23.1.

Reviewer Claude, 2026-09-30

What needs improving: Plans at ceiling; finished-work exit cleared her ledger (blocked rows 136 → 12)

She treats storing plans (the ledger, dependencies, detached units) as planning. Planning proper is whether the sequence of actions is good: valid order, realistic time, not wasteful. She is honest that nothing scores it. The weekly work-management scorecard measures throughput, not plan quality. There are 28 active projects and 75 blocked deliverables with reasons, which is a lot of blocked work. That is a hint that plans stall more than they fail.

Update 2026-10-09 — ledger exit worked: blocked rows 136 → 12, every retirement made by Piper with a reason

The fix (10-06). Her plan tool gave a finished or dead row no way to close, so such rows sat as blocked: 136 of them in active projects, about 69% of them finished or dead. retire added an exit: the row closes as superseded with a recorded class. It was not mass-migrated; she decides each row herself.

Result (read-only count of her live ledger, 10-09 16:40). Blocked rows in active projects: 136 → 12. Reconstructed from the transition log, 134 rows were blocked at the baseline. Where they went:

outcome rows
retired with a class (satisfied, premise false, duplicate, not viable, …) 86
superseded by a replacement row 21
actually finished and verified 17
back to todo 1
still blocked 9

3 rows were newly blocked since, so 12 are blocked now. In all she made 138 retirements: 90 from blocked rows, 28 from failed, 19 from todo, 1 from doing. Every one was hers, row by row. By class: satisfied 68, premise false 33, duplicate 14, not viable 13, obsolete 7, operator reverted 2, moot 1. Most were done on 10-06; there were 7 more from blocked rows on 10-08.

The summary score doesn't move. The planning bench is at its ceiling (99.6 her seat, 99.8 bare). This fix was to her live bookkeeping, not to how well she plans, so it is recorded on the intervention row: decision keep, 136 → 12.

Update 2026-10-06 — benchmark built; she plans well-specified problems at the optimum, at every size we could generate; the real gap was a ledger with no exit for finished work — that exit is now live

Built from the operator's paper ("full planning doc"): PLANNING-v1. Every answer is executed in a simulator against an exact optimum (A* with an admissible heuristic, Dijkstra, and an exact makespan search, each checked against brute force). No judge model. Scored with the paper's §21 weights, frozen in SPEC.md before the first run. Probes are blind: each problem is worded as an operator request in a fork of her conversation ("finish with one line PLAN: … so I can paste it into the runner").

Track Items per set What
A dependency 24 steps with preconditions, effects (including steps that undo things) and costs; each structure shown twice, as abstract lists and as maintenance prose
B path 20 one-way weighted networks with cheap-first traps and dead ends
D scheduling 15 jobs, durations, dependencies, 2–3 workers, exclusive resources, some deadlines
E replanning 18 partway through an optimal plan: a step disappears, gets expensive, is already done, gains a precondition, the goal grows, or an irrelevant change
G infeasible 20 12 impossible (missing producer, dead end, unreachable, deadline below the minimum) + 8 feasible lookalikes (deadline exactly at the minimum)

Reference points on the same items: the optimal planner scores 100, a plausible greedy planner 86.9, and answering "impossible" to everything 2.7.

Step 0: her live planning (read-only).

Results.

Set Piper (her seat) Bare model
dev D1 (97 items) 99.9 99.9
held-out H1 (97 items, sealed control) 99.6 99.8
ladder level 2 (12–16-step plans, 50–80-node networks, 10–11 jobs) — 100
ladder level 3 (100–150-node networks; 12–13 jobs) — 100

Every answer on H1 executed (validity 100). Prose versus lists made no difference (99.7 / 99.7), so building the model from a description is not a weakness. She never called a feasible problem impossible. The losses are tiny: a few plans 2–6% above the optimum, and one empty reply.

The paper's central question: is she better as the heuristic inside a search than as the planner? Tested on the bare model. Asked for distance-to-goal at sampled states, its estimates equal the true remaining cost (rank correlation 1.0). Asked to write the A heuristic, it cut search from thousands of states (Dijkstra: 3,000–73,000 at level 2) to 29–150 and still found the optimal route in 5 of 6. Both roles work. At sizes like these the model does the search itself, so a solver has nothing to add. A solver tool (solver: exact search for steps, routes and schedules) was built and is ready, but it is not installed*: there is no headroom to show.

The fix that went live: an exit for finished work. The root cause of the blocked graveyard was the interface, not her judgment. Her plan tool offered only todo, doing, verified, failed and blocked. The ledger's terminal superseded refused a replacement in another project, and the seat could not reach it at all. So blocked was the only place to put a row that was done, moot or a duplicate, and she said as much in the reasons ("parked to stop the waker re-offering it").

Live baseline (06 Oct, 00:45): 136 blocked rows in active projects, 64 of them opening with an explicit finished/dead marker. The live-use reading for this component is the share of blocked rows that are real blockers, re-read after 7 days.

Parts that do it

Test suite built · PLANNING-v1, 5 simulator-scored tracks + ladder · held-out H1 Piper 99.6 (bare 99.8; dev 99.9 both) · ladder L2/L3 bare 100 · heuristic test: model estimates exact h*, its A* heuristic cuts search 20–500x · solver tool built, not installed (no headroom) · LIVE: ledger retire path (work_ledger 1.3.44) KEPT — blocked rows 136 → 12 (10-09; 86 retired with a reason, 21 replaced, 17 verified, all by Piper)

Planning problems with checkable plans: dependency and resource puzzles run in a small simulator, so a plan either executes or it doesn't, with a known optimum. Plus real-work forecasting: for each plan she makes this week, the predicted steps and duration against what actually happened. Scoring: validity rate, closeness to the optimum, and error on duration and step count. The live-use part is the share of blocked deliverables unblocked within their stated time.

Scoring rules shared by all 38 suites: How the tests work.

Research & attempts

Score history

Suite v1 current
0255075100Oct 699.6

Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.

DateSuiteTypeScore±Live useCostNotes
2026-10-06v1anchor 100— —— Ladder level 2 (36 items: 12-16-step plans, 50-80-node networks, 10-11 jobs, replanning), bare model. Level 3 (100-150-node networks, 12-13 jobs): also 100. No collapse at any size generated.
2026-10-06v1anchor 99.9— —$1.52 Dev D1 (97 items), her seat, blind. Bare 99.9. Prose vs lists: no difference.
2026-10-06v1sealed 99.6— —$0.81 HELD-OUT H1 (97 items: dependency formal/prose, paths, scheduling, replanning, infeasible), her seat, blind, k=1. Validity 100, A 99.7, B 100, D 99.6, E 99.8, G 95.0 (one empty reply); false-infeasible 0. Bare deepseek-flash on H1: 99.8. Optimal planner 100, greedy planner 86.9.

Experimental record

One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.

2026-10-06 · v1 ✓ kept 136 → 12 (-124) ± 0 · $0.00

Hypothesis.

Research sources
Implementation / version
Predicted improvement
Regressions
Lessons learned
2026-10-06 · v1 ↺ reverted 99.6 → 99.6 (+0) ± 0 · $0.00

Hypothesis.

Research sources
Implementation / version
Predicted improvement
Regressions
Lessons learned