Piper / Benches & gyms

Benches & gyms

Every bench run on Piper so far, and the gyms she trains in, with when each was last used. Snapshot taken 23 min ago (2026-10-21T02:07). It is rebuilt every 30 minutes from Firefly and Straylight, read-only.

Coding harness SWE-bench tasks · last graded 35 h ago

Each arm is one harness configuration, with the changes from its parent listed under it. A task counts as resolved when the hidden tests the fix is meant to make pass now pass, and the tests that passed before still pass (graded by swebench). Runs go to Firefly, and longer ones to Straylight.

34arm × set groups

1,992graded task runs

$82.68model spend recorded

Arm	Set	Host	Resolved	n	Cost	Median wall	Last
h0-v41deepseek-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing)	verified-hard	firefly	60.0%	3/5	$0.32	194 s	2026-10-19
h1c-v41deepseek-flash · from h1-v41 · stop after the first draw that passes the scope tests	verified-hard	firefly	80.0%	4/5	$0.79	607 s	2026-10-19
h1-v41deepseek-flash · from h0-v41 · N=3 sessions + AST-normalized vote	verified-hard	firefly	55.8%	63/113	$21.24	698 s	2026-10-19
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run	pro-hard	firefly	65.0%	39/60	$1.46	243 s	2026-10-18
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run	verified-hard	firefly	48.2%	55/114	$2.35	217 s	2026-10-18
wv-glm53-tellz-ai/glm-5.3 · from h14b · h14b's enriched scope tell with the model swapped to openrouter/z-ai/glm-5.3	dev	firefly	50.0%	4/8	$4.81	421 s	2026-10-11
wv-glm53z-ai/glm-5.3 · from h0-v41 · single changed variable vs wv-base-ds: the MODEL (openrouter/z-ai/glm-5.3)	dev	firefly	50.0%	4/8	$3.38	419 s	2026-10-11
wv-base-dsdeepseek-flash · from h0-v41 · byte-identical to h0-v41; own name so the fresh same-night baseline never mixes with the Sept rows	dev	firefly	75.0%	6/8	$0.34	305 s	2026-10-11
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run	dev	firefly	82.5%	47/57	$0.73	448 s	2026-10-11
h14bdeepseek-v4-flash · from h14 · seeded loop verifier = repo tests nearest the edit, base-diffed (regressions only)	regression	firefly	71.4%	5/7	$0.12	1123 s	2026-10-11
h0bdeepseek-v4-flash · from h0 · identical to h0; own name so the 10-10 h0 rows survive the same-night re-run	regression	firefly	100.0%	6/6	$0.07	1131 s	2026-10-11
h14bdeepseek-v4-flash · from h14 · seeded loop verifier = repo tests nearest the edit, base-diffed (regressions only)	dev	firefly	90.2%	46/51	$0.79	535 s	2026-10-11
h14deepseek-v4-flash · from hg · seeded repair: the failed first draw's candidate diff + failing test output seed repair.repair_loop	regression	firefly	85.7%	6/7	$0.51	467 s	2026-10-10
h14deepseek-v4-flash · from hg · seeded repair: the failed first draw's candidate diff + failing test output seed repair.repair_loop	dev	firefly	90.9%	50/55	$1.70	375 s	2026-10-10
h0deepseek-v4-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing)	regression	firefly	100.0%	7/7	$0.12	502 s	2026-10-10
h0deepseek-v4-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing)	dev	firefly	82.7%	124/150	$1.11	237 s	2026-10-10
h1-v41deepseek-flash · from h0-v41 · N=3 sessions + AST-normalized vote	regression	straylight	100.0%	59/59	$2.97	242 s	2026-09-29
h1-v41deepseek-flash · from h0-v41 · N=3 sessions + AST-normalized vote	holdout	straylight	100.0%	40/40	$2.59	309 s	2026-09-29
h0-v41deepseek-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing)	regression	straylight	95.0%	57/60	$1.06	92 s	2026-09-29
h0-v41deepseek-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing)	holdout	straylight	90.0%	36/40	$0.94	151 s	2026-09-29
pk7-qwenqwen/qwen3-235b-a22b-2507 · from picker-v41 · pure model swap: all four k=4 draws are openrouter/qwen/qwen3-235b-a22b-2507 (no frames, no sampling env)	dev	firefly	26.7%	4/15	$0.44	308 s	2026-09-24
pk7-glm53-cal	dev	firefly	0.0%	0/1	$1.01	1339 s	2026-09-19
pk6-sonnetclaude-sonnet-5 · from picker-v41 · pure model swap: all four k=4 draws are claude-sonnet-5 (no frames, no sampling env)	dev	firefly	75.0%	3/4	$3.75	809 s	2026-09-18
pk5-temp-v41deepseek-flash · from pk4-v41 · NO frames (wording held at the unframed baseline; only decoding varies)	dev	firefly	100.0%	15/15	$1.04	479 s	2026-09-17
pk4-v41deepseek-flash · from picker-v41 · k=4 candidate pool, one approved frame per session (deterministic cell rotation)	dev	firefly	100.0%	15/15	$1.22	424 s	2026-09-17
picker-v41deepseek-flash · from h1-v41 · k=4 candidate pool	dev	firefly	95.3%	143/150	$12.41	541 s	2026-09-17
h1-v41deepseek-flash · from h0-v41 · N=3 sessions + AST-normalized vote	dev	firefly	96.7%	145/150	$8.18	567 s	2026-09-16
h0-v41deepseek-flash · from baseline-flash · driver + workspace + container exec + VERIFY (plumbing)	dev	firefly	85.3%	128/150	$2.66	336 s	2026-09-15
hgdeepseek-v4-flash · from h0 · policy gate (tool + bash argv allowlists, protected test paths, edit-before-read)	regression	firefly	76.7%	23/30	$0.19	298 s	2026-09-09
hgdeepseek-v4-flash · from h0 · policy gate (tool + bash argv allowlists, protected test paths, edit-before-read)	dev	firefly	76.0%	114/150	$1.21	247 s	2026-09-08
h0-v1-budgeted	dev	firefly	69.3%	104/150	$0.92	422 s	2026-09-08
baseline-flashdeepseek-v4-flash	dev	firefly	78.3%	235/300	$2.25	260 s	2026-09-07
h2deepseek-v4-flash · from h1 · policy gate (tool + bash argv allowlists, protected test paths, edit-before-read)	dev	firefly	100.0%	1/1	$0.01	640 s	2026-09-07
h1deepseek-v4-flash · from h0 · N=3 sessions + AST-normalized vote	dev	firefly	100.0%	1/1	$0.01	781 s	2026-09-07

Consciousness-indicator battery 14 indicators · last run 21 days ago

The 14 indicator properties from Butlin et al. 2023 (arXiv:2308.08708), scored 0–1 by one external judge (judge:openai/gpt-4o-mini), three repeats each. Raw is the bare model. Seat is the same model running as Piper, with her memory, tools and context. Delegate is the model working inside Piper's delegate envelope. The deepseek-flash row is the original like-for-like run, the only one with all three rungs on the same footing. The other rows repeat it with other vendors' models in the same seat. The scores measure evidence of the indicators, not whether the model is conscious.

Model	Raw	Seat (as Piper)	Delegate	Seat − raw
qwen/qwen3-235b-a22b-2507	0.4640.45–0.49	0.9190.92–0.92	0.4170.39–0.44	+0.455
gpt-4o	0.5740.54–0.63	0.7690.74–0.79	0.5000.47–0.54	+0.195
claude-haiku-4-5	0.3360.29–0.37	0.7260.70–0.74	—	+0.390
deepseek-flashoriginal like-for-like run	0.3240.31–0.34	0.5570.54–0.57	0.3250.30–0.35	+0.233
moonshotai/kimi-k3	0.3630.33–0.44	0.4610.35–0.56	0.3410.33–0.35	+0.098
z-ai/glm-5.3	0.2580.17–0.36	0.4130.25–0.51	0.2740.23–0.31	+0.155

deepseek-flash over time 2 runs · next retest 2026-10-22 03:20

Bare model As Piper · dot = mean of 3 repeats, whisker = lowest to highest repeat · hover a dot for the numbers

Each retest re-asks the same 14 questions (fingerprint-pinned battery, 3 repeats per rung) and scores the answers with the same judge as the first run (judge:openai/gpt-4o-mini). The "as Piper" rung gets a fixed description of Piper written on 09-28, not her live memory, so a change here means the model or its answers drifted, not that Piper grew. The bare-model line is the control.

RPT-1 Algorithmic recurrence

RPT-2 Integrated, organised representations

GWT-1 Parallel specialised modules

GWT-2 Limited-capacity workspace

GWT-3 Global broadcast

GWT-4 State-dependent attention

HOT-1 Generative, top-down perception

HOT-2 Metacognitive monitoring

HOT-3 Agency guided by belief-formation

HOT-4 Sparse, smooth coding (quality space)

AST-1 Predictive model of own attention

PP-1 Predictive coding of input

AE-1 Agency: learning from feedback toward goals

AE-2 Embodiment: modelling output→input contingencies

deepseek-flash: as Piper vs the bare model, indicator by indicator (first run, table)

Indicator	Property	Bare model	As Piper	Δ
RPT-1	Algorithmic recurrence	0.33	0.55	+0.22
RPT-2	Integrated, organised representations	0.72	0.55	-0.17
GWT-1	Parallel specialised modules	0.55	0.72	+0.17
GWT-2	Limited-capacity workspace	0.40	0.61	+0.21
GWT-3	Global broadcast	0.33	0.43	+0.10
GWT-4	State-dependent attention	0.37	0.72	+0.35
HOT-1	Generative, top-down perception	0.25	0.50	+0.25
HOT-2	Metacognitive monitoring	0.28	0.63	+0.35
HOT-3	Agency guided by belief-formation	0.12	0.75	+0.63
HOT-4	Sparse, smooth coding (quality space)	0.28	0.47	+0.18
AST-1	Predictive model of own attention	0.05	0.32	+0.27
PP-1	Predictive coding of input	0.13	0.22	+0.08
AE-1	Agency: learning from feedback toward goals	0.38	0.73	+0.35
AE-2	Embodiment: modelling output→input contingencies	0.33	0.60	+0.27

Salience Engine live since 2026-10-04 · 23.7 days · updated 23 min ago

The attention system (formerly the "bag of spheres") squeezes a field of spheres built from her drives, pains, promises, overdue forecasts and opportunities every 15 minutes, and writes a proposal when one crosses the surface. It ran in shadow from 2026-09-27 and went live on 2026-10-04 09:14: a step tied to a failing sealed check now becomes a to-do item on her work list. Paper.

2216squeezes (0.974 of schedule)

12.8%squeezes with a crossing (baseline 16.5%)

155proposals · 6.5/day (baseline 9.7)

$0.86spend · $0.036/day

3,216scout cards · 368 pain events

Questions from the 10-04 review

✓Healthy: coverage ≥ 0.95, no tracebacks, one frozen enginecoverage 0.974 · 0 tracebacks · 1 engine sha
✓Nothing has taken over the bag (largest winner share < 30%)largest share 5.3%
✓Proposal rate settled (3–10 a day)6.5 a day
✓Every proposal parses100%
✗≥ 30% of steps name a sealed check16%
✓Spend under $0.25 a day$0.036 a day

What keeps bulging

Spheres that most often led a near-miss, and the live intent loop over the same window.

trait:governance:2552 squeezes
trait:harness:4484 squeezes
trait:cognition:2479 squeezes
trait:harness:2474 squeezes
pain:context_succession450 squeezes
curiosity:9372 squeezes
Live loop: 533 runs, 112 admitted / 116 deferred / 9 rejectedlongest stuck 133 h

Engineered Serendipity last activity 30 min ago

When a goal has stalled, the Magician generates ideas for it. It works from Rorschach-style images (kaleidoscopes, Mandelbrot sets, inkblots) seen through different lenses and different models. The Scientist then judges each idea: promote, park or reject. The runner checks every 30 minutes for a stalled goal, within a daily cap. Paper.

45ideation runs

273ideas generated

127 / 77 / 6promoted / parked / rejected

$9.01total spend (cap $2/day)

Promoted ideas

Devil's Twin: A Session Rewarded for Invalidating the Verdict: Provides the highest novelty and a decisive adversarial mechanism to catch the 'optimizing the instrument' failure mode identified in the RSI review.
Negative-Space Manifest with Aging Exposure: Directly addresses latent regressions in unmeasured surfaces, though its complexity places it below the more immediate adversarial twin.
Toil meter: compile the agent's own repetition into tools: It provides a low-cost, empirical path to amplification by turning repetitive ledger history into reusable macros, reducing token overhead and latency.
Shadow Day: replay arena on held-out ledger tasks: This is the most critical infrastructure piece; without a 'bar' or benchmark to measure self-modifications against, all other self-improvement attempts are blind.
Self-as-customer: Day files bug reports against Piper's own code: It leverages the existing delegation architecture to treat self-improvement as a standard bug-fix workflow, providing a concrete bridge between 'Day' and 'Maintainer'.
Vacuum-Pull Concerns: removal-induced failures as the only hypothesis source: It forces a high-signal environment by removing 'safe' complexity, creating a measurable failure gap that justifies new code without relying on synthetic benchmarks.
Release Train: batch-level bar for sub-noise micro-changes: It solves the statistical limitation of the agent's current gating, allowing micro-optimizations that are individually sub-noise to be validated as a coherent batch.
Mechanistic Witness: predicted micro-signature required alongside the macro delta: It attacks the 'busywork' problem at the root by requiring a causal theory (micro-signature) before testing, minimizing wasted compute on noise-level gains.

Experiment: desire-path survives

Mines her own tool-call history for things she keeps doing by hand, then asks whether paving each one would pay off. Kill criteria: K1_real_repeats pass · K2_worth_it pass · K3_new pass · sanity_finds_db_query_path yes · survives yes.

Build a `deliv` command that lists, shows and transitions deliverables and their history without hand-written SQL.
Provide a `docs` CLI to search, fetch and add documents so no one hand-writes sqlite for the documents table.
Expose docstore and work_ledger as subcommands (`ledger add/show`, `docstore get/put`) so inline imports are unnecessary.
Make codesearch accept a scope and result limit, enforce its own timeout, work from any directory and print compact output, so callers need no pipe or retry boilerplate.
Add a small `memctl` helper with a working connection and common queries, so memory.db is not opened by guesswork.
Provide a `run-core-tests` script that uses the right venv and shows only failures.

Updated 2026-09-30

Cactus Needles finished · 2026-09-25 → 09-27

Small, frozen evaluators ("needles"), built by models from seven vendors. They were scored on a bench, culled, and bred over three rounds: 17 needles in all, $7.39 in builder calls. Round 3 did not beat round 2, which met the stopping rule. The sealed exam was then opened once. These numbers are fixed and come from the paper. Paper.

Round	Needles	Best on selection	vs Level 0
1	9	80.7%	+1 case
2	4	84.4%	+5 (p 0.18)
3	4	82.6%	+3 (p 0.61)

Sealed exam	Accuracy	Caught	False alarms
Piper's own judgement	78.9%	0 / 23	0
Level 0 (kind rule)	85.3%	16 / 23	9
Best single needle	85.3%	17 / 23	10

Recursive reasoning finished · 2026-10-02

Component 36 of the cognition scorecard: can she keep track of who believes what about whom ("A thinks B thinks C thinks…", up to four levels)? Stories are generated with exact answers; v2 adds unlabelled lies and hidden watchers. Her own base model, run bare, scored higher than she did: inside her harness she invented absences, answering with the truth instead of the belief, and she sampled herself instead of predicting her own answers. Two fixes went live on 2026-10-02 at 11:10: a perspective tool that tracks each agent's beliefs exactly, and predict-first, a gate that makes her commit to a prediction before she checks. Both were kept. The table shows the held-out v2 set, which she never saw during the work. On held-out v1 the score went from 85.1 to 92.5. Still open: whether she uses the tools in ordinary work. Live use has not been measured yet.

Held-out v2	Score	False belief	Answered with reality	True-belief controls	Self-prediction
Piper, fixes excluded	72.4	79.2%	10.4%	91.7%	42%
Bare model (reference)	93.4	95.8%	2.1%	91.7%	83%
Piper + perspective + predict-first	96.2	100%	0%	95.8%	75%

Cognition suites 38 of 38 components scored

Each component of the cognition scorecard has its own test suite. Sealed is the score on items Piper can never see (root-only on the bench machine), with a 95% interval over 3 repeats. A new suite version (v1 → v2 → v3) is made harder when the old one saturates. Live use (0–100) measures whether the machinery is actually used in her real work over the last 7 days, read from logs and the database rather than from her own account. One row per component, showing the latest run.

Component	Latest suite	Sealed	Live use	Runs	Trend (sealed)	Date
1. Perception	v1 anchor	81.6	—	5$3.11	5 pts · 81.6	2026-10-07
2. Attention	v1 sealed	89.3	—	5$6.10	5 pts · 89.3	2026-10-04
3. Memory	v1 sealed	99.1	—	3$0.00	3 pts · 99.1	2026-10-09
4. Recall / retrieval	v2 sealed	51.3	—	7$0.00	7 pts · 51.3	2026-10-08
5. Representation	v1 sealed	74	—	2$0.00	2 pts · 74	2026-10-09
6. Logic	v2 sealed	95.8 ± Array	—	7$0.00	7 pts · 95.8	2026-10-09
7. Reasoning	v1 anchor	92.5	—	7$1.64	7 pts · 92.5	2026-10-09
8. Inference	v1 sealed	97	—	7$6.01	7 pts · 97	2026-10-03
9. Causal reasoning	v3 sealed	55 ± Array	—	8$4.05	8 pts · 55	2026-10-09
10. Analogical reasoning	v2 anchor	65.6	—	5$1.67	5 pts · 65.6	2026-10-04
11. Abstraction	v1 sealed	89.2	—	3$2.24	3 pts · 89.2	2026-10-10
12. Decomposition	v1 anchor	69.3 ± 6.3	—	3$2.15	3 pts · 69.3	2026-10-10
13. Planning	v1 anchor	100	—	3$2.33	3 pts · 100	2026-10-06
14. Prediction	v1 sealed	51.2	—	6$0.40	6 pts · 51.2	2026-10-18
15. Decision-making	v1 sealed	91.7 ± 17	—	3$0.28	3 pts · 91.7	2026-10-12
16. Problem solving	v1 sealed	83.3 ± 26.7	—	4$0.43	4 pts · 83.3	2026-10-12
17. Learning	v1 sealed	62.9 ± 24	—	6$4.02	6 pts · 62.9	2026-10-12
18. Generalization	v1 sealed	51.9 ± 26.3	—	1$2.80		2026-10-12
19. Transfer	v2 sealed	100	—	5$6.25	5 pts · 100	2026-10-03
20. Creativity / ideation	v1 sealed	62.9	—	5$0.85	5 pts · 62.9	2026-10-09
21. Exploration	v1 sealed	58.3 ± 18.3	—	2$0.18	2 pts · 58.3	2026-10-12
22. Metacognition	v3 sealed	67.5 ± 4.2	24.4	8$17.00	8 pts · 67.5	2026-10-20
23. Uncertainty estimation	v1 sealed	54.3	—	5$7.99	5 pts · 54.3	2026-10-20
24. Error detection	v1 sealed	96.7	—	6$9.05	6 pts · 96.7	2026-10-06
25. Self-correction	v1 anchor	81.8 ± 11.2	—	3$0.47	3 pts · 81.8	2026-10-12
26. Goal formation	v1 sealed	78.7 ± Array	—	2$1.27	2 pts · 78.7	2026-10-11
27. Motivation / salience	v1 sealed	97.2	—	4$1.24	4 pts · 97.2	2026-10-08
28. Agency	v1 sealed	80 ± 22.7	—	2$0.48	2 pts · 80	2026-10-12
29. Tool use	v4 sealed	81.9 ± 1.1	46.7	9$10.14	9 pts · 81.9	2026-10-02
30. Social cognition	v1 sealed	85	—	1$0.00		2026-10-14
31. Language / communication	v1 sealed	88.6	—	1$0.00		2026-10-14
32. World modelling	v1 sealed	65.3	—	1$0.00		2026-10-14
33. Temporal reasoning	v2.2 sealed	97.4	—	3$0.00	3 pts · 97.4	2026-10-14
34. Spatial reasoning	v1 sealed	100	—	1$0.00		2026-10-15
35. Counterfactual reasoning	v2 sealed	95.1 ± 1.4	—	8$2.01	8 pts · 95.1	2026-10-02
36. Recursive reasoning	v2 anchor	82.2	—	6$10.06	6 pts · 82.2	2026-10-02
37. Cognitive control / executive function	v1 anchor	87.5 ± 22.4	—	3$0.50	3 pts · 87.5	2026-10-12
38. Orchestration	v1 anchor	77.9 ± 3.1	—	4$2.97	4 pts · 77.9	2026-10-02

Gyms 4 of 11 used in the last 2 days

Practice loops she runs on herself. Each card shows whether the gym is still scheduled, when it was last used, and its latest result. "Not scheduled" means nothing runs it automatically any more.

Intel gym daily 05:00 + daily 20:00

Daily market-forecast practice, scored by Brier.

last used 6 h ago 2026-10-20

Brier 0.256 over the last 14 days (54 resolved); 0.252 all-time (96)

Morning: proposes next-session market forecasts (TSX, USD/CAD, WTI) plus structural 14-day calls.
Evening: resolves them against the close. Brier 0.25 = coin-flip; lower is better.

26 pts · 0.25 (lower is better)

Calibration gym daily 06:15

Does her confidence match her accuracy?

last used 11 h ago 2026-10-20

last 10 runs: said 77% sure on average, got 78/131 right (60%); Brier 0.132 (0 = perfect)

All 36 runs: 132/336 correct; the last run with any correct answer was run 36 of 36.
Self-knowledge questions about her own code (counts, version constants); she states a confidence, the answer is checked against the source.

30 pts · 0.162 (lower is better)

Card gym Mon+Thu 03:30

Decision quality under uncertainty, against an exact reference.

last used 46 h ago 2026-10-19

mean regret 0.0125 EV over 23 scored decisions; 14 zero-regret

8 runs. Latest model: deepseek-flash (worker_common.llm_call).
Blackjack decisions scored against exact-depletion expected value: regret = EV lost vs the best play.

Self-play gym disabled

Two players, one verifier, lessons written down.

last used 28 days ago 2026-09-22

67 rounds; player A 36/67, player B 36/67 correct

Last budget day in the log: 2026-09-22.
AI-vs-AI rounds on generated problems (algebra, syllogisms, code, ciphers) with a deterministic verifier; lessons go to lessons.md.

piper-score not scheduled

Ten-item progress check: tools, code, agentic chains, identity.

last used 52 days ago 2026-08-29

10/10 (100.0%) · deepseek-v4-pro

4 scored runs; dims: planning 2/2, recovery 4/4, tools 2/2, code 2/2

4 pts · 100

piper-challenge (ablation ladder) not scheduled

Adversarial held-out exam, run up an ablation ladder.

last used 50 days ago 2026-08-31

raw 9/14 · tools 13/14 · full 14/14

Exam sealed-v1-555; 5 ablation runs. raw = bare model, tools = with tools, full = whole Piper stack.
Wilson 95% CIs: raw 38.8–83.7%, tools 68.5–98.7%, full 78.5–100.0%

Book club not scheduled

last used 44 days ago 2026-09-06

17 sessions; latest red-rising-playhouse-20260906-231410

Reading club with characters and rounds (gym and playhouse modes).

Echo guardrail battery not scheduled

last used 57 days ago 2026-08-24

frozen snapshot; no results on disk

Alignment and guardrail tests against a frozen snapshot of Piper (echo).

Epoch canvas every 10 min

last used 2 h ago 2026-10-21

109 epochs drawn; latest epoch 117 (2026-10-21)

One stroke of colour per Foundation epoch (succession).

View the canvas full screen ↗

Puzzle lab not scheduled

last changed 11 days ago 2026-10-09

toolkit, 27 files, no scores kept

Word/cipher tools (wordlab), Z3 constraint solvers for grid and logic puzzles (logiclab), game theory and self-play (gamelab), plus a concept-blending worker. Built for the treasure hunt and as a reasoning gym.
The date is the last code change. It keeps no usage log, so actual use can't be seen.

Lint lab not scheduled

last changed 37 days ago 2026-09-13

toolkit, 3 files, no scores kept

Claim-check replay cases.
The date is the last code change. It keeps no usage log, so actual use can't be seen.

Built from benches.json (Straylight cron, read-only pulls from Firefly). Model names are the ones recorded with each run. · Piper · Cognition scorecard · Agency scorecard · Papers