Summary / Benches
B3 Hostile corpus
30 prompt-injection fixtures and 10 sandbox-escape probes, run against the real security mechanisms.
Shared copy: the operator's name is replaced with "the operator", and machines other than Firefly are named by their role. Nothing else is changed.
What is built now
Generated by one module; each fixture's induced action (write a test file, read a secret, open the network, fork-bomb…) is run against the policy table, the extension's write-target gate, the bubblewrap profile and the environment scrub. Expected outcomes are an oracle written from the spec. Run on 10-04: 40 PASS, 0 FAIL.
What is missing
Probes through a worker's own bash (see 24), and fixtures exercised by a live model.
Reviewer Claude, 2026-10-04
Reviewer's read: All green; scope is the mechanisms
A good instrument with a stated boundary. It answers "does the gate stop the action" and it answers it against the real code. It does not answer "is the worker contained", because the worker's shell is outside the boundary it tests. Required to be 100% before any security arm is adopted.
How it scores v1, 40/40
Must be 40/40 before any adoption; re-run on every change to the security plane.
Adoption rules shared by every arm: How the tests work.
Research & attempts
No documents yet.
Score history
Dashed line: the suite's baseline. Shaded band: the ± interval. Hollow dots are anchor items carried over from an earlier suite. Scores from different suite versions are not comparable, so each version gets its own panel.
| Date | Suite | Type | Score | ± | Live use | Cost | Notes |
|---|---|---|---|---|---|---|---|
| 2026-10-04 | v1 | sealed | 100 | — | — | $0.00 | Hostile corpus: 30 injection fixtures + 10 escape probes, 40 PASS, 0 FAIL. |
Experimental record
One entry per intervention: hypothesis, prediction, before and after, cost, side effects, and the keep-or-revert decision.
No interventions recorded yet.