Summary / 14. Prediction / Document
2026-10-16
Shared copy: the operator's name and machine names are replaced by roles. Nothing else is changed.
Rendering…
# Piper: From Forecasting to Predictive Intelligence ## A research-grounded roadmap based on Yann LeCun's world-model architecture **Research date:** October 16, 2026 **Starting point:** Piper Cognition #14 — Prediction, score 45.7 **Objective:** Move Piper from predicting whether tasks will succeed to learning how her environment changes in response to actions, simulating alternatives, and using those simulations to improve decisions. ## 1. The central finding After examining LeCun's research and comparing it with Piper's prediction benchmark, I think we can identify a specific architectural gap. **Piper currently predicts outcomes. LeCun wants agents to predict state transitions.** These are related but fundamentally different capabilities. Your current system answers questions such as: - Will this dispatch pass its acceptance criteria? - Will Bitcoin close higher tomorrow? - How confident am I that this intervention will succeed? LeCun's proposed architecture addresses a different question: **Given the state of my environment, and an action I might take, what will the resulting state of that environment be?** And then: **What sequence of actions would move the environment toward a desired future state?** That distinction is particularly important for recursive self-improvement. Consider a hypothetical intervention. Piper discovers that her delegation system is wasting resources by repeatedly dispatching similar tasks. Her existing approach might look like: `Identify problem → Generate solution → Predict probability of success → Implement → Benchmark` A world-model-based approach would look like: `Observe state → Construct possible actions → Predict their effects → Simulate action sequences → Select the best sequence → Execute → Observe changes → Learn` Your existing prediction ledger can support that second architecture, but it does not establish that Piper already possesses it. ### The destination I propose a new capability called the **Predictive World Model**, initially developed alongside Cognition #14 rather than replacing its existing benchmark. Its long-term objective would be: > Piper maintains an evolving internal representation of her environment, learns how actions alter that environment, predicts consequential future states across multiple time horizons, and uses those predictions to select actions that advance her goals. Her model continuously improves by comparing predicted and observed transitions. This is a much more ambitious goal than improving her Brier score. It also gives us a concrete definition of progress. A successful predictive model should demonstrate three capabilities: 1. **Prediction:** It forecasts consequential changes in the environment better than simple baselines. 2. **Control:** Using those forecasts leads to better decisions than acting without them. 3. **Learning:** Its predictions and resulting decisions improve through accumulated experience, including on unfamiliar situations. I would not declare this capability successful until it demonstrates all three. --- # 2. The research: Five papers worth building from I've selected four papers by LeCun or his collaborators, plus one complementary paper from DeepMind. These are not simply papers about forecasting. They address the architecture needed to make prediction useful for intelligence. ## Paper 1 — A Path Towards Autonomous Machine Intelligence **Yann LeCun — June 2022** [Read the original paper — OpenReview PDF](https://openreview.net/pdf?id=BZ5a1r-kVsf) This is the most important theoretical foundation for what we're trying to achieve. LeCun proposes an architecture consisting of interacting components responsible for perception, world modeling, memory, cost evaluation, intrinsic motivation and action selection. The world model predicts how the environment will evolve, including how it will respond to the agent's actions. Crucially, LeCun proposes doing this at multiple levels of abstraction and multiple time horizons. For Piper, that might mean: | Horizon | Example prediction | |---|---| | Immediate | This code change will alter these three functions. | | Short-term | The next benchmark will expose a regression in dispatch handling. | | Medium-term | This change will reduce wasted delegate calls over 100 tasks. | | Long-term | Improving delegation efficiency will free resources for other cognitive work. | A useful model must connect those levels rather than treating each prediction independently. LeCun also separates **predicting the future** from **evaluating whether that future is desirable**. That is important because an accurate world model doesn't automatically make intelligent decisions. ### What Piper should borrow An explicit state representation, a transition model, a goal representation, an evaluator of predicted future states, and a mechanism for choosing actions. These can initially be implemented using conventional Python, structured data, SQLite and Piper's existing LLM. They do not need to begin as neural networks. **The key principle: Make predictions about states, not merely scores.** Source: [OpenReview](https://openreview.net/pdf?id=BZ5a1r-kVsf\&utm_source=chatgpt.com) ## Paper 2 — Introduction to Latent Variable Energy-Based Models **Anna Dawid and Yann LeCun — June 2023** [Read the paper — arXiv](https://arxiv.org/abs/2306.02572) This is the companion paper I would use to understand the mechanisms behind LeCun's proposal. It explains latent-variable models, energy-based models and hierarchical JEPA. Its relevance is that the future is not always uniquely determined. The same action can produce different outcomes depending on circumstances the agent cannot observe. Suppose Piper attempts to improve a coding harness. A prediction of: > "Benchmark score will increase by 6%." is too narrow. A better predictive representation might include: - Expected improvement in successful cases. - Probability of regression. - Additional CPU and token consumption. - Effects on concurrent delegates. - Dependencies that could fail. - Uncertainty about unobserved conditions. Some uncertainty is reducible by gathering more information. Some is irreducible. LeCun's latent-variable framework provides a theoretical approach to representing multiple possible futures without requiring an exhaustive prediction of every detail. ### What Piper should borrow Represent future states as **structured distributions or sets of plausible outcomes**, rather than single confident narratives. Initially, that means ranges, probabilities and explicit unknowns. It does not require implementing an energy-based neural network immediately. Source: [arXiv](https://arxiv.org/abs/2306.02572?utm_source=chatgpt.com) ## Paper 3 — Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture **Assran, LeCun and collaborators — January 2023** [Read I-JEPA — arXiv](https://arxiv.org/abs/2301.08243) This is where LeCun's ideas become particularly interesting. JEPA attempts to predict useful abstract representations instead of reconstructing all the raw details of an observation. The distinction matters. Imagine watching a glass fall from a table. A pixel-prediction system attempts to predict what every future pixel will look like. A representation-based system can instead learn information such as: - The glass is moving downward. - The table no longer supports it. - Impact is approaching. - Breakage is possible. The original I-JEPA paper demonstrates a representation-learning approach using images. It does not itself establish action-conditioned planning. But its central principle translates well to Piper. **Piper should predict the consequential features of her future environment, not every byte of her future environment.** For example, if Piper changes a database query, she doesn't need to predict the contents of every affected log file. She might only need to predict: - Query latency. - Result correctness. - Memory consumption. - Downstream dependencies. - Expected failure conditions. This gives us a practical path toward a small, structured predictive representation. Source: [arXiv](https://arxiv.org/abs/2301.08243?utm_source=chatgpt.com) ## Paper 4 — V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning **Assran, Bardes, LeCun and collaborators — June 2025** [Read V-JEPA 2 — arXiv](https://arxiv.org/abs/2506.09985) This is probably the strongest experimental inspiration for Piper. V-JEPA 2 moves beyond learning useful visual representations. The researchers subsequently trained an action-conditioned variant, V-JEPA 2-AC, using robot interaction videos. That variant learned to predict future representations conditioned on actions. The researchers then used those predictions for robotic planning. The system demonstrated object manipulation using visual goals, with relatively little robot interaction data compared with its earlier video pretraining. The important idea is: **First learn useful representations of the world. Then learn how actions change those representations. Finally, plan using the learned transitions.** For Piper, the analogous training data would not be videos. It would be her existing operating history: `State before → Action performed → State afterward` Your ledgers, dispatch records, test outcomes and operational telemetry may already contain much of the necessary information. The major missing element may be reconstructing those records into reliable, time-ordered transitions. We should treat that as a hypothesis to audit, not assume the data are complete. ### What Piper should borrow The separation between: 1. State representation. 2. Action-conditioned prediction. 3. Planning toward a desired state. V-JEPA 2 provides evidence that this general approach can work in a physical setting. It does not demonstrate that directly translating its methods to software agents will work. That's what we would test. Source: [arXiv](https://arxiv.org/abs/2506.09985?utm_source=chatgpt.com) ## Paper 5 — DreamerV3: Mastering Diverse Domains through World Models **Danijar Hafner, Jurgis Pasukonis, Jimmy Ba and Timothy Lillicrap** [Read the paper — Nature, April 2025](https://doi.org/10.1038/s41586-025-08744-2) This paper is not from LeCun's team, but I would include it because it demonstrates something his architecture needs: learning effective behaviour from a model of the environment. DreamerV3 uses a world model to imagine future trajectories, an actor to select actions and a critic to assess their value. It was demonstrated across more than 150 tasks. The important lesson for Piper is that **a predictive model must affect action selection**. A model that accurately predicts failures but never changes what Piper does is not delivering its full value. We can borrow the conceptual separation without implementing DreamerV3's neural architecture. Source: [DOI](https://doi.org/10.1038/s41586-025-08744-2?utm_source=chatgpt.com) --- # 3. What Piper's existing results tell us Before writing new code, it's important to recognize what you've already established. Your prediction suite is considerably more sophisticated than a typical agent evaluation. It has independent testing, scoring against baselines, leakage checks, preregistered interventions and explicit failures. We shouldn't discard any of that. ### Your current results | Capability | Result | Interpretation | |---|---:|---| | World forecasting | BSS +0.243 | Genuine improvement over the chosen reference baseline | | Self-forecasting | BSS −0.414 | Worse than the rolling pass-rate reference | | Self-forecast discrimination | AUROC 0.528 | Little demonstrated ability to distinguish successful from unsuccessful dispatches | | Sealed seat probe | 96.4 | Strong performance when relevant reference-class information is explicitly supplied | | Reference-class card | Inconclusive | Improvement not clearly distinguishable from the placebo | | Failure-mode decomposition | Reverted | Produced worse forecasts | | State-transition modeling | Not measured | We don't yet know | | Counterfactual action selection | Not measured | We don't yet know | | Multi-step predictive planning | Not measured | We don't yet know | The last three rows are where I would concentrate our next effort. There is also an important distinction between *uncertainty calibration* and *environment modeling*. Piper could improve her Brier score substantially without learning anything new about how her environment works. For example, if 92% of her dispatches pass, predicting 0.92 every time can produce a good Brier score. But that doesn't help her distinguish two interventions that have different consequences. A world model has to answer something richer: **What happens if I do A, compared with B, compared with doing nothing?** ### Another important finding Your results indicate that simply prompting Piper to reason harder is unlikely to close this gap. U2 demonstrates that additional failure-mode reasoning can make her predictions worse. That's an important warning. I would not solve this by inserting a giant prompt that says: > Think deeply about every possible consequence before acting. Instead, I would give her a small, structured mechanism that forces predictions to correspond to observable quantities, records what happens and learns from the differences. And I would measure whether consulting that mechanism improves actual decisions. --- # 4. Proposed architecture: Piper's Predictive World Model I suggest starting with a separate subsystem. Working name: **Piper World Model — PWM** Initially, it would operate as an advisory module, not as a new autonomous controller. That distinction matters because Piper already has governance, delegation boundaries, a salience engine and an RSI workflow. We want to improve those systems, not bypass them. ### Proposed data flow ```text PIPER FOUNDATION | v SALIENCE ENGINE | v PROPOSED GOAL | v STATE READER | v CURRENT STATE | v ACTION GENERATOR | A1 A2 A3 \ | / \ | / v v v WORLD MODEL | PREDICTED STATES | GOAL EVALUATOR | ACTION SELECTION | EXISTING DELEGATES | ACTUAL OUTCOME | STATE COMPARISON | PREDICTION ERROR | MODEL UPDATE | IMPROVEMENT MEMORY ``` The existing Foundation would still select goals, authorize actions and operate through delegates. PWM would contribute predictions and evidence to that process. ## The five components | Component | Purpose | Initial implementation | |---|---|---| | State encoder | Describe Piper's current environment | Python + JSON | | Transition model | Predict how an action changes that environment | Rules + statistical model + optional LLM | | Goal evaluator | Compare predicted outcomes with desired outcomes | Python | | Planner | Compare possible actions and sequences | Small bounded search | | Learning engine | Improve transition predictions using experience | SQLite + online updates | Notice what's absent. No specialized GPU, vector database, transformer training, giant benchmark or new foundation model is required for the first experiment. We're testing whether the architecture provides useful information before investing in a more sophisticated implementation. --- # 5. Implementation Phase 1 — Give Piper a state **This is where I would start.** Not prediction prompts. Not fine-tuning. Not model training. First, give her a representation of the current environment. ### Proposed file `world_model/state.py` It gathers a compact snapshot of relevant system variables. For an initial experiment, I'd limit the model to one domain: **Piper's delegation and execution environment**. This is a good starting point because it already has substantial instrumentation, frequent actions and measurable outcomes. ### Example state ```json { "timestamp": "2026-10-16T10:00:00Z", "domain": "delegation", "state": { "queue_depth": 18, "active_workers": 2, "worker_capacity": 3, "recent_pass_rate": 0.92, "p95_completion_seconds": 185, "recent_cost_usd": 1.42, "retry_rate": 0.08 } } ``` These are illustrative values, not claims about Piper's present state. The state snapshot needs to capture useful information without trying to represent everything happening on her host. This reflects LeCun's representation-learning principle: retain the information required to predict consequential changes. ### One important addition: history A current snapshot may not be sufficient to predict what happens next. For example, a queue depth of 18 means something different when arrivals are accelerating than when the queue is rapidly shrinking. So the encoder should optionally include a short history: ```json { "queue_depth_history": [8, 12, 15, 18], "arrival_rate_5m": 1.8, "completion_rate_5m": 1.2 } ``` The goal is a useful representation of the system's dynamics, not simply an inventory of its current values. ### Phase 1 test Capture 100 state snapshots during normal operations. For each snapshot, use the current state and recent history to predict the state five minutes later. Start with only three targets: 1. Queue depth. 2. Active-worker count. 3. Number of completed dispatches. Compare against two simple baselines: persistence (no change) and a basic queue-flow calculation. **Success criterion:** Predictions must improve on the stronger baseline using genuinely future observations. If they don't, examine whether the missing ingredient is information, model quality or measurement timing. This phase can initially be entirely passive. **Provisional implementation budget:** 1–3 hours of coding, provided existing telemetry is readily accessible. --- # 6. Phase 2 — Learn action-conditioned transitions This is the first major architectural advance. Instead of asking: > What will the system look like in five minutes? Piper asks: > What will the system look like in five minutes if I take this particular action? Mathematically, we're trying to approximate: \[ P(S_{t+1}\mid S_t,A_t) \] Where: - \(S_t\) is the current system state. - \(A_t\) is an action. - \(S_{t+1}\) is the resulting state. The practical distinction is that the model must learn the relationship between **actions and consequences**. ## Proposed code `world_model/transitions.py` Each observed transition becomes a record: ```json { "transition_id": "wm-0001", "state_before": { "queue_depth": 18, "active_workers": 2, "worker_capacity": 3 }, "action": { "type": "dispatch_task", "task_class": "code_review" }, "horizon_seconds": 300, "state_after": { "queue_depth": 14, "active_workers": 3, "completed_tasks": 5 } } ``` Once again, these are illustrative observations. The model's objective is to learn which kinds of state changes tend to follow which actions under which conditions. ### Start without neural networks I'd first implement three predictors: **Predictor 1 — Persistence baseline** Assume the state remains unchanged. **Predictor 2 — Empirical transition model** Look for similar previous states and actions, then use their observed outcomes to estimate the next state. **Predictor 3 — LLM-assisted transition model** Give Piper the state, action, relevant historical observations and a fixed output schema. Ask for predicted state changes and uncertainty. The LLM is a candidate predictor, not the source of truth. Its predictions must be scored against subsequent observations. This is important because you already know from Cognition #14 that her verbal confidence can be unreliable. ### Prevent false causal conclusions There is one complication. In ordinary operational logs, actions aren't randomly assigned. For example, Piper may dispatch more workers precisely when her queue is unusually difficult. If we observe that adding workers correlates with slow completion, it would be a mistake to conclude that additional workers cause slower completion. The transition model initially learns observational relationships. We need controlled interventions or other credible causal evidence before interpreting its outputs as reliable counterfactuals. That brings us to the next phase. --- # 7. Phase 3 — Counterfactual prediction This is the experiment I consider most important. **Can Piper predict which of two actions will produce the better outcome before performing either one?** We don't need an elaborate benchmark. We need a small environment where the consequences can be measured reliably. ## The first experiment: A simulated delegation queue Build a small Python simulator representing Piper's delegation system. Not an entire copy of Foundation. Just the following: - Tasks arrive. - Tasks have varying durations and difficulty. - Workers have capacity limits. - Dispatching consumes resources. - Some tasks fail and require retries. - Queues accumulate. - Delays and costs can be measured. Define three possible actions: - A: Dispatch another task immediately. - B: Wait for an existing task to finish. - C: Prioritize a shorter or more urgent task. Piper receives the current state and predicts the consequences of each action. For example: | Action | Predicted 5-minute completions | Predicted cost | Predicted queue | |---|---:|---:|---:| | A | 5 | $0.18 | 12 | | B | 4 | $0.11 | 13 | | C | 6 | $0.14 | 11 | These are hypothetical predictions. The simulator then executes each action from the **same saved starting state**, including matched random conditions. This gives us a controlled counterfactual experiment. We can measure whether Piper correctly predicts the changes and whether her chosen action is actually better. ### Why the simulator matters You don't have to risk live operations. You also don't need to wait for the real environment to produce sufficiently comparable situations. And unlike an LLM-generated evaluation, the simulator determines outcomes through code rather than deciding whether Piper's explanation sounds plausible. This addresses one of the problems you've repeatedly encountered with evaluator independence. ### Initial benchmark Generate 20 held-out initial states, including congestion, idle capacity, retry-heavy workloads and changing arrival rates. At each state: 1. Give Piper the state and permitted actions. 2. Record her predicted outcomes. 3. Have her select an action. 4. Simulate all alternatives using matched conditions. 5. Compare the actual results. A useful primary metric is **decision regret**: \[ R = U(A_{\text{best}})-U(A_{\text{chosen}}) \] Here, \(U\) is a predefined utility function incorporating completions, cost and delay. Regret measures how much value Piper lost by choosing something other than the best available action. Zero means she chose the best action. ### The actual success criterion I'd require two findings: **Prediction:** Her forecast errors are lower than a simple empirical baseline. **Decision quality:** Her action choices have lower average regret than a simple heuristic policy. Both results should be evaluated on held-out cases. This is the first meaningful evidence that predictive intelligence is improving control, rather than just producing better-sounding forecasts. --- # 8. Phase 4 — Introduce a small planner Once the action-conditioned model demonstrates predictive skill, we can allow it to simulate sequences of actions. This is where LeCun's architecture starts to become especially relevant. A model predicting one action's consequences isn't necessarily good at predicting what happens after several actions. Yet many of Piper's important decisions involve sequences. For example: 1. Inspect the failing component. 2. Modify a dependency. 3. Run a focused test. 4. Modify the main component. 5. Run the benchmark. 6. Accept or revert. The order matters. An intervention can be beneficial in isolation but harmful when combined with another change. ### Proposed code `world_model/planner.py` I'd implement a tiny model-predictive controller. Given an initial state and a goal: 1. Generate at most three candidate actions. 2. Predict the resulting states. 3. Evaluate their expected utility. 4. Explore at most two steps into the future. 5. Select the first action from the best sequence. 6. Execute that action through Piper's existing governance and delegation mechanisms. 7. Observe the actual state. 8. Replan. The last step is essential. **Never assume the predicted future became reality.** This is the model-predictive-control principle used in action-conditioned approaches such as V-JEPA 2-AC. ### The test Use the same delegation simulator. Compare: - A reactive policy using only the current state. - A model-based policy using two-step planning. Use 20 held-out scenarios with matched conditions. Measure completed work, delay, cost, failures and decision regret. The planner must beat the reactive policy on the predefined utility function without violating operational constraints. If planning isn't beneficial at depth two, don't increase it to depth ten. First identify whether the transition model, utility function or search procedure is responsible. --- # 9. Phase 5 — Make prediction errors teach the model This is where the system begins moving toward the larger goal. Piper already has prediction and improvement ledgers. I wouldn't create a parallel memory architecture unless necessary. Instead, I would add a structured record of **prediction residuals**. ### Proposed code `world_model/learning.py` An entry could look like this: ```json { "prediction_id": "wm-00042", "model_version": "0.1", "action": "dispatch_task", "predicted": { "queue_depth": 10, "completed_tasks": 6 }, "observed": { "queue_depth": 15, "completed_tasks": 3 }, "error": { "queue_depth": 5, "completed_tasks": -3 }, "context": { "worker_utilization": 1.0, "retry_rate": 0.18 } } ``` The learning engine would examine recurring errors. For example, it might discover that its model systematically overestimates completions when worker utilization exceeds 90%. That observation should result in a measurable model adjustment. Not just a lesson written in prose. ### First learning implementation Use an inexpensive online regression model, such as an SGD regressor, for numerical state changes. Include predictors such as: - Current queue depth. - Worker utilization. - Recent throughput. - Task class. - Action selected. - Retry rate. The model gets updated after verified outcomes. For the initial version, I would favor a simple model we can inspect over an opaque neural network. ### The essential test: Is Piper actually learning? Freeze 20 evaluation scenarios that aren't used for training. Measure prediction error before learning. Allow the model to learn from a separate stream of 50 transitions. Then evaluate the frozen scenarios again. But add a second, more important test: **20 new scenarios from a different workload regime**. Otherwise, Piper could merely be memorizing familiar conditions. #### Proposed acceptance gates | Measure | Initial target | |---|---| | State prediction error | At least 10% below strongest baseline | | Action-selection regret | At least 10% below reactive policy | | Learning benefit | At least 10% error reduction after training | | Generalization | Improvement on new workloads, not just familiar cases | | Governance violations | Zero | | Production regressions | Zero during initial shadow deployment | These are proposed engineering thresholds, not numbers derived from LeCun's papers. Treat them as preliminary gates and freeze the final definitions before testing. With samples this small, confidence intervals and repeatability matter; a 10% apparent improvement alone would not be conclusive. If a phase fails, retain its measurements and diagnose the weakness before expanding the system. --- # 10. Phase 6 — Bring the model into Piper's RSI loop This is ultimately where I want the work to lead. The delegation simulator is a controlled test environment. **The real destination is enabling Piper to predict the consequences of changing herself.** Consider Cognition #14 itself. Currently, Piper can identify that her forecasts are inaccurate, propose an intervention, implement it and evaluate the result. A predictive RSI loop would add a model of the expected consequences before implementation. For example, suppose she considers improving the salience engine. She might evaluate three options: | Proposed intervention | Expected effect | Principal uncertainty | |---|---|---| | Change salience thresholds | Fewer unnecessary dispatches | May miss worthwhile opportunities | | Improve salience memory | Less repeated investigation | Additional retrieval costs | | Modify scheduling | Better resource utilization | Possible queue instability | Initially, these would be explicit hypotheses, not authoritative forecasts. Over time, Piper would accumulate observations about how different classes of intervention affect her system. She might learn, for example, that certain changes reliably improve accuracy but degrade latency, or that modifications to one subsystem commonly cause regressions in another. That becomes a predictive model of her own architecture. ### The RSI integration I would modify the existing workflow conceptually as follows: **Current:** `Salience → Research → Intervention → Benchmark → Keep/Revert` **Proposed:** `Salience → State → Candidate actions → Predict transitions → Select → Intervention → Benchmark → Compare predictions → Update model → Keep/Revert` The existing independent benchmark remains the authority on whether the intervention improved the system. The world model does not get to rewrite acceptance criteria, adjust its own evaluation records or declare success based on predicted utility. That keeps the new mechanism inside your existing governance architecture. ### The critical RSI metric Measure: **Improvement achieved per unit of experimentation cost.** If predictive planning works, Piper should eventually become better at identifying promising interventions before spending resources implementing them. For example, compare 20 live improvement decisions made with world-model advice against a contemporaneous shadow baseline that uses the existing selection process. Track: - Independently verified improvement. - Total delegate cost. - Number of rejected interventions. - Regressions introduced. - Improvement per dollar. The goal isn't simply to make forecasts more accurate. It's to make Piper more efficient at improving herself. --- # 11. A realistic implementation sequence Here is how I'd organize the work to avoid the large, expensive benchmark projects you've been trying to eliminate. | Step | Deliverable | Initial test | Budget | |---|---|---|---| | 0 | Audit existing state/action/outcome records | Reconstruct 10 complete transitions | 30–60 min | | 1 | State encoder | 20 valid snapshots | 1–3 hours | | 2 | Passive next-state predictor | 100 observations | 1–2 hours | | 3 | Action-conditioned predictor | 20 held-out transitions | 2–3 hours | | 4 | Small queue simulator | Deterministic replay and 20 counterfactual cases | 2–4 hours | | 5 | Two-step planner | 20 planning scenarios | 1–3 hours | | 6 | Online learning | 50 training transitions, 20 held-out cases | 1–3 hours | | 7 | RSI shadow integration | 20 real improvement decisions | Reuse existing workflow | These are rough coding-effort budgets for small prototypes, not promised completion times. They depend heavily on how accessible Piper's existing telemetry and interfaces are. I would cap the initial exploratory API spend at approximately **$10 USD**, excluding existing operational activity. There is no need to spend heavily to determine whether a useful signal exists. ## Keep, stop or revert I would impose three rules. **Rule 1 — No useful state representation, no model.** If we cannot identify a compact set of variables that predict meaningful outcomes, fix the representation before adding more sophisticated reasoning. **Rule 2 — No predictive advantage, no planner expansion.** If action-conditioned predictions can't outperform a simple empirical model, additional search depth will probably amplify model errors rather than improve decisions. **Rule 3 — No better decisions, no production integration.** Even if prediction accuracy improves, the model must demonstrate a decision-making benefit before influencing live RSI behaviour. This avoids building an elaborate world-model architecture that turns out to be another source of confident but unhelpful predictions. --- # 12. Should Piper eventually use JEPA itself? Potentially, but I wouldn't start there. The first prototype described above is a **structured, software-specific world model inspired by LeCun**. It is not an implementation of JEPA. That's an important distinction. JEPA learns representations through neural training. Our initial prototype uses manually specified state variables and conventional predictors. I would consider moving toward learned representations only when the initial system demonstrates value and we can identify limitations caused by its manually defined state space. For example, suppose Piper's model consistently fails because it cannot capture complex relationships between task descriptions, code changes, dependencies and runtime behaviour. At that point we could introduce an encoder: `Rich system observation → Learned state embedding` Then train an action-conditioned predictor: `State embedding + Action embedding → Future state embedding` That would be much closer to the architectural principle demonstrated by V-JEPA 2-AC. An inexpensive experiment could use frozen embeddings from an existing small model and train only a lightweight transition predictor. We could compare those learned representations with the original structured-state model. The question would be whether the embeddings capture useful predictive information that our hand-designed variables miss. Only if they do would I consider a more substantial neural world model. --- # 13. Where this belongs among Piper's cognitive components I would not simply replace the existing #14 prediction suite. You've already done considerable work making it rigorous. Its world forecasting and self-calibration measurements remain useful. Instead, I would distinguish several capabilities: | Capability | Proposed responsibility | |---|---| | #14 Prediction | Probability and outcome forecasting | | State representation | Capturing consequential aspects of the environment | | Transition learning | Learning how states evolve following actions | | Counterfactual reasoning | Comparing alternative possible futures | | #13 Planning | Constructing action sequences | | #15 Decision-making | Selecting actions based on expected consequences and goals | | Improvement memory | Retaining predictive errors and learned relationships | | Amplification | Converting better predictions into better improvements | This suggests something important about your existing 38-component framework. **Prediction may not be one isolated cognitive capability. It may be part of the machinery connecting several cognitive capabilities together.** LeCun's architecture is interesting partly because prediction, planning, memory and action selection are integrated. Improving each separately may accomplish less than making them exchange useful information. That doesn't mean throwing away your component model. It means evaluating whether improvements are actually transferring across components. --- # 14. The experiment I would prioritize above everything else If I were choosing one immediate task for Piper, it would be this: ## Build a minimal action-conditioned world model and demonstrate that it improves a real decision. Forget the full architecture temporarily. Give her a constrained environment with: - A measurable state. - Three possible actions. - An explicit goal. - A deterministic way to execute or simulate actions. - A record of predicted versus actual consequences. Run 20 scenarios. Make her predict the resulting state for each action. Then compare her choice with the best action determined by the environment. After that, expose her to some additional transitions and repeat on new scenarios. The interesting result is not necessarily whether she performs well initially. It is whether she can **learn to predict consequences and thereby improve action selection**. If she can, you have established a basic version of the capability LeCun argues is necessary for autonomous intelligence. Not human-level predictive intelligence, and not a full JEPA. But a working, measurable foundation. --- # 15. My assessment of Piper's current position There's something noteworthy in the data you've collected. Piper scored 96.4 on a sealed seat probe when useful reference-class information was provided, yet her live dispatch predictions perform poorly. That suggests some of her difficulties may be caused by how information reaches her decision process, rather than being entirely attributable to limitations in the underlying LLM. However, the probe measured performance on structured probability questions. It did not test whether she could simulate environmental transitions. We cannot infer that she already possesses a reliable world model. The encouraging part is architectural. You already have: - Persistent memory. - Continuous operational observations. - A salience mechanism. - Delegated actions. - Instrumented outcomes. - Independent benchmark infrastructure. - A mechanism for incorporating improvements. That is a useful foundation for building a predictive system. **The missing element worth investigating is the explicit connection between observed states, actions, resulting states and future action selection.** I would make that the immediate research objective. ## The goal I would give Piper > **Develop an action-conditioned predictive world model that learns from Piper's operating experience, forecasts how candidate actions will change consequential system states, and improves goal-directed decision-making through model-based planning.** > > Begin with a small, independently testable software environment. Demonstrate predictive skill over simple baselines, reduced decision regret, and improvement through experience before integrating the model into live RSI. > > Extend the system incrementally toward multi-step planning, longer time horizons, learned representations and prediction-guided self-improvement. > > The ultimate measure of success is not forecast accuracy alone, but whether Piper can use an increasingly accurate understanding of her environment to achieve independently verified improvements more reliably and efficiently. ### One final distinction There are two separate milestones here. **Milestone A: Predictive competence.** Piper learns to predict what actions will do. **Milestone B: Predictive amplification.** Piper uses that competence to make increasingly effective improvements to herself. A successful demonstration of A does not establish B. But B is much harder to achieve without A. And this gives you something valuable: rather than guessing whether Piper is approaching an RSI inflection point, you can measure whether she is acquiring one of the capabilities that could plausibly contribute to it. **My recommendation is to begin with Phase 0: determine whether her existing ledgers can reconstruct reliable state–action–next-state transitions.** If they can, we may already have most of the raw material needed for the first prototype.