01Experience
A History A
B History B
Extract information.
Encode observations and actions from independent experiences.
SPRII / From experience to task value
Learning what persists.
From independent experiences to persistent information—and the conditions that turn it into better predictions.
FFormation
zA
Related factors≈distinct codes
zB
What is retained?
Relations shape which persistent factors become organized and accessible.
UUse
CorrectNullWrong
↓
Fixed predictor → Prediction
Is it actually used?
Substitute context while keeping the prediction route and query fixed.
VValue
SpringWorld · sealed MSE ↓
When does it help?
Measure task gains, then examine the horizon, target and consuming interface.
Three measurements: what is retained, what is used, and what improves prediction.
SpringWorld values: sealed prediction MSE, reported below.
01 / Environments
Many worlds. One question.
Choose an environment to watch its dynamics and explore the experiment.
01 / Predictive tasks
02 / Mechanism studies
03 / Beyond one setting
01 / 13 · Formation · Use · Value
Explore experiment SpringWorld
Motion preview
Different motion. The same physical system.
A spring system changes its trajectory while mass, drag and stiffness persist. Independent histories help predict a new encounter.
22.3%lower MSE than native JEPA256 sealed systems · paired 95% CI: 13.2–30.7%
- Persists
- Mass m, drag γ and stiffness k.
- Changes
- Independent interaction history: initial state and actions.
02 / 13 · Generality
Explore experiment CoPhy · Collision
Motion preview
Experience before the next collision.
Related support interactions provide context for predicting how objects move after contact, across two different base learners.
CPC + RSSMheld-out prediction gains1,994 episodes · adapted support/query task
- Persists
- Matched objects under the scene-specific relation.
- Changes
- Independent support interactions.
03 / 13 · Use
Explore experiment NOD · Burgers
Motion preview
A changing field. A shared viscosity.
Independent initial fields evolve under the same viscosity. The comparison also separates relational training from extra prediction updates.
3 distributionsID, viscous OOD, inviscid OODMatched-update NOD remains slightly better on OOD
- Persists
- Viscosity ν.
- Changes
- Independent initial fields.
04 / 13 · Formation · Use · Value
Explore experiment PokeWorld
Motion preview
What you call “shared” changes what is learned.
Change the relation between interactions and the representation shifts its emphasis between mass, drag and stiffness.
Formation → Usedoes not guarantee more task valueControlled relation semantics and reliability studies
- Persists
- G1: mass m; G2: (m, γ); G3: (m, γ, k).
- Changes
- The shared-factor rule; separately, correct-pair fraction α at fixed history budget.
05 / 13 · Value
Explore experiment D-Clean
Motion preview
Knowing the parameter is only part of the story.
Drag governs a simple dynamical system. Direct context and decoded physical parameters offer different ways to consume the same representation.
R² > .996still does not identify the best readerDirect context outperforms the decoded bottleneck at h32
- Persists
- Drag γ; mass is fixed at 1.
- Changes
- Initial state and piecewise force sequence.
06 / 13 · Use · Generality
Explore experiment CoPhy · Balls
Motion preview
Shared objects, different trajectories.
Ball interactions connect physical-information probes in actual reader memory with tests of whether donor context changes prediction.
Probe + interventionaccessibility and functional useDifferent measurements answer different questions
- Persists
- Matched objects under the scene-specific relation.
- Changes
- Support interactions and initial motion.
07 / 13 · Generality
Explore experiment CoPhy · Blocktower
Motion preview
From a single collision to a falling tower.
Multibody interactions test how the same training principle behaves across scene geometry and different prediction models.
4 learner familiesone multibody settingBenefits vary across model and reader choices
- Persists
- Matched objects under the scene-specific relation.
- Changes
- Independent support interactions.
08 / 13 · Generality · Formation
Explore experiment NOD · FHN
Motion preview
Persistent parameters in a spatial world.
Reaction parameters shape two evolving fields. Multiple histories provide another way to aggregate evidence about the system.
2D fieldsprediction and parameter accessibilityReproducible simulations with regenerated initial conditions
- Persists
- Reaction parameters (k, β).
- Changes
- Independent initial two-dimensional fields.
09 / 13 · Formation
Explore experiment Baxter
Motion preview
A grasp can reveal hardness—or shape.
Real tactile recordings test whether changing the shared relation changes which material or shape information becomes accessible.
Real tactile datarelation-dependent readoutHardness-sharing and shape-sharing are separate conditions
- Persists
- Hardness under one relation, or shape under the other.
- Changes
- The other factor, with a separate grasp interaction.
10 / 13 · Use · Generality
Explore experiment RH20T
Motion preview
A past robot episode provides task context.
Recorded multimodal interactions test how task-related context enters force, torque and motion forecasts.
Offline forecastswith donor-specificity controlsEvidence concerns recorded episodes, not closed-loop control
- Persists
- Task identity.
- Changes
- Different recorded episodes.
11 / 13 · Use
Explore experiment Swimmer
Motion preview
Does the update belong to this system?
Swap own-system and wrong-system persistent updates while the predictor stays fixed, then measure how the response changes.
Fixed predictorrecipient-specific update testsA test of functional use through frozen computation
- Persists
- The recipient system.
- Changes
- Own-system versus wrong-system persistent update.
12 / 13 · Generality · Formation
Explore experiment Overcooked
Motion preview
What persists can be a partner.
Across distinct cooperative episodes, a fixed partner identity supplies a relation for learning behavioral context.
Partner identityacross disjoint episodesBehavioral readout and cooperation · held-out returns remain unstable
- Persists
- The fixed neural partner identity.
- Changes
- Disjoint episodes, positions, actions and outcomes.
13 / 13 · Value · Use · Generality
Explore experiment Pendulum
Motion preview
From dynamics context to online control.
Transition histories provide context as a pendulum controller faces systems with different masses and lengths. One ID-selected recipe is held fixed across all groups.
4 / 5 groupshigher mean return than CaDM3 training seeds · lower mean return in OOD c2
- Persists
- Mass and length within the same system.
- Changes
- Episode histories; mass and length across systems.
02 / The methodA principle that fits
A principle that fits
different prediction models.
SPRII trains across related experiences while preserving the base learner’s objective. The persistent information enters a learner-specific prediction route.
JEPA
An encoder maps observations into embeddings. A predictor uses the current context and actions to predict a future embedding.
ObservationEncoderPredictorFuture embedding
+ Persistent context
Persistent context conditions the native prediction route. Align and Cross can be studied separately.
SPRII keeps the native prediction objective.
RSSM
A recurrent state-space model combines recurrent memory with stochastic latent variables to model dynamics over time.
Observation + actionLatent stateDynamicsFuture state
+ Persistent context
Related support experience supplies persistent context to the adapted prediction route.
The evaluated RSSM retains its variational dynamics objective.
CPC
A context representation is trained to distinguish the future representation from contrasting alternatives.
HistoryContextPredictionContrastive target
+ Persistent context
Cross-interaction context enters the learner-specific route in the CoPhy experiments.
The contrastive objective remains part of the base learner.
Structure → Align → CrossThree parts.
Three parts.
Different jobs.
The interface gives context a route. Relations shape its content and train the predictor to use it.
01 / 03Structural inductive bias
P/T biasGive history and the present
Give history and the present
distinct routes.
Separate history-derived persistent context from the recipient’s current state. Prediction can use both.
The persistent–current interface combines history-derived context with the current input.
History→zpPersistent context
Current input→zsCurrent context
Both codes + permitted actions ↓Prediction
A route for shared informationNext: shape the history codes ↓
02 / 03Relation objective
Align
Make related context agree.
Bring related history codes together while preserving variation across the code population.
Relations identify shared information; code dispersion retains differences across systems.
zpA
↔Related histories
zpB
Consistency + code dispersionNext: use context across interactions ↓
03 / 03Cross-interaction prediction
CrossUse one experience
Use one experience
in another.
Use a related donor’s persistent code in the recipient prediction, keeping the recipient’s current input, permitted actions and target.
The donor context is substituted into the prediction route. The learner retains its native objective.
Donor A zpA
↓Recipient B zsB + actions
↓Prediction ŷB
Shared context, recipient targetContinue to the evidence ↓
03 / Core evidenceWhat forms. What is used.
What forms. What is used.
What the task gains.
Read the mechanism and the predictive results together. Each comparison retains its own task, controls and evaluation population.
The relation selects
the information.
Adding drag to the shared relation increases drag accessibility while reducing mass accessibility.
G1 · shared massG2 · shared mass + drag
log mass R²
.5271
.0709
Drag R²
.3075
.9249
0Linear probe R² · common scale1
Frozen-code probes on validation systems. Accessibility is a measurement of retained information, not a task-gain guarantee.
Matching donor physics
changes prediction.
With the prediction route fixed, matching donor and target physics produces lower error than mismatching them.
Mass-matching advantage0.1820595% CI [0.13200, 0.23429]
Mismatched donor error−Matched donor error> 0
Align + Cross · h16 MSE · 64 development systems · 3 frozen sources, reader 0. Off-diagonal minus diagonal error; interval resamples systems conditional on fitted models.
Value / Principal predictive comparisons
Does it improve prediction?
All errors below are lower-is-better. Scales differ across tasks.
SpringWorld
Standardized state-increment MSE · lower is betterSealed new-system prediction
256 sealed systems · 3 sources × 3 readers
| Source recipe | Mean ± source SD | SPRII reduction [paired 95% CI] |
|---|---|---|
| Native JEPA | .138 ± .002 | 22.3% [13.2, 30.7] |
| TDS | .130 ± .013 | 17.5% [8.4, 25.9] |
| SPRII · Align + Cross | .107 ± .004 | — |
Bold marks SPRII’s primary mean and the two paired error reductions.
- SPRII · Align + CrossThe absolute means and paired reductions use the same sealed population; uncertainty has the definitions shown above.
Source SD is computed after averaging readers. Percentage intervals resample paired systems.
CoPhy · Collision
Physical trajectory MSE · lower is betterHeld-out prediction across learners
1,994 test episodes · 3 fitted source/reader seeds per learner and method
| Learner | Native | Structure | Random | Cross |
|---|---|---|---|---|
| CPC | .248 ± .008 | .223 ± .006 | .221 ± .006 | .189 ± .007 |
| RSSM | .268 ± .007 | .228 ± .005 | .222 ± .007 | .202 ± .007 |
Bold marks the Cross result within each separately evaluated learner family.
- CPCCompare methods within a learner row; this is the adapted S3/query3 held-out task.
Means ± SD across jointly fitted source/reader seeds. S3 support histories and query3 access are shared.
NOD · Burgers
Native trajectory MSE × 10⁻³ · lower is betterPublic operator prediction
3 source seeds · one fixed continuation recipe across distributions
| Method | ID | Viscous OOD | Inviscid OOD |
|---|---|---|---|
| Released-default NOD | 9.096 ± .544 | 8.012 ± .815 | 11.857 ± .628 |
| SPRII continuation | 8.999 ± .798 | 7.487 ± .183 | 11.510 ± .360 |
Bold identifies the fixed SPRII endpoint in the released-default comparison.
- SPRII continuationThe endpoint includes extra prediction updates. Continued NOD has slightly lower OOD means under the matched-update comparison.
Matched extra-update control: continued NOD achieves 7.408 / 11.488 on the two OOD distributions, slightly below SPRII’s 7.487 / 11.510. The released-default comparison alone does not isolate relational training.
Formation asks what the representation contains.
Use asks whether the predictor relies on it.
Value asks what the task actually gains.
04 / What we learnedTurn mechanism
Turn mechanism
into better decisions.
Training shapes what context contains. The task and the way context is consumed determine how much value it delivers.
A concrete example · SpringWorldKeep the representation.
Keep the representation.
Change how it is used.
A persistent-conditioned residual reader recovers additional value from the same frozen source.
7.9% lower development MSEQuery-only residual · M₁0.618548
Conditioned residual · M₂0.569539
3 sources × 3 readers · development data
Active capacity is not exactly matched. The same reader does not improve the tested PokeWorld families.
Measured mechanism · design implication
Choose the relation for the information you need.
In PokeWorld, changing a mass-sharing relation to mass-and-drag sharing raises drag accessibility while reducing mass accessibility.
In practice
Inspect each task-relevant factor. A more specific relation need not preserve everything the earlier relation captured.
The change in accessibility is measured. Task-aware relation selection is a promising strategy, not a demonstrated universal gain.Diagnostic practice
Check that the right context actually matters.
A good probe or low prediction error alone cannot establish that a predictor uses the correct persistent information.
In practice
Pair factor probes with Correct / Null / Wrong context interventions on the actual downstream prediction route.
Hold the predictor, current query, actions and target fixed. A zero-code ablation alone is not a factor-specific test.Observed recovery · conditional
When value stalls, examine the reader.
With the source frozen, a persistent-conditioned residual reader reduces SpringWorld development MSE by 7.9%. The same design does not recover value in PokeWorld.
In practice
Compare direct context, decoded factors and a conditioned residual reader before deciding the representation needs to be retrained.
Development evidence; active capacity is not perfectly matched. The best interface depends on the environment.Task-dependent component evidence
Treat Align and Cross as separate choices.
Align carries much of the component gain in SpringWorld; Cross improves the tested CPC and RSSM prediction routes in CoPhy Collision.
In practice
Keep the native objective, compare the components, and select on the task you want to improve.
The combined objective is not established as the best recipe for every learner and task.Measured pattern · candidate intervention
Measure where persistent information should help.
SpringWorld history gains grow with horizon. In PokeWorld, improvements in object prediction can be offset by another output component.
In practice
Report horizon-specific and component-specific errors alongside the aggregate task score.
Longer-horizon training or reweighting outputs are candidate interventions; these analyses do not demonstrate their benefit.