02 / Andromeda · Research note
Long-WAM vs. VLA: what should a foundation learn?
The important choice begins before a robot learns a particular job. We examine Long-WAM, strong VLA methods, and the evidence for learning physical dynamics during pre-training.
Literature analysis · Orion research perspective
Based on the three cited papers; proposed experiments are identified below.
01 / Start with the physical problem
Two identical images can require different actions. A cup may be moving toward a gripper or away from it; the current frame alone may not distinguish the two. A useful physical foundation needs representations that preserve what matters for the next decision.
For Orion, this begins with a research question: what should pre-training teach before a national team adds task-specific behavior? Our working priorities are motion, interaction, useful memory, and transfer. A model family is a way to investigate those priorities, not the objective itself.
02 / Understand the distinction
Vision-language-action models connect visual and language context to actions. World-action models incorporate prediction of future world states into action generation. The boundary is porous: VLA training can include physical dynamics, and a WAM can predict latent representations rather than render complete video. [1] [2] [3]
| Approach | Pre-training question |
|---|---|
| VLA | What representation best supports action learning? |
| Long-WAM | How can learning to predict from history make memory useful? |
| VLA-Δ | Can explicit physical changes improve the foundation? |
Research questions, not a universal performance ranking.
03 / What the three papers establish
Long-WAM: memory depends on pre-training
Huang and colleagues pre-train an autoregressive video foundation, then retain its temporal structure during world-action adaptation. Their experiments find that longer history is more useful with this initialization than with the compared bidirectional foundation. Gains depend on the task and context length; larger windows also cost latency. “Causal” here describes temporal information access, not proof that the model has learned causal reasoning. [3]
VLA-Δ: dynamics can be a training target
Lu and colleagues supervise physical transitions through structured state changes, optical flow, and descriptions of future-state differences. These auxiliary signals are used during pre-training. At deployment, the policy produces actions without decoding those targets. This is a reason to test stronger VLA objectives alongside WAMs; it is not a direct comparison with Long-WAM. [2]
Robustness: the model family is not the whole explanation
Zhang and colleagues find strong WAM performance on several visual perturbations, while camera and initial robot configuration changes remain difficult. Some VLA configurations are competitive. The compared systems have different training histories and inference settings, so the study does not isolate architecture as the cause of every difference. [1]
These are findings reported by the cited authors. Orion has not independently reproduced them. This note is a literature analysis, not an announcement of an Orion model or benchmark result.
04 / Our interpretation
Physical dynamics belong near the beginning of the research program. We want to investigate whether a foundation can learn a reusable account of change before being specialized for a particular robot or national application.
That makes Long-WAM an important direction to study, and VLA-Δ an important comparison. We would choose between them using controlled evidence about transfer, training cost, adaptation effort, and the resources required by the receiving team. We would not select a foundation by placing scores from different benchmarks in a single league table.
A further distinction matters for our scope: evaluating a foundation after adaptation is not the same as selling a deployment stack. Orion’s responsibility is pre-training. The national team completes subsequent development.
05 / The experiments we propose
- Choose one physical capability. Define a bounded task family, the available observations, and success criteria. Reserve genuinely unfamiliar settings before training begins.
- Compare objectives before scaling. Start with a strong action-oriented baseline. Evaluate temporal prediction and explicit transition supervision as additions or alternatives. Keep the input data and resource budget comparable where possible.
- Test whether history is used. Compare real history with shuffled, truncated, or irrelevant history. A gain should reflect information the model learned to use, rather than only additional tokens or capacity.
- Measure transfer under a fixed adaptation budget. Use the same downstream examples and evaluation protocol. Report uncertainty, failures, and the full pre-training bill, including inherited checkpoints and their dependencies.
- Validate independent continuation. Ask a separate team to load the checkpoint, reproduce the evaluation, and continue development from the documentation in an environment it controls.
Proposed Orion research agenda. Experiments and results are forthcoming.
06 / Why this matters for sovereignty
Our intended customer receives the foundation rather than permanent dependence on a hosted model. That requires more than a checkpoint: development rights, an intelligible training recipe, provenance, evaluations, and a clear account of external dependencies.
National teams should be able to make the next research decision themselves. The foundation should give them a technically credible place to start and enough knowledge to keep going. Read our mission →
Sources / exact versions reviewed
- Zhanguang Zhang et al. Do World Action Models Generalize Better than VLAs? A Robustness Study.arXiv:2603.22078v1 · March 23, 2026 · Read HTML ↗
- Qi Lu et al. VLA-Δ: Learning Physical State Transition for Vision-Language-Action Models.arXiv:2610.21306v1 · October 15, 2026 · Read HTML ↗
- Wei Huang et al. Long-WAM: Scaling the Context of World-Action Models.arXiv:2610.10528v1 · October 7, 2026 · Read HTML ↗
Our priority is the quality of the foundation—not allegiance to a model family.