NVIDIA Long-WAM: Robot Success Jumps From 63% to 79% With 19 Seconds of Memory (2610.10528)

Long-WAM: Scaling the Context of World-Action Models (arXiv 2610.10528, submitted 7 October 2026) comes from 16 authors at NVIDIA, MIT, HKU and UCSD, including Linxi Fan, Song Han and Yukang Chen. The headline result: on RoboCasa GR-1, increasing visual context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%. The more interesting claim sits next to it: a bidirectionally pretrained initialization shows no net gain from the same extra history.

All figures below are the authors' claims unless noted. Result figures come from the abstract and introduction of the arXiv HTML version; the method description comes from its method sections.

What problem does Long-WAM solve?

Real-time robot control needs enough visual history to infer motion and task progress, but processing that history can delay the next action. The abstract frames this as the tension Long-WAM addresses: more context should help a policy, yet more context costs latency.

A world-action model (WAM) predicts future video and robot actions together. Earlier systems in this family include DreamZero (arXiv 2602.15922, NVIDIA), a 14B autoregressive video diffusion model that the authors say runs closed-loop control at 7 Hz. Long-WAM keeps the WAM recipe but asks how much history it can carry while staying fast enough for dynamic tasks.

How does Long-WAM work?

Long-WAM is a causal model. It conditions on a window of past observations, predicts future video latents, then denoises an action chunk. According to the method section, five design choices carry the paper.

1. Autoregressive video pretraining. The backbone, called LongLive2.0-Robot, continues from LongLive-2.0's 16-second autoregressive checkpoint. It trains on about 10,000 window-equivalent hours of robot and egocentric video from RoVid-X, AgiBot World, EgoDex, EgoVerse and VITRA, with sequences up to 30 seconds and no action labels.
2. Asymmetric experts. Video and action experts are coupled so that video queries never see action tokens. Action queries see observed history, partially denoised future latents and the noisy action chunk.
3. Predict-then-act (IDM mode). The video expert runs for 4 steps and stops at a noise level of 0.9. Future latents are never decoded to pixels. The resulting video KV cache is reused while actions are denoised.
4. Streaming and asynchronous execution. A streaming causal VAE encodes frames as they arrive, and execution is asynchronous with no blending or prefix guidance.
5. Hardware-specific acceleration. Video linear layers use W4A4 NVFP4, action compute and KV stay in BF16, and the system uses CUDA Graph replay and torch.compile.

The word "bidirectional" needs care. In the paper it refers only to how the baseline initialization (Wan2.2) was pretrained. The adapted policy uses a causal mask in every case, so the comparison isolates the pretraining, not the policy architecture.

What are the results?

The authors report gains on four simulated benchmarks and two real robots.

The introduction also states that the gap to the bidirectional baseline is 3.3 points with no history and 17.1 points at 19.2 seconds. That widening gap is the core evidence for the paper's central finding that longer histories pay off far more when the video foundation is pretrained autoregressively.

On real robots, the abstract reports 95% success on dynamic cup stacking on a Unitree G1, where Pi0.5 and Fast-WAM succeed in none of 20 trials. AI/TLDR puts the Long-WAM count at 19 of 20. The abstract also reports deployment on a YAM arm and describes Long-WAM as a memory-informed executor that complements higher-level planning in composite tasks.

How fast is it?

The authors report 107.4 ms per action chunk on an RTX 5090, including future-video latent prediction. They say streaming encoding, asynchronous execution and hardware acceleration also allow deployment on DGX Spark and Jetson AGX Thor without dropping future prediction.

Two caveats matter for anyone budgeting a control loop. First, that figure is for the optimized configuration with 4-bit video layers, so it is not the speed of a plain BF16 implementation. Second, context is not free: the abstract does not state the context length at which 107.4 ms was measured, and a longer window means more tokens to attend over. Check the paper's latency tables for your target context length before assuming the headline number applies.

Does the evidence hold up?

Partly. The comparison design is stronger than most, but the sample sizes and reproducibility details are thin.

What is convincing. The key control is well chosen. By holding the policy's causal structure fixed and varying only the pretraining of the initialization, the authors separate "has history" from "can use history". A flat curve for the bidirectional model and a rising curve for the autoregressive one is a clean pattern.

What to question.

• One benchmark carries the headline. The 63.3% to 78.7% result is a single simulated benchmark (RoboCasa GR-1). LIBERO-Long and RoboTwin 2.0 results are at or near the top of their scales, where there is little room left to show a history effect.
• 20 real-world trials. On the G1, a single trial is worth 5 points. A 95% result is 19 of 20, and the baselines' 0 of 20 is stark, but the experiment tells us little about variance across objects, lighting or object variation.
• Self-reported. All numbers are the authors' own. The Pi0.5 and Fast-WAM results on dynamic cup stacking are reported in the Long-WAM paper, on a task that demands fast reaction and so favors a low-latency design.
• Different budgets. Long-WAM pretrains on about 10,000 hours of robot and egocentric video. The abstract does not describe the baselines' training data, so some of the gap may come from data and not from memory.
• Contamination and overlap. Pretraining data includes large robot and egocentric collections. The abstract and introduction do not address domain overlap with the benchmarks.

How does it compare with related work?

Long-WAM sits in a fast-moving family of video-based robot policies.

Fast-WAM and Long-WAM ask different questions. Fast-WAM asks whether imagining the future at test time is necessary. Long-WAM asks how much of the past a policy can use. They do conflict on one practical point: Fast-WAM argues the training-time video signal does most of the work, while Long-WAM keeps future latent prediction in the loop. The two papers measure different things on different setups, so this is a design disagreement, not a settled one.

What does a working researcher take from this?

• Pretraining objective can gate whether context helps. If you add history to a policy and see no gain, the cause may be the initialization, not the context length.
• Action-free video is a usable pretraining source. The backbone trains without action labels, which sidesteps the need for a shared action space across embodiments.
• Report latency with context length. A latency number without the window it was measured at is hard to use. Ask for both.
• Treat window length as a hyperparameter. In the paper, each context window is a separately trained model evaluated at its training length, and missing early history is filled by repeating the first frame.
• Check the repo before building on it. AI/TLDR reports that the README marks full benchmark reproduction as not yet verified, so plan time for your own reproduction.

What's still unclear?

• Eval size on GR-1. The task and trial counts behind the 63.3% and 78.7% figures were not stated in the abstract or introduction, so the uncertainty around a 15.4-point gain cannot be judged from those sections alone.
• Where the gain saturates. The paper sweeps window lengths, but each window is a separately trained model, so it does not show whether one model could adapt its window at run time. Adaptive windows are a natural next step.
• Real-world context studies. The real-robot results show that the system works, but the abstract does not report a real-robot sweep over history length, so the sim finding is not yet confirmed on hardware.
• Independent replication. As of 13 October 2026 we found no third-party reproduction. The checkpoint and code release should make one possible.
• Licence. The Apache 2.0 licence is reported by one secondary source. Confirm it in the repository.

Where to find the code

The arXiv HTML version links the code at github.com/NVlabs/LongLive/tree/main/Long-WAM and a project page at nvlabs.github.io/LongLive/Long-WAM. AI/TLDR reports checkpoints on Hugging Face under the Efficient-Large-Model organisation for LIBERO, RoboTwin 2.0, RoboCasa365 and YAM, plus five RoboCasa GR-1 history variants.