Preprint · 2026
Flex-π: A Multi-Stream World-Action Model with Compute Flexibility
One checkpoint. Any observed inputs, any generated futures, and a speed–accuracy operating point you pick at deployment.
*Equal contribution. †Equal advising.
Problem: World-action models (WAMs) predict the future to act better, but they mainly predict RGB latents trained for pixel reconstruction — with no explicit signal for the 3D geometry or object semantics that manipulation needs.
Method: We introduce Flex-π, a 6B-parameter WAM that jointly denoises RGB, 3D geometry, and object-centric DINO semantics with actions in a shared latent space. Per-stream dropout with cross-modality forcing yields a single checkpoint that runs on any subset of streams — from fast action-only inference to full joint generation — with no new sensors or visual priors.
Result: A demonstration-efficient policy that beats the strongest baselines by up to 2–7× on dexterous, precise, real-world bimanual tasks in and out of distribution — while running faster than π0.5.
Flex-π in action
Deployed on a bimanual YAM workcell across contact-rich, sub-millimeter and long-horizon tasks, including gripper self-repair.
Task completion on the real robot, averaged over the tasks in each panel. The line is the best baseline; the dark green is Flex-π's margin over it.
View as table
| Task | Flex-π (full joint) | Flex-π (action-only) | ManiFlow | π0.5 | Fast-WAM |
|---|---|---|---|---|---|
| Seen tasks 5 tasks, in distribution | |||||
| Put Plate on the Rack | 95.0 | 84.2 | 75.8 | 72.5 | 12.5 |
| Sort Utensils | 75.0 | 70.0 | 55.0 | 45.0 | 5.0 |
| Kitchen Organization | 98.8 | 96.2 | 93.8 | 73.8 | 77.5 |
| Self-Repair Gripper | 76.0 | 66.9 | 33.3 | 26.2 | — |
| Soft-Bag Zipping | 70.0 | 64.9 | 31.9 | 42.8 | — |
| Average | 83.0 | 76.4 | 58.0 | 52.1 | 31.7 |
| Unseen conditions 3 tasks | |||||
| Put Plate on the Rack | 95.0 | 85.0 | 55.0 | 72.5 | 33.8 |
| Sort Utensils | 70.0 | 70.0 | 32.5 | 40.0 | 0.0 |
| Soft-Bag Zipping | 63.3 | 57.5 | 6.9 | 17.2 | — |
| Average | 76.1 | 70.8 | 31.5 | 43.2 | 16.9 |
| Half the data 1 task | |||||
| Put Plate on the Rack | 95.0 | 80.0 | 60.0 | 42.5 | 25.0 |
Task completion (%). Fast-WAM was not run on Self-Repair Gripper or Soft-Bag Zipping, so its averages cover 3 of 5 seen tasks and 2 of 3 unseen conditions.
Click on any video below to expand it.
Three views of one moment
Given 1 RGB image, Flex-π encodes into latent visual streams for appearance, semantics, and geometry, and can generate all three futures alongside the action. What goes in and what comes out are runtime arguments rather than architecture, so all 56 combinations are deployable from the same weights — try them below. How that is trained is in How it works.
No future visual stream is read, so none is computed. This is the cheapest point on the frontier and recovers VLA-level latency.
What Flex-π does
An eight-stage self-repair, five contact-rich tasks scored against three baselines, and three conditions it was never fine-tuned on.
One row per task, against whichever baseline was strongest on it. The action-only inference mode of Flex-π beats every baseline; joint future generation mode adds additional gains to every task.
Task completion against inference latency. The shaded region is everything both slower and less accurate than the action-only path; hover a point for its numbers.
01Precision and long horizon
The hardest task in the suite: an eight-stage repair whose tightest stage is a sub-millimeter insertion, made with an electric screwdriver held between two moving arms.
Self-repair gripper
The robot repairs its own gripper over eight stages that must be completed in order, split between the two arms, with the tightest insertion near the end.
Where π0.5 and ManiFlow lose the run
Two insertions and the screwing — the three stages they rarely get past. Each is shown twice: the baseline attempting the stage on the left, Flex-π completing it on the right. Stage 7 is the tightest of the eight, a 4 mm bit into a 4.5 mm socket.
STAGE 3Insert gripper — 19 mm part into a 20 mm holder, ±0.5 mm
STAGE 5Insert screw — 4.5 mm M5 screw into an 8 mm hole, ±1.75 mm
STAGE 7Screw in — 4 mm bit into a 4.5 mm socket, ±0.25 mm, the tightest stage
Recovery behavior
What the policy does after a failed attempt, on the two stages with the least clearance to spare.
Retrying the gripper insertion
Retrying the screwing
Eight stages in order, so we score both partial progress and the full sequence. Flex-π (full joint) finishes all eight in 11 of 20 rollouts; the best baseline manages it once.
02Dexterity on deformable objects
Zipping a soft bag: the fabric shifts under every grasp, so the zip never stays in one place. Five bags the policy was trained on, and one it was never shown.
Unzip the bag
Zip the bag closed
On the unseen bag π0.5 falls from 42.8% to 17.2% and ManiFlow from 31.9% to 6.9%. Flex-π (full joint) goes from 70.0% to 63.3%.
03Robustness in cluttered scenes
Deployment scenes are rarely clean. We re-run the suite with novel distractor objects filling the workspace, and with objects swapped for types the policy has never handled. Seen is the setting it was fine-tuned on; Unseen is one it never saw.
Flex-π (full joint) holds 95.0% on Put Plate and 70.0% on Sort Utensils under the unseen conditions, 2.5 and 5.0 points below its in-distribution scores. ManiFlow has 3D inputs and drops 32.5 and 22.5 points.
Training on 50% of the data
Faded bars are the full training set, solid bars half of it. Flex-π with half the data, action only, performs similarly to π0.5 with all the data. With full joint prediction, Flex-π half data outperforms every baseline with full data.
How it works
The shared latent space behind all of it, and the two masks that let one set of weights cover every input/output combination.
One latent space, one backbone
Appearance, geometry and object semantics normally mean three separate pipelines. Flex-π carries all three through a single shared backbone.
Three streams, one shared space
RGB and 3D pointmaps go through the same frozen video-generation VAE. DINOv3 semantics come from a separate frozen encoder and are projected into the same backbone by a linear adapter. Proprioception and the language instruction condition every stream. The diagram at the top of the page runs that path live, with the same two masks.
RGB — appearance
Latent tokens that carry priors from internet-scale video pre-training.
Enc(ot) · frozen Wan-2.2 VAE
DINO — object semantics
Object-level semantic tokens that ground appearance and geometry.
DINO(ot) · frozen DINOv3
Pointmap — 3D geometry
A 3D pointmap pushed through the same frozen VAE, so no geometry-specific encoder is needed.
Enc(pt) · shared VAE weights
Why latents?
Flex-π uses a joint flow-matching prediction objective, but predicts future observations as latents from the pre-trained encoders rather than as pixels, all three streams in one space. Each encoder's priors carry over, the joint representation is stronger, and inference is faster. Actions are generated jointly with those latents under shared self-attention, so the policy reads the predicted future without decoding it.
Cross-modality forcing
Flex-π trains to predict future observations for each modality, even when one stream is missing. This alone raises RoboTwin success by 47% relative.
Robustness to a missing sensor is not the only reason we train this way. Requiring each modality to be predictable from the others keeps the shared backbone from splitting into three weakly-coupled channels, and pushes it toward a representation in which appearance, geometry and semantics are mutually predictive.
The futures behind the actions
Every action chunk is generated jointly with a latent future in each stream. Control never decodes those latents; decoding them afterwards shows what the policy predicted while it was acting.
The decoded future appears under each rollout, in whichever stream you pick.
How it compares to other VLAs and WAMs in simulation
RoboTwin and LIBERO benchmarks — and what the design costs.
Simulation benchmarks
Two suites that measure how well a policy fits its training tasks. Flex-π is fine-tuned separately for each, and within each one the same checkpoint runs either mode.
Flex-π leads both categories on RoboTwin, at 94.6% whether or not it generates futures. On LIBERO everything but π0 lands within two and a half points, so it reads as saturation rather than a ranking; the fixed-mode Flex-π* reaches 99.2%, level with the best published result there. Dashes are numbers nobody has published, not zeros — Flex-π* is reported on LIBERO only.
Conclusion and limitations
Flex-π embeds RGB, 3D and DINO semantics into a shared latent space, yielding a single checkpoint that supports any combination of input and output streams at inference. It matches or beats the strongest VLA in fast action-only mode and the strongest WAM at full joint generation across RoboTwin, LIBERO and a real bimanual YAM robot. Compute flexibility at inference comes from multi-stream latent dropout.
BibTeX
@article{yan2026flexpi,
title = {Flex-$\pi$: A Multi-Stream World-Action Model with Compute Flexibility},
author = {Yan, Ge and Liu, Jinghao and Fan, Yuzhi and Cai, Lei and
Liao, Minwen and Zhang, Jesse and Fox, Dieter},
journal = {arXiv preprint arXiv:2608.10860},
year = {2026}
}