Xiaomi Auto World Model - Xiaomi's assisted driving world model - AiBoss

What is the Xiaomi Auto World Model?

Xiaomi Auto World Model is an assisted driving world model launched by Xiaomi Auto, which is the first to deeply couple 3D reconstruction (WorldRec) and video generation (WorldGen). WorldRec uses sparse 3D anchor points to achieve fast reconstruction of 10-second videos in 10 seconds, while WorldGen achieves 4-step denoising and 0.19 seconds/frame generation through two-stage training and ODE distillation, supporting videos up to 1 minute long. It has achieved state-of-the-art (SOTA) results in Waymo reconstruction accuracy (PSNR 28.48) and nuScenes generation quality (FVD 64.97), and has been deployed in three major business scenarios: synthetic data, simulation testing, and assisted driving training.

Main functions of Xiaomi Auto World Model

WorldRec 3D Reconstruction ModuleIt adopts sparse 3D query anchors instead of traditional dense Gaussian to achieve feature aggregation and visibility weighted fusion from multiple perspectives and times. A 10-second video can be reconstructed in just 10 seconds, eliminating multi-view conflicts and ghosting problems.
WorldGen video generation moduleBased on a two-stage training model of bidirectional temporal attention pre-training and causal fine-tuning, the number of denoising steps is compressed from 50 steps to 4 steps through ODE distillation. Future frames, unobserved viewpoints and occluded area content are generated at 0.19 seconds/frame, and the maximum number of frames (about 1 minute) can be generated continuously.
Reconstruction-Generation Deep CouplingThe reconstruction side provides 3D geometric prior constraints to generate stability, while the generation side extends the reconstruction boundary to the unobserved spatiotemporal region. The two sides mutually correct each other to suppress long-term time-series drift.
Extreme scene generationIt supports high-quality synthesis of extreme weather conditions such as heavy rain, heavy snow, and dense fog, as well as long-tailed dangerous scenarios such as animal intrusion, providing scarce data for perception model training.
Three major businesses launchedSynthetic data generation (100,000+ clips delivered), closed-loop simulation testing (reproducing real accidents and optimizing accordingly), and assisted driving training (dynamically generating first-person driving instruction videos).

The technical principles of Xiaomi Auto World Model

WorldRec Sparse Anchor Point RepresentationInstead of using the pixel-by-pixel prediction Gaussian paradigm, sparse 3D query points are used as scene representations. Each anchor point actively aggregates features from multiple cameras and time points, and reliable observations are automatically filtered through visibility weighting, ensuring cross-view consistency from the source.
4D Gaussian Global RepresentationMaintain a 4D Gaussian scene representation that expands with the observation increment, project it onto the vehicle's viewpoint and use it as a rendering prior input to generate the model, providing deterministic geometric constraints for generation.
WorldGen Two-Phase TrainingThe first stage involves pre-training bidirectional temporal attention to establish a global understanding of the driving scenario in time and space. The second stage involves fine-tuning causal attention and teacher forcing, combined with ODE distillation to speed up the process by 12 times, and distribution matching distillation to solve exposure bias.
Reconstruction-Generation of Closed-Loop ConstraintsThe deterministic geometric prior of the reconstruction is continuously corrected and generated to fill in the spatiotemporal regions that are not covered by the reconstruction. The objective functions of the two are structurally mutually constrained to achieve high stability, high consistency and high realism.

How to use Xiaomi Auto World Model

Online platformThe "Assisted Driving Academy - Realistic Simulation Scenarios" has been launched for all Xiaomi vehicle models.
Usage conditionsYou need to own a Xiaomi car (such as the SU7 model) and access the Assisted Driving School module in the in-vehicle system.

Xiaomi Auto World Model's core advantages

SOTA performanceWaymo's reconstruction PSNR was 28.48 (about 1 point higher than DGGT), and nuScenes' zero-shot generalization PSNR of 26.54 also led; the generative FVD of 64.97 surpassed all bidirectional and autoregressive baseline models.
Speed ReasoningThe single-view generation time is 0.19 seconds/frame, and the three-view generation time is 0.46 seconds/frame, which is 5.6 times faster than the similar autoregressive method Epona (1.06 seconds/frame).
Ultra-long timingIt supports continuous generation of 81 frames (10Hz/30Hz, up to 1 minute), far exceeding the 8-16 frame limit of the public baseline model.
Zero-shot generalizationnuScenes continues to lead in zero-shot testing, demonstrating its strong adaptability to new scenarios.
Production has commenced.It has completed the business loop in three core scenarios: synthetic data, simulation testing, and smart cockpit for Xiaomi cars.

Xiaomi Auto World Model project address

Project official websitehttps://JointWM.github.io/
arXiv technical paper: https://arxiv.org/pdf/2605.18137

Xiaomi Auto World Model Comparison with Similar Products

Comparison Dimensions	Xiaomi Auto World Model	Waymo World Model
Company	Xiaomi Auto	Waymo (Alphabet/Google)
Release time	May 2026	February 2026
technical route	Reconstruction + Generation Deeply Coupled Integrated Architecture(WorldRec + WorldGen are mutually constrained)	Generative World Model Based on Genie 3(Purely generated routes, then trained to adapt to driving scenarios)
Reconstruction Module	WorldRec: Sparse 3D anchor point representation, 10-second video reconstruction in 10 seconds, PSNR 28.48 (Waymo dataset)	There is no independent reconstruction module; the entire scene is generated by relying on Genie 3's pre-trained world knowledge.
Generate module	WorldGen: 4-step noise reduction, 0.19 seconds/frame, supports continuous generation of 81 frames (~1 minute).	Genie 3-based generation, supports multi-sensor output (camera + LiDAR), and can simulate extreme scenarios.
Architectural features	Reconstruction lays the foundation for generation (geometric constraints), while generation expands the boundaries of reconstruction (completing unobserved areas).	Purely generative, adjusting the simulation through three control mechanisms: language, action, and scene layout.
Sensor support	Primarily designed for camera data (multi-view image input).	Camera + LiDAR multi-sensor outputIt can convert ordinary dashcam video into multi-sensor analog data.
Benchmarking	Waymo PSNR 28.48 (about 1 point higher than DGGT); nuScenes FVD 64.97, FID 7.04	Specific quantitative metrics were not disclosed, but the emphasis was placed on its ability to simulate "never-before-seen" long-tail scenarios.
Generation speed	Single-view: 0.19 seconds/frame; Three-view: 0.46 seconds/frame	The specific inference speed was not disclosed, but the emphasis was placed on "scalable inference" and efficient variants.
Maximum generation time	81 frames (10Hz/30Hz, up to approximately 1 minute).	It has not been explicitly stated, but Genie 3 natively supports consistent generation at the minute level.
Extreme scenario capabilities	Generation of long-tail scenes such as heavy rain, heavy snow, dense fog, and animals intruding	Tornadoes, floods, snow-covered Golden Gate Bridge, rare objects such as elephants/lions, reckless drivers, etc.
Business Implementation	Three major scenarios have been implemented.Synthetic data (100,000+ clips), simulation testing, and driver assistance training.	Used for Waymo Driver training and validation, supporting the expansion of robotaxi (target of 1 million weekly orders by 2026).

Application scenarios of Xiaomi Auto World Model

Synthetic data generationThis provides high-quality, diverse, and high-risk long-tail scene synthetic data for training assisted driving perception models, addressing the problem of insufficient real-world scene samples.
Simulation test: Construct a closed-loop simulation environment to reproduce real accident scenarios for targeted optimization, improve testing efficiency and completeness, and reduce the cost and risk of real vehicle testing.
Assisted Driving SchoolThe system dynamically generates first-person driving instruction videos in the intelligent cockpit, demonstrating the correct operation to users in complex road conditions, thereby improving the safety and user experience of human-machine co-driving.