MindVLA-o1 - Li Auto's next-generation autonomous driving foundation model

What is MindVLA-o1?

MindVLA-o1 is Li Auto's next-generation autonomous driving foundation model, employing a native multimodal MoE architecture that unifies and integrates visual, linguistic, and behavioral modalities. The model achieves spatial understanding through a 3D ViT encoder, predicts the future using an implicit world model, and outputs driving trajectories through a unified behavior generation mechanism. Combining closed-loop reinforcement learning with hardware and software co-design, MindVLA-o1 can see further, think deeper, and drive more steadily, marking a crucial step in the evolution of autonomous driving towards a general embodied intelligent agent.

Main functions of MindVLA-o1

• 3D spatial perceptionMindVLA-o1 achieves 3D spatial perception through a 3D ViT encoder and a feedforward 3DGS representation, accurately understanding the static environment and dynamic objects in a scene.
• Multimodal thinking and reasoningThe model introduces a predictive latent world model to extrapolate the evolution of future scenarios in the latent space, achieving a deep integration of visual understanding and language reasoning.
• Unified behavior generationThe system adopts a VLA-MoE architecture and a parallel decoding mechanism to generate high-precision driving trajectories that conform to dynamic constraints and meet real-time requirements.
• Closed-loop self-evolutionBased on the Feed-forward scene reconstruction and reinforcement learning framework, the model continuously evolves itself in the simulation environment, breaking through the limitations of real data scale.
• High-efficiency edge deploymentThrough optimization using the principles of hardware and software co-design, the system achieves efficient deployment on the vehicle-mounted edge chip, balancing model accuracy and inference efficiency.

The technical principle of MindVLA-o1

• 3D Self-Supervised Visual CodingThe model uses a vision-centric 3D ViT encoder that uses LiDAR point clouds as geometric cues and introduces feedforward 3DGS representation to model static environments and dynamic objects respectively. Self-supervised training is achieved through the next frame prediction task, enabling the model to have both semantic understanding and 3D perception capabilities.
• Predictive hidden world modelTo avoid the high computational cost of directly generating future images, the model makes efficient predictions in a compact latent space. Through three stages of training, it constructs the latent space representation and inference capabilities of future scenes, achieving a unity of understanding the present, imagining the future, and making logical judgments.
• Unified behavior generationIn the VLA-MoE architecture, the Action Expert is specifically designed to handle driving trajectory generation. It uses parallel decoding to output all trajectory points at once to meet real-time requirements. It also uses discrete diffusion for multiple rounds of iterative optimization to ensure that the trajectory space is continuous and meets dynamic constraints.
• Closed-loop reinforcement learningThe traditional incremental optimization reconstruction is upgraded to feed-forward scene reconstruction, combined with generative models to expand simulation capabilities, and relies on a unified 3DGS rendering engine and a distributed training framework to achieve a low-cost and high-efficiency reinforcement learning closed loop.
• Hardware and software co-designBased on the Roofline model to characterize hardware constraints, we evaluated nearly 2,000 architecture configurations to find the Pareto optimal solution for accuracy and latency. We found that a wider and shallower model architecture is more efficient in edge scenarios, shortening the architecture exploration cycle from months to days.

Key information and usage requirements for MindVLA-o1

• positionThe next-generation autonomous driving foundation model of Li Auto is a native multimodal VLA architecture oriented towards embodied intelligence.
• Release timeOn March 17, 2026, it was officially released by Zhan Kun, the person in charge of the pedestal model, at NVIDIA GTC 2026.
• Five major technological innovations3D spatial understanding, multimodal thinking, unified behavior generation, closed-loop reinforcement learning, and hardware/software co-design.
• Technological EvolutionFrom end-to-end to VLA and then to native multimodal, it represents the beginning of the era of physical AI.
• Application extensionThe same VLA model can control vehicles and robots; autonomous driving is just the starting point of physical AI.
• Data levelIt relies on MindData's unified VLA data engine to continuously collect, clean, and automatically label large-scale driving data.
• computing power levelIt requires the use of MindSim's controllable multimodal world model and RL Infra reinforcement learning infrastructure to support large-scale closed-loop training.
• Hardware levelDeployment based on NVIDIA Drive Orin or Thor platform requires Pareto's optimal configuration to balance model accuracy and inference latency.
• Simulation levelIt leverages a unified 3DGS rendering engine and a distributed training framework to achieve low-cost and high-efficiency reinforcement learning iteration.

MindVLA-o1's core advantages

• Native multimodal unified architectureMindVLA-o1 integrates visual, language, and behavioral modalities into the same framework for joint training and alignment, without post-processing, achieving higher efficiency and better generalization ability.
• 3D spatial depth understandingThrough a 3D ViT encoder and a feedforward 3DGS representation, the model possesses both semantic understanding and 3D perception capabilities, overcoming the limitations of traditional BEV flat scene shooting and overly dense OCC.
• Efficient deduction of hidden spacePredictive latent world models "imagine" the future in a compact latent space, avoiding the high computational cost of directly generating images, and achieving a unity of understanding the present and predicting the future.
• Real-time accurate decision makingThe VLA-MoE architecture combines Action Expert, parallel decoding, and discrete diffusion optimization to balance trajectory generation accuracy and real-time requirements.
• Efficient deployment on the edgeThe hardware-software co-design principle shortens the architecture exploration cycle from months to days, finding the optimal balance between accuracy and latency in automotive chips.

Comparison of MindVLA-o1 with similar competing products

Comparison Dimensions	MindVLA-o1	Tesla FSD	Huawei ADS
Architecture Roadmap	Native multimodal VLA unified architecture	End-to-end pure vision	End-to-end + multi-sensor fusion
Perception Scheme	Visual-centric with LiDAR geometric cues	Pure visual	Multi-sensor fusion
reasoning ability	Hidden World Model Predicts the Future	End-to-end implicit reasoning	Rule + AI Hybrid
Behavior generation	MoE+ Parallel Decoding+ Discrete Diffusion	End-to-end direct output	Segmented decision-making
Simulation training	Feed-forward reconstruction + reinforcement learning	Shadow mode + simulation	Data closed loop as the main focus
Deployment optimization	Hardware-Software Co-design Laws	Self-developed chip Dojo/HW4.0	Ascend chip optimization
Application extension	VLA for Vehicles and Robots	Focus on autonomous driving	Focus on autonomous driving
Technical stage	Physical AI/Embodied Intelligence	AI-based end-to-end	AI-based end-to-end

Application scenarios of MindVLA-o1

• autonomous drivingMindVLA-o1, as the next-generation autonomous driving foundation model, can handle driving tasks in all scenarios, including urban roads, highways, and complex intersections, achieving end-to-end intelligence from perception and understanding to decision-making and planning.
• Intelligent cockpit interactionLeveraging the language understanding capabilities of its native multimodal architecture, the system can comprehend passenger voice commands and, combined with visual perception, achieve natural human-computer interaction and proactive services.
• Robot controlThe same VLA model can be extended to a robotic platform to drive different forms of embodied intelligent agents, such as robotic arms and wheeled robots, to complete physical world tasks.
• Simulation test verificationIt generates high-fidelity virtual scenes using the MindSim world model, supporting large-scale closed-loop testing and model iteration for long-tail scenarios such as extreme weather and rare accidents.
• Intelligent traffic managementBased on 3D spatial understanding and prediction capabilities, it can be extended to city-level intelligent transportation systems such as vehicle-road cooperation and traffic flow prediction.