MirageLSD - A real-time video generation model from Decart AI
MirageLSD, developed by the Decart AI team, is the world's first Live-Stream Diffusion AI video model. It enables real-time video generation of unlimited duration with latency as low as 40 milliseconds and supports...
What is MirageLSD?
MirageLSD, developed by the Decart AI team, is the world's first Live-Stream Diffusion AI video model. It enables real-time video generation of unlimited duration with latency as low as 40 milliseconds and supports smooth output at 24 frames per second. Through Diffusion Forcing and historical augmentation training, it solves the error accumulation problem of traditional autoregressive models during long-term generation, achieving unlimited video generation. Based on Hopper-optimized Mega Kernels, architecture-aware pruning, and Shortcut Distillation, MirageLSD significantly improves generation speed while maintaining high image quality, enabling true real-time interaction.
MirageLSD's main functions
-
Unlimited duration real-time video generationMirageLSD can generate video streams of unlimited length with latency as low as 40 milliseconds and supports a real-time generation speed of 24 frames per second, solving the error accumulation problem of traditional video generation models during long-term generation.
-
Real-time interactivityUsers can receive prompts, make transitions, and edit videos in real time during the video generation process, enabling a continuous interactive experience.
-
Low latency processingThe model achieves ultra-low latency processing of 40 milliseconds through optimization techniques such as Hopper-optimized Mega Kernels and architecture-aware pruning, enabling real-time video generation.
-
Wide range of application scenariosIt can be used in multiple fields such as live streaming, video calls, game development, and animation production, and can transform ordinary scenes into user-specified styles or scenes in real time.
MirageLSD Technical Principles
-
Diffusion Forcing TechnologyBy denoising frame by frame, the model is allowed to generate single-frame images without a complete video context, thus achieving frame-level generation.
-
History Enhancement TrainingNoise data from historical frames is introduced during training, enabling the model to predict and correct errors in the input, thus achieving infinite generation.
-
Optimization strategy:
-
Hopper-optimized Mega KernelsOptimized for the NVIDIA Hopper GPU architecture, reducing latency per layer of the model.
-
Architecture-aware pruningBy adjusting the model parameter size to fit the GPU architecture, the computational load can be reduced.
-
Shortcut DistillationBy training a smaller model to match the denoising trajectory of a larger model, the diffusion steps required for generation are reduced.
-
MirageLSD project address
- Technical Papershttps://about.decart.ai/publications/mirage
Application scenarios of MirageLSD
-
Live streaming and video callsIt can transform ordinary video calls or live streams into user-specified scenes in real time, such as turning real-world scenes into science fiction worlds.
-
Game developmentIt can transform game visuals into different styles in real time, such as turning ordinary battle scenes into lightsaber duels.
-
Animation production and virtual dress-upProvides real-time visual effects support for animation production and virtual dress-up.