SkyReels-V2 - Kunlun Wanwei's open-source infinite-duration movie generation model

What is SkyReels-V2?

SkyReels-V2 is an unlimited-duration movie generation model developed by the Kunlun Tech SkyReels team. Based on the diffusion-forcing framework, it combines multimodal large language modeling (MLLM), multi-stage pre-training, and reinforcement learning techniques to generate high-quality, unlimited-duration video content. SkyReels-V2 addresses the challenges of existing technologies in terms of cue word adherence, visual quality, motion dynamics, and video duration coordination. It supports various application scenarios such as story generation, image-to-video compositing, camera directing functions, and multi-agent consistency video generation. The model and related code are open-source, providing a powerful tool for creative content creation and virtual simulation.

Main features of SkyReels-V2

• Unlimited duration video generationIt supports the generation of theoretically unlimited video content, breaking through the time limit of traditional video generation models.
• Story generation: Arrange complex multi-action sequences based on narrative text prompts to achieve dynamic narration.
• Image-to-video synthesisIt provides two methods, including fine-tuning the full-sequence text-to-video diffusion model (SkyReels-V2-I2V) and combining the diffusion forcing model with frame conditions (SkyReels-V2-DF), to transform still images into coherent videos.
• Camera Director FunctionIt supports the generation of smooth and diverse camera motion effects, enhancing the cinematic feel of videos.
• Element to video generationIt combines any visual elements (such as people, objects, and backgrounds) into a coherent video guided by text prompts, suitable for applications such as short dramas, music videos, and virtual e-commerce content creation.

The technical principles of SkyReels-V2

• Multimodal Large Language Model (MLLM)The system generates an initial description of the video based on a multimodal large language model, and provides a more detailed description of the camera language by combining sub-expert models (such as shot type, shot angle, shot position, facial expressions, and camera movement). Based on human annotation and model training, it further enhances the understanding of camera language and significantly improves the ability of the generated video to follow cue words.
• Multi-stage pre-training:
  • Progressive resolution pre-trainingThe model's generation capability is gradually enhanced by progressively increasing the resolution from low (256p) to high (720p).
  • Multi-stage post-training optimizationThis includes supervised fine-tuning (SFT) for initial concept balancing, motion-specific reinforcement learning (RL) training, diffusion-forced frame (DF) training, and high-quality SFT to ensure the model achieves optimal performance in various aspects.
• Reinforcement Learning (RL)This study utilizes reinforcement learning to optimize motion quality, addressing the shortcomings of existing models in terms of motion dynamics, smoothness, and physical plausibility. It employs a semi-automatic data collection pipeline to generate preference comparison data pairs, trains a reward model, and performs Direct Preference Optimization (DPO) to improve motion quality.
• Diffusion Forcing FrameworkAssigning an independent noise level to each frame enables unlimited scalability of video generation. Using a non-decreasing noise timetable, the search space for the denoising timetable of consecutive frames is reduced from O(1 e 48) to O(1 e 32), significantly improving generation efficiency.
• Efficient data processing and optimizationThis approach integrates general datasets, self-collected media, and art resource libraries, employing multi-stage filtering and annotation to ensure the quality of training data. FP8 quantization, multi-GPU parallelism, and model distillation techniques significantly reduce inference time and computational cost, improving the model's practicality.

SkyReels-V2 project address

• GitHub repository:https://github.com/SkyworkAI/SkyReels-V2
• HuggingFace model library:https://huggingface.co/collections/Skywork/skyreels-v2
• arXiv technical paper:https://arxiv.org/pdf/2504.13074

Application scenarios of SkyReels-V2

• FilmmakingGenerate continuous videos of unlimited length for complex narratives and long takes.
• Advertising creationTransform static images into dynamic videos to enhance the appeal and expressiveness of advertisements.
• Video shooting assistanceGenerates smooth camera movement effects, helping to design and implement complex shooting shots.
• Short dramas and music videosQuickly generate high-quality videos, reducing shooting costs and time.
• Virtual Reality and Game DevelopmentGenerate realistic virtual scenes and character animations to enhance user experience and immersion.