AB
AB
AiBoss

All Categories

project

LONGLIVE - An interactive long-form video generation framework developed by NVIDIA and others.

LONGLIVE is a real-time interactive long-form video generation framework jointly developed by NVIDIA and other leading institutions. The framework utilizes a frame-level autoregressive (AR) model, combined with a key-value (KV) caching mechanism, streaming long-form video fine-tuning, and short-window attention...

What is LONGLIVE?

LONGLIVE is a real-time interactive long-form video generation framework jointly developed by NVIDIA and other leading institutions. The framework addresses the dual bottlenecks of efficiency and quality in long-form video generation by employing a frame-level autoregressive (AR) model, combined with a key-value (KV) cache mechanism, streaming long-form video fine-tuning, and short-window attention + frame ingestion technology. LONGLIVE can generate high-quality videos up to 240 seconds long at 20.7 FPS on a single H100 GPU, supporting real-time prompt switching and dynamic adjustments. It opens up new creative possibilities for fields such as creative, educational, and film industries, representing a crucial step in propelling AI video generation from a "toy" to a "productivity tool."

LONGLIVE's main functions

  • Real-time interactionIt supports users to input streaming prompts in real time during video generation, dynamically adjusting video content, guiding the narrative, or changing the style.
  • Long video generationThe model can generate high-quality videos that are several minutes long, supporting complex narratives and scene developments.
  • Efficient ReasoningAchieve a real-time speed of 20.7 FPS on a single NVIDIA H100 GPU, supporting video generation up to 240 seconds while maintaining high fidelity and temporal continuity.
  • High-quality generation: Innovative technologies ensure visual coherence and semantic consistency in generated videos, maintaining smooth transitions even when frequently switching prompts.
  • Low deployment costIt supports INT8 quantization inference, further reducing model size and deployment costs with almost no performance loss.

LONGLIVE's technical principles

  • KV-recache mechanismWhen switching prompts, the key-value (KV) cache is recalculated to "refresh" the state, clearing residual information from old prompts while preserving visual and motion cues. This ensures a smooth transition and accurate execution of new instructions. Integrating the recache operation into training allows the model to learn how to smoothly transition after prompt changes during training.
  • Streaming Long TuningThis approach addresses the quality degradation issue of autoregressive (AR) models in long video generation by simulating the inference process through "rolling expansion," reducing inconsistencies between training and inference. Local supervision and gradient separation are used to avoid out-of-memory (OOM) errors caused by backpropagation of long sequences, ensuring reliable supervision of the teacher model.
  • Short window attention + Frame sinkBased on short-window attention, the attention scope is limited to a local window, significantly reducing computational complexity and memory requirements. A frame inflow mechanism is introduced to restore long-range consistency while maintaining the efficiency of short windows by preserving global anchor points (such as the first frame block of a video).

LONGLIVE's project address

  • GitHub repository: https://github.com/NVlabs/LongLive
  • HuggingFace model libraryhttps://huggingface.co/Efficient-Large-Model/LongLive-1.3B
  • arXiv technical paper: https://arxiv.org/pdf/2509.22622

Application scenarios of LONGLIVE

  • Creative Video ProductionCreators can adjust video content and style in real time, quickly generate long videos that meet their creative needs, and improve creative efficiency and flexibility.
  • Educational content generationTeachers can generate teaching videos in real time according to the teaching progress, dynamically inserting knowledge points or cases to enhance the interactivity and interest of teaching.
  • Film and television productionDirectors and screenwriters can preview different scenes and narrative paths in real time before filming, quickly adjust the script and shooting plan, and reduce production costs.
  • Advertising creationThe advertising team can generate ad videos in real time based on client needs, quickly adjust creative direction, and improve the targeting and appeal of ads.
  • Game developmentDevelopers can generate game cutscenes or dynamic backgrounds in real time and adjust the content according to the game's storyline to enhance player immersion.
Back