LTX-2.3 - Lightricks' latest open-source video generation model

What is LTX-2.3?

LTX-2.3 is the latest generation video generation model open-sourced by the Israeli AI company Lightricks. It employs the Diffusion Transformer architecture and boasts 22 billion parameters. The model supports text, image, and audio input to generate videos, with a maximum output resolution of 4K. It also natively supports 9:16 portrait mode and 24/48 FPS frame rate selection. Compared to its predecessor, LTX-2.3 significantly improves image detail sharpness through a newly trained VAE architecture, resolving texture blurring issues at high resolutions. It also adds native audio generation functionality, enabling synchronized audio and video output. The model provides 7 generation endpoints, can generate up to 20 seconds of video per iteration, and supports LoRA fine-tuning.

Main functions of LTX-2.3

• Multimodal video generationIt supports three core generation methods: text-to-video, image-to-video, and audio-to-video, to meet different creative needs.
• Native portrait mode supportAdded 9:16 portrait format (up to 1080×1920), directly adapted to short video platforms and social media content creation.
• Audio and video synchronization generationThe new native audio generation function can output ambient sound, sound effects and dialogue simultaneously, and can also drive video image generation through audio input.
• Flexible frame rate selectionSupports two frame rate modes: 24FPS cinematic and 48FPS smooth motion.
• Video extension and reshootIt provides extend-video and retake-video functions, with a maximum generation time of 20 seconds, which can be further extended through extension.
• Quick generation modeFor efficiency-critical scenarios, we offer accelerated versions for text-to-video fast and image-to-video fast.
• High resolution outputSupports up to 4K resolution, achieving sharper image details and textures through the new VAE architecture.
• LoRA fine-tuning supportCreators can train the LoRA adapter locally and complete customized model fine-tuning in less than an hour.
• Supporting super-resolution toolsProvides a post-processing model with 2x/1.5x spatial super-resolution and 2x frame rate boost to optimize the final output quality.
• Local desktop editorSimultaneously, we launched the LTX Desktop open-source video editor, based on the LTX-2.3 engine, which runs entirely locally without the need for cloud access.

Technical Principles of LTX-2.3

• DiT Diffusion Transformer ArchitectureBased on the Diffusion Transformer architecture, it combines the diffusion model with the Transformer to generate high-quality videos through an iterative denoising process, with a scale of approximately 22 billion parameters.
• New VAE Variational AutoencoderThe retrained Variational Autoencoder significantly improves encoding-decoding quality, dramatically enhancing image sharpness, texture detail, and facial feature clarity, resolving the issue of blurred details at high resolutions in the previous generation.
• Spatiotemporal joint modelingThe system employs a spatiotemporal attention mechanism to process video data, simultaneously modeling both spatial image content and temporal motion changes to ensure the temporal coherence of the generated video.
• Native audio generation moduleIt integrates an audio generation sub-network to achieve end-to-end synchronous generation of audio and video, supports the generation of visual content driven by audio input, and ensures audio-visual synchronization.
• Multimodal conditional injectionBy using different conditional encoders, text, image, and audio modal inputs are uniformly mapped to the latent space, enabling flexible multimodal control.
• Distillation Accelerated VersionIt provides a distilled version of the model, which compresses the model size through knowledge distillation technology, improving inference speed while maintaining quality.
• LoRA low-rank adaptationIt supports Low-Rank Adaptation technology, allowing users to quickly inject specific styles or concepts into pre-trained models to achieve low-cost customization.
• Super-resolution post-processingIt is equipped with an independent super-resolution model, which uses spatial upsampling (2x/1.5x) and frame rate interpolation (2x) techniques to perform secondary optimization on the generated video.

LTX-2.3 project address

• Project official websitehttps://ltx.io/model/ltx-2-3
• Hugging Facehttps://huggingface.co/Lightricks/LTX-2.3
• arXiv technical paper: https://arxiv.org/pdf/2601.03233

Application scenarios of LTX-2.3

• Short videos and social media contentIt natively supports 9:16 portrait mode and offers 24/48FPS frame rate options, perfectly adapting to the content creation needs of platforms such as Douyin, TikTok, and Instagram Reels.
• Advertising and Marketing VideosQuickly generate product showcases and brand promotional videos, support image-to-video conversion, and transform static product images into dynamic advertising materials.
• Film and television pre-visualization and concept designDirectors and producers can use text to quickly generate shot previews, verify creative concepts, and then proceed with the actual filming, reducing pre-production costs.
• Game and animation productionGenerate game cutscenes, character movement references, or serve as initial source material for animation production, accelerating the content production process.
• Music Video and Audio-Visual CreationThe audio-to-video feature supports audio-driven video generation, making it ideal for musicians to quickly create lyric-based music videos or visual audio tracks.
• Education and training contentIt generates teaching demonstration videos and visualizes operation steps, transforming static teaching materials into dynamic explanation content.