SANA 1.5 - A new framework for textural graphs launched by NVIDIA in collaboration with MIT, Tsinghua University, Peking University, and other institutions.

What is SANA 1.5?

SANA 1.5 is a new, highly efficient Linear Diffusion Transformer developed by NVIDIA in collaboration with MIT, Tsinghua University, Peking University, and other institutions for text-to-image generation tasks. Building upon SANA 1.0, it introduces three major innovations: efficient training scaling, which expands the model from 1.6 billion parameters to 4.8 billion parameters through a deep growth paradigm, significantly reducing computational resources, and incorporating an efficient 8-bit optimizer; deep model pruning, which efficiently compresses the model by analyzing block importance, allowing for flexible adjustment of model size under different computational budgets; and inference-time scaling, which enables smaller models to achieve the quality of larger models during inference through repeated sampling and a selection mechanism based on a Visual Language Model (VLM).

Main functions of SANA 1.5

• High-efficiency training extensionSANA 1.5 employs a deep growth paradigm, which can scale the model from 1.6 billion parameters to 4.8 billion parameters, significantly reducing computational resources.
• Model depth pruningSANA 1.5 introduces a model compression technique based on block importance analysis, which can efficiently compress large models to arbitrary sizes while minimizing quality loss. By analyzing the similarity patterns of the input and output in the diffusion transformer, unimportant blocks are pruned, and model quality is quickly restored through fine-tuning.
• Expansion during reasoningSANA 1.5 introduces an inference-time scaling strategy that enables small models to achieve the quality of large models during inference through repeated sampling and a selection mechanism based on the visual language model (VLM).
• Multilingual supportSANA 1.5 supports multilingual text input, including Chinese, English, and emojis, making it suitable for global content creation and localized design.
• Open source and community supportThe code and pre-trained models of SANA 1.5 have been open-sourced, allowing researchers and developers to customize and extend them, further promoting their adoption in academic research and industrial applications.
• Reasoning efficiencyWith the CAME-8bit optimizer, SANA 1.5 enables large-scale model fine-tuning on a single consumer-grade GPU, making high-quality image generation more efficient and accessible.

SANA 1.5 Performance Testing

• Model GrowthSANA 1.5 scaled up from 1.6 billion parameters to 4.8 billion parameters through a model growth strategy, improving the GenEval score from 0.66 to 0.72, approaching the industry-leading Playground v3 (2.4 billion parameters) at 0.76, while reducing inference latency by 5.5 times.
• Model pruningThrough deep pruning, SANA 1.5 can flexibly adjust the model size under different computational budgets. For example, after pruning the 4.8 billion parameter model to 1.6 billion parameters, and after 100 fine-tuning steps, the GenEval score reached 0.672, exceeding the 0.664 of the SANA 1.0 1.6 billion parameter model.
• Inference ScalingBy generating multiple samples and selecting the best sample based on VLM, SANA 1.5 improved its GenEval score from 0.72 to 0.80, surpassing Playground v3's 0.76.

SANA 1.5 project address

• arXiv technical paper:https://arxiv.org/pdf/2501.18427

Application scenarios of SANA 1.5

• Creative DesignSANA 1.5 can generate high-quality images based on text prompts, making it suitable for creative design fields such as advertising design, illustration, and game art.
• Teaching aidsTeachers can use SANA 1.5 to generate course-related images to help students better understand abstract concepts.
• Film and television productionIn film and television production, SANA 1.5 can generate concept art, scene design drawings, etc., to help directors and art directors quickly conceive and validate ideas.
• Engineering DesignEngineers can use SANA 1.5 to generate visual renderings of engineering designs, helping teams better understand design intent and optimize design solutions.
• Mobile applicationThrough deep model pruning and inference-time scaling, SANA 1.5 can run efficiently on mobile devices, providing real-time image generation capabilities for mobile applications.
• Content moderationBy combining security inspection models (such as ShieldGemma-2B), SANA 1.5 can review the text entered by the user before generating the image, ensuring that the generated content meets security standards and avoiding the generation of inappropriate content.