SnapGen - A mobile text-based image model developed by Snap in collaboration with institutions such as HKUST.

What is SnapGen?

SnapGen, a text-to-image (T2I) diffusion model jointly developed by Snap Inc., Hong Kong University of Science and Technology, and the University of Melbourne, can quickly generate high-resolution (1024×1024 pixels) images on mobile devices in just 1.4 seconds. The model achieves this performance with only 379M parameters, significantly reducing model size and computational requirements. It also achieves a high score of 0.66 on the GenEval metric, surpassing many larger models like SDXL and IF-XL. SnapGen leverages techniques such as optimized network architecture, cross-architecture knowledge distillation, and adversarial step distillation to achieve efficient image generation on mobile devices.

SnapGen's main functions

• High-resolution image generationGenerate high-resolution images up to 1024×1024 pixels quickly on mobile devices.
• Rapid generation capabilityImage generation is completed in 1.4 seconds, significantly improving generation efficiency.
• Optimized model sizeThe SnapGen model has only 379M parameters, yet it surpasses models with more parameters in terms of image quality.
• Cross-architecture knowledge distillationImprove the generation quality of smaller models by transferring knowledge from larger models.
• antagonistic distillation stepsIt combines adversarial training and knowledge distillation to achieve the ability to generate high-quality images in a few steps.

SnapGen's technical principles

• Network architecture optimizationThe network architecture of the denoising UNet and autoencoder (AE) was thoroughly examined to achieve the best balance between latency and performance, reduce model parameters and computational complexity, and maintain the quality of generated images.
• Multilevel knowledge distillationBased on cross-architecture knowledge extraction from larger models, and using multi-level methods to guide model training, high-quality image generation is achieved while maintaining a small model size.
• Scaling with time step awarenessDuring training, a time-step-aware scaling technique is used, which combines multiple training objectives to adapt to the prediction difficulty at different time steps.
• competitive trainingBy combining adversarial training and knowledge distillation, a few steps of teacher modeling are used to achieve rapid, high-quality image generation.
• Integrated adversarial guidanceIn the knowledge distillation process, adversarial guidance is integrated to further improve the quality and realism of the generated images.
• Highly efficient training techniquesImproved training techniques are introduced, including stream matching as the target and logit-normal sampling during training, to improve training stability and generation quality.

SnapGen project address

• Project official website:snap-research.github.io/snapgen
• arXiv technical paper:https://arxiv.org/pdf/2412.09619

Application scenarios of SnapGen

• Social media content creationUsers can quickly generate personalized images for use in posts or stories on social media platforms such as Snapchat.
• Mobile application integrationIntegrate SnapGen into mobile applications to provide users with the ability to create images spontaneously, such as virtual try-on and filter effect previews.
• Games and entertainmentUsed for the rapid generation of in-game assets, or to provide players with the ability to customize characters and environments in mobile games.
• Education and trainingUse SnapGen to generate images for teaching materials, such as scientific diagrams or historical scenes, to enhance the learning experience.
• News and MediaIt enables journalists and media professionals to quickly generate the images needed for news reports, enhancing the appeal and expressiveness of their narratives.