OmniGen2 - An open-source multimodal generative model from the Beijing Academy of Artificial Intelligence

What is OmniGen2?

OmniGen2 is an open-source multimodal generative model developed by the Beijing Academy of Artificial Intelligence. It can generate high-quality images based on text prompts and supports instruction-guided image editing, such as modifying backgrounds or facial features. OmniGen2 employs a dual-component architecture, combining a visual language model (VLM) and a diffusion model to achieve unified processing of various generative tasks. Its advantages lie in its open-source and free nature, high performance, and powerful contextual generation capabilities, making it suitable for commercial, creative design, and research and development scenarios.

Main functions of OmniGen2

• Text to Image GenerationIt can generate high-fidelity and aesthetically pleasing images based on text prompts. It performs excellently in multiple benchmark tests, such as scoring 0.86 and 83.57 on GenEval and DPG-Bench, respectively.
• Instruction-guided image editingIt supports complex, command-driven image modifications, including local modifications (such as changing clothing color) and overall style transformations (such as converting a photo to an anime style). In image editing tasks, OmniGen2 achieves a balance between editing accuracy and image fidelity across multiple benchmarks.
• Context generationIt can handle and flexibly combine various inputs (such as people, reference objects, and scenes) to generate novel and coherent visual outputs. In the OmniContext benchmark, OmniGen2 outperforms existing open-source models by more than 15% in visual consistency metrics.
• Visual understandingIt inherits the powerful image content parsing and analysis capabilities of the Qwen-VL-2.5 basic model.

OmniGen2's technical principles

• Dual-path architectureOmniGen2 employs separate text and image decoding paths, handling text and image modalities separately. The text generation part is based on the Qwen2.5-VL-3B multimodal language model (MLLM), while image generation is accomplished through a separate diffusion Transformer module. This avoids the negative impact of text generation on image quality.
• Diffusion TransformerThe image generation part employs a 32-layer diffusion Transformer with a hidden dimension of 2520 and a total of approximately 4 billion parameters. This module uses the Rectified Flow method for efficient image generation.
• Omni-RoPE Position EncodingOmniGen2 introduces a novel multimodal rotational position embedding (Omni-RoPE), which decomposes positional information into sequence and modal identifiers, two-dimensional height coordinates, and width coordinates. It can accurately encode information for each location in an image, while supporting multi-image spatial localization and identity differentiation.
• Reflection mechanismOmniGen2 employs a dedicated reflection mechanism to improve the quality and consistency of generated images. The model can self-evaluate its output and make improvements across multiple rounds.
• Training strategyOmniGen2 employs a phased training approach. First, it pre-trains a diffusion model on a text-to-image task, then introduces mixed task training, and finally performs end-to-end training of reflective capabilities.
• Data processingThe training data is extracted from videos and undergoes multiple filtering processes, including DINO similarity filtering and VLM consistency checks, to ensure data quality.

OmniGen2 project address

• Project official websitehttps://vectorspacelab.github.io/OmniGen2/
• Github repositoryhttps://github.com/VectorSpaceLab/OmniGen2
• arXiv technical paper: https://arxiv.org/pdf/2506.18871

Application scenarios of OmniGen2

• Design concept generationDesigners can quickly generate design concept diagrams and sketches using simple text descriptions.
• Story creation assistanceContent creators can generate corresponding scene and character images based on the story's plot and character descriptions.
• Video production material generationCreators can generate various scenes, character actions, and special effects images, and then import them into video editing software to create supplementary materials for animations, special effects videos, or live-action videos.
• Game scene and character generationDevelopers can quickly generate game scenes and characters using text descriptions.
• Teaching resource generationEducators can generate relevant images and diagrams based on the teaching content. For example, when explaining historical events, they can generate images of ancient war scenes or historical figures.