SeFi-Image - An open-source text-to-image model based on semantic-first diffusion. - AiBoss

What is SeFi-Image?

SeFi-Image is a text-to-image model based on semantic-first diffusion, offering 1B, 2B, and 5B specifications. The model separates high-level semantic structure from texture details, allowing for pre-denoising of the semantic flow and providing clear structural anchors for texture generation. The 5B model, trained on a 125K A800 GPU for hours, performs strongly on benchmarks such as GenEval, LongTextBench, CVTG-2K, and OneIG, supporting scene compositing, rich text layout, anime characters, diverse styles, and portrait generation.

Main functions of SeFi-Image

Text to Image GenerationGenerates high-quality images based on text prompts and supports natural language descriptions.
Multi-specification modelIt offers three parameter scales: 1B, 2B, and 5B, to meet different computing power requirements.
Semantic Priority DiffusionSeparating semantic structure from texture details enables clearer geometric and compositional control.
Multi-domain supportIt covers scene compositing, rich text layout, anime characters, stylized art, and portrait generation.
Long text renderingIt supports long text prompts in both Chinese and English, and performs best on LongTextBench.

The technical principle of SeFi-Image

Semantic-texture decoupling architectureSeFi-Image employs a semantic-first diffusion architecture, which splits the image generation process into two independent denoising trajectories: a semantic stream and a texture stream.
Timing offset mechanismThe model introduces a temporal offset mechanism, which allows the semantic flow to start denoising one time step ahead, providing clear structural anchors for texture generation.
Three-stage generation processThe entire generation process is divided into three stages: the first stage is semantic initialization, the second stage is asynchronous joint denoising of semantics and texture, and the third stage is texture refinement and restoration.
Reconstruction-Generation EquilibriumWhile improving the fidelity of texture reconstruction, the architecture increases the training difficulty of the diffusion model, thus achieving an effective balance between reconstruction and generation.

How to use SeFi-Image

Obtain resourcesVisit the SeFi-Image website or arXiv page to download the paper, open-source code, and model weight files of the corresponding specifications.
Configure environmentInstall deep learning dependencies such as PyTorch on your local server or workstation, and prepare GPU memory resources that match the model size.
Loading ModelLoad the downloaded 1B, 2B, or 5B model checkpoints into the inference framework and complete the weight initialization and parameter settings.
Input prompts: Write Chinese and English text prompts that include scene descriptions, style requirements, or text content, supporting long texts and complex combinations of descriptions.
Execution generationThe semantic-first diffusion sampling process is executed. The model will first complete the semantic structure denoising, and then generate texture details based on semantic anchors.
Output imageThe final denoised latent variables are converted into pixel images by a decoder and saved as an output file with the required resolution and format.

SeFi-Image's core advantages

Precision structural controlThe semantic priority mechanism enables the model to achieve more precise structural control over complex spatial combinations and text layout.
Low training costThe 5B parameter model was trained using only 125K A800 GPU hours, achieving top-tier generation performance with relatively low computational cost.
Long texts take precedenceIn the long text rendering benchmark, SeFi-Image achieved a score of 0.978 on LongTextBench, significantly outperforming similar competitors.
Visual text accuracyThe model achieved an accuracy of 0.895 in the CVTG-2K character-level visual text generation test, and the generated posters and labels were highly readable.
Multi-benchmark firstSeFi-Image ranks first in several authoritative benchmark tests such as GenEval and OneIG-EN, demonstrating comprehensive generation capabilities.

SeFi-Image project address

Project official website: https://jmliu206.github.io/sefi-web/
GitHub repository: https://github.com/jmliu206/SeFi-Image
HuggingFace model libraryhttps://huggingface.co/SeFi-Image
arXiv technical paper: https://arxiv.org/pdf/2606.22568

Comparison of SeFi-Image with similar competitors

Comparison Dimensions	SeFi-Image-5B	Qwen-Image
Architectural features	Semantic-priority diffusion, decoupling of semantic flow and texture flow, and introduction of a temporal offset mechanism.	Conventional diffusion or autoregressive architectures do not explicitly employ a semantic-texture separation mechanism.
GenEval	0.88(First)	0.87
LongTextBench	0.978(First)	0.945
CVTG-2K	0.895(First)	0.829
DPG-Bench	87.27	88.32(First)
OneIG-EN	0.5606(First)	0.5390
OneIG-ZH	0.5379	0.5480(First)

Application scenarios of SeFi-Image

Advertising poster designIn the field of advertising and poster design, SeFi-Image can generate brand posters, product menus, and product labels with clear and readable text.
Anime character creationFor anime and character creation, the model supports generating various composition forms such as anime characters, fantasy scenes, and full-body or close-up shots.
Natural Scene GenerationIn terms of generating natural scenes and landscapes, SeFi-Image can create images of weather, cities, animals, and landscapes with free aspect ratios.
Artistic Style CreationThe model supports various art styles such as illustration, ink painting, plush toys, stickers, and sketches, making it suitable for creating stylized visual content.
Portrait photography assistanceIn portrait and photography-assisted scenarios, SeFi-Image can generate portraits and environmental portraits with diverse lighting, poses, and materials.