PixelFlow - An image generation model jointly developed by the University of Hong Kong and Adobe.

What is PixelFlow?

PixelFlow, a collaborative image generation model developed by the University of Hong Kong and Adobe, supports image generation directly in pixel space. Based on efficient cascaded flow modeling, PixelFlow progressively scales from low to high resolution, reducing computational costs. PixelFlow achieves an FID score of 1.98 on the 256×256 ImageNet category-conditional image generation task, demonstrating excellent image quality and semantic control. PixelFlow also performs well in text-to-image generation tasks, supporting the generation of high-quality images that are highly consistent with the text descriptions. PixelFlow's end-to-end trainability and efficient multi-scale generation strategy provide a new research direction for next-generation visual generative models.

PixelFlow's main features

• High-quality image generationSupports the generation of high-resolution, high-quality images.
• Category-based conditional image generationGenerate the corresponding image based on the given category label.
• Text to Image GenerationIt generates matching images based on text descriptions, supporting complex semantic understanding and visual representation.

The technical principles of PixelFlow

• Stream matchingStream matching is a generative modeling technique that uses a series of linear paths to progressively transform samples from a prior distribution (such as a standard normal distribution) into samples from a target data distribution. During training, training samples are constructed based on linear interpolation, and the model is trained to predict the conversion speed from intermediate samples to real data samples.
• Multiscale generationThis approach involves progressively increasing the image resolution through a multi-stage denoising process. Each stage starts with a noisy image at a lower resolution and gradually increases the resolution until the target resolution is reached. This progressive resolution increase method avoids performing all denoising steps at full resolution, significantly reducing computational costs.
• Transformer architecture:
  • Patchify: Converts the spatial representation of the input image into a label of a 1D sequence.
  • RoPE (Rotary Position Embedding)Replace the original sine and cosine position coding with RoPE to better handle different image resolutions.
  • Resolution Embedding: Introduce additional resolution embedding to distinguish different resolutions.
  • Text to Image GenerationA cross-attention layer is introduced in each Transformer block to align visual features with the text input.
• End-to-end trainingIt trains directly in pixel space based on a unified parameter set, without the need for pre-trained VAEs or other auxiliary networks. During training, the model samples training data uniformly from all resolution stages and performs joint training using sequence packing techniques, improving training efficiency and model scalability.
• Efficient reasoning strategiesDuring inference, PixelFlow starts with the lowest resolution Gaussian noise, gradually denoising and increasing the resolution until the target resolution is reached. It supports multiple ODE solvers (such as Euler and Dopri5), allowing users to choose different solvers to balance speed and generation quality as needed.

PixelFlow project address

• GitHub repository:https://github.com/ShoufaChen/PixelFlow
• arXiv technical paper:https://arxiv.org/pdf/2504.07963
• Experience the demo online:https://huggingface.co/spaces/ShoufaChen/PixelFlow

Application scenarios of PixelFlow

• Art and DesignGenerate creative paintings, graphic design elements, and virtual characters.
• Content creation: Assists in video production, game development, and social media content creation.
• Education and ResearchAs a teaching tool, it helps to understand complex concepts and assists in the visualization of scientific research.
• Business and MarketingGenerate product design prototypes, advertising images, and brand promotion content.
• Entertainment and InteractionUsed in interactive storytelling, VR/AR content generation, and personalized image customization.