xAR - An autoregressive visual generation framework developed by ByteDance in collaboration with Johns Hopkins University

What is xAR?

xAR is a novel autoregressive visual generation framework jointly proposed by ByteDance and Johns Hopkins University. The framework addresses the issues of insufficient information density and accumulated error in traditional autoregressive models for visual generation through "Next-X Prediction" and "Noisy Context Learning" techniques.

Main functions of xAR

• Next-X PredictionIt extends the traditional "next label prediction" to support models in predicting more complex entities (such as image patches, cells, subsamples, the entire image, etc.) and capturing richer semantic information.
• Noisy Context LearningBy introducing noise during training, the robustness of the model to errors can be improved, thus mitigating the problem of accumulated errors.
• High-performance generationOn the ImageNet dataset, the xAR model outperforms existing techniques, such as DiT and other diffusion models, in both inference speed and generation quality.
• Flexible prediction unitIt supports various prediction unit designs (such as unit, subsampling, multi-scale prediction, etc.) and is suitable for different vision generation tasks.

The technical principle of xAR

• Flow MatchingxAR, based on the flow matching method, transforms the discrete label classification problem into a continuous entity regression problem. Specifically:
  • The model generates noisy inputs through interpolation and noise injection.
  • In each autoregressive step, the model predicts the directional flow (Velocity) from the noise distribution to the target distribution, thereby progressively optimizing the generated results.
• Reasoning strategiesDuring the inference phase, xAR uses an autoregressive approach to progressively generate images.
  • First, predict the initial cells (e.g., 8×8 image blocks) from Gaussian noise.
  • Based on the generated units, the model progressively generates the next unit until the entire image is generated.
• Experimental resultsxAR achieved significant performance improvements in ImageNet-256 and ImageNet-512 benchmarks.
  • xAR-BThe model with 172 million parameters is 20 times faster than DiT-XL (675 million parameters) in inference speed, and achieves a Fraser inception distance (FID) of 1.72, which is better than existing diffusion and autoregressive models.
  • xAR-HThe 1.1 billion parameter model achieved an FID of 1.24 on ImageNet-256, setting a new state-of-the-art level without relying on a visual base model (such as DINOv2) or advanced guided region sampling.

xAR's project address

• Project official website:https://oliverrensu.github.io/project/xAR/
• arXiv technical paper:https://arxiv.org/pdf/2502.20388

Application scenarios of xAR

• Artistic CreationArtists can use xAR to generate creative images, which can serve as inspiration for their artworks or be used directly in their creations. xAR can generate images with rich detail and diverse styles, supporting creative needs at different resolutions and in various styles.
• Virtual scene generationIn game development and virtual reality (VR), xAR can quickly generate realistic virtual scenes, including natural landscapes, urban environments, and virtual characters, enhancing the user experience.
• Restoration of old photosBy generating high-quality image content, xAR can repair damaged parts of old photos, restoring their original details and colors.
• Video content generationxAR can generate specific scenes or objects in videos for use in video effects creation, animation generation, and video editing.
• Data AugmentationBy generating diverse images, xAR can expand the training dataset and improve the model's generalization ability and robustness.