AB
AB
AiBoss

All Categories

project

GLM-Image - A multimodal image generation model jointly developed by Zhipu and Huawei

GLM-Image is a new generation of multimodal image generation model open-sourced by Zhipu in collaboration with Huawei. It is trained on the Ascend Atlas 800T A2 device and the Ascend MindSpore framework, and is the first state-of-the-art model trained entirely on domestically produced chips.

What is GLM-Image?

GLM-Image is a next-generation multimodal image generation model open-sourced by Zhipu in collaboration with Huawei. Trained on the Ascend Atlas 800T A2 device and the Ascend MindSpore framework, it is the first state-of-the-art (SOTA) model trained entirely on domestically produced chips. The model employs a hybrid architecture of "autoregressive + diffusion decoder," combining a 9B autoregressive model and a 7B diffusion decoder to balance global semantic understanding and high-frequency detail generation, making it particularly adept at text rendering and knowledge-intensive scenes. It ranked first among open-source models on the CVTG-2K and LongText-Bench leaderboards, supports image generation at multiple resolutions, and boasts high cost-effectiveness and speed, providing an important reference for the development of domestic open-source image generation models.

Main functions of GLM-Image

  • High-quality image generationThe model can generate high-resolution images (up to 2048×2048), covering a variety of scenes such as portraits, landscapes, and still life.
  • Complex text renderingGLM-Image excels at generating complex text content within images, supports multi-region text generation, and is suitable for knowledge-intensive scenarios such as posters, PPTs, and science illustrations.
  • Multitasking supportGLM-Image supports various image-to-image tasks, including image editing, style transfer, and multi-subject consistency generation.
  • Multi-resolution adaptiveThe model can adaptively handle image generation tasks of different resolutions without retraining, and is highly flexible.

GLM-Image Technical Principles

  • Autoregressive module:ModelAn autoregressive model based on 9B parameters is responsible for global semantic understanding and low-frequency layout information of the image.Enhance the ability to understand complex instructions through joint training of text-to-image and image-to-image methods.The model uses MRoPE (Multi-Dimensional RoPE) as a location embedding, supporting the interleaved generation of images and text.
  • diffusion decoder:ModelThe DiT (Diffusion Transformer) structure based on 7B parameters focuses on generating high-frequency details, such as text strokes and fine textures in images.By combining semantic VQ (Vector Quantization) tokens and VAE (Variational Autoencoder) latent representations, the model achieves the fusion of semantic information and high-frequency details.The Glyph-byT5 model is introduced to perform character-level encoding of text regions, thereby improving the accuracy of text generation.
  • Training and optimization:The entire process was completed on an Ascend Atlas 800T A2 device, based on the Ascend MindSpore framework, to verify the feasibility of using domestically produced chips in high-performance model training. (Model)Dynamic graph multi-level pipeline optimization and multi-stream parallel strategy are adopted to improve training efficiency and performance.The reinforcement learning optimization module optimizes the autoregressive generator and the diffusion decoder respectively, improving semantic consistency and visual detail quality.

GLM-Image project address

  • Project official websitehttps://z.ai/blog/glm-image
  • GitHub repositoryhttps://github.com/zai-org/GLM-Image
  • HuggingFace model libraryhttps://huggingface.co/zai-org/GLM-Image

Application scenarios of GLM-Image

  • Science illustration and educationGLM-Image can generate science illustrations that include complex logic and textual explanations, helping to present educational content more intuitively.
  • Multi-panel drawings and comicsThe model is suitable for generating e-commerce images, comics, and other multi-panel graphics, maintaining a consistent style and accurately generating multiple text elements.
  • Social media and content creationThe model can quickly generate social media covers, advertising creative images, etc., and supports complex graphic and text layouts to enhance the appeal of content.
  • Commercial posters and advertisementsThe model can generate commercial posters and holiday promotional images with strong design sense and precise text embedding to meet brand promotion needs.
  • Realistic photography and artistic creationGLM-Image excels at generating realistic images of portraits, landscapes, and still lifes, and supports customized artistic styles to meet creative needs.
Back