MCA-Ctrl - A custom image generation framework developed by the Chinese Academy of Sciences and the University of Science and Technology of China

What is MCA-Ctrl?

MCA-Ctrl (Multi-party Collaborative Attention Control) is an image customization and generation framework developed by the Institute of Computing Technology, Chinese Academy of Sciences, and the University of Chinese Academy of Sciences. It achieves high-quality image generation based on text and complex visual conditions. MCA-Ctrl introduces two attention control strategies, Self-Attention Global Injection (SAGI) and Self-Attention Local Query (SALQ), as well as a subject localization module (SLM), to address issues such as inconsistent backgrounds and subject confusion. MCA-Ctrl outperforms existing methods in zero-shot image customization, effectively maintaining the consistency of subject features and conditional information.

Main functions of MCA-Ctrl

• High-quality image customizationGenerate high-quality custom images based on text or image conditions, maintaining semantic consistency with the conditional input.
• Main features preservedIn complex visual scenes, accurately capture the appearance and content of a specific subject while avoiding subject leakage and confusion.
• Background ConsistencyMaintain background consistency during image conditional generation.
• Zero-sample generationSupports generating high-quality images directly under zero-sample conditions.
• Multiple task supportIt supports various image customization tasks, including subject generation, subject replacement, and subject addition.

MCA-Ctrl Technical Principles

• Multi-agent collaborative diffusion processIt is based on three parallel diffusion processes: subject diffusion (Bsub), conditional diffusion (Bcon), and target diffusion (Btgt). The subject diffusion process is responsible for processing the subject image, the conditional diffusion process processes the conditional image or text, and the target diffusion process generates the final customized image.
• Self-attention layer operationSelf-Attention Global Injection (SAGI) injects global self-attention features from the subject and conditional images into the target diffusion process, enhancing the detail realism and content consistency of the target image. SAGI helps correct feature confusion caused by local queries. The target diffusion process acquires the appearance of the subject and background content based on the local features of the query subject and conditional images. SALQ operations ensure that the generated image maintains a high degree of consistency with the conditional image in terms of both subject and background.
• Main Positioning Module (SLM)To accurately identify and locate subjects in complex visual scenes, a subject localization module is introduced. SLM combines object detection models (such as DINO) and segmentation models (such as SAM), processes multimodal instructions, and outputs accurate subject image layers and editable image layers, reducing feature confusion and artifacts.
• No tuning frameworkMCA-Ctrl does not require individual fine-tuning training for each subject. Based on attention control strategies and subject localization modules, it achieves high-quality image customization under zero-shot conditions.

MCA-Ctrl project address

• GitHub repository:https://github.com/yanghan-yh/MCA-Ctrl
• arXiv technical paper:https://arxiv.org/pdf/2505.01428

Application scenarios of MCA-Ctrl

• Digital content creationQuickly generate characters and scenes for games and animations.
• Advertising and MarketingCreate personalized advertising images and brand promotion materials.
• Entertainment and social mediaGenerate personalized avatars, images, and social media content.
• Education and Training: To assist in the production of teaching materials and create virtual laboratory scenarios.
• Art and DesignProvides inspiration for artistic creation and assists in interior design previews.