Edicho - A multi-image consistency editing method jointly developed by Ant Group and universities such as HKUST.
Edicho, developed by the Hong Kong University of Science and Technology, Ant Group, Stanford University, and the Chinese University of Hong Kong, is an image editing method based on a diffusion model that enables consistent editing across multiple images. Edicho uses a training-free approach, requiring no...
What is Edicho?
Edicho, developed by the Hong Kong University of Science and Technology, Ant Group, Stanford University, and the Chinese University of Hong Kong, is an image editing method based on a diffusion model that enables consistent editing across multiple images. Edicho uses a training-free approach, requiring no additional training before application. Its core lies in guiding the editing process with explicit image correspondences. Based on an attention operation module (Corr-Attention) and a classifier-guided free-classification (CFG) denoising strategy, it ensures consistent editing across different images. Corr-Attention enhances the self-attention mechanism with explicit correspondences, effectively transferring features from the source image to the target image. Corr-CFG, based on modified CFG calculations and combined with pre-computed correspondences, guides the generation process to more closely approximate the desired editing while maintaining high image quality.
Edicho's main functions
- Consistent Image EditingAchieve consistent editing across multiple images, including local editing (such as image restoration) and global editing (such as image style transfer), ensuring that the editing results remain highly consistent across various scenarios.
- Plug and play compatibilityAs an algorithm for inference, it is seamlessly compatible with most diffusion-based editing methods (such as ControlNet and BrushNet), requiring no additional training data and can be directly applied to real-world tasks.
- Wide range of task applicabilityIt is suitable for image editing and can also be extended to more fields such as personalized content creation, 3D reconstruction and consistent texture application, thus expanding the scope of application of editing tools.
Edicho's technical principles
- Explicit Image Correspondence GuidanceThis approach guides the editing process based on explicit image correspondences, avoiding the limitations of traditional implicit correspondence methods. Robust correspondences are extracted from the input image using pre-trained correspondence extractors (such as DIFT and Dust3R), and then the pre-computed explicit correspondences are injected into the denoising process of the diffusion model to ensure consistent editing.
- Attention operation module (Corr-Attention)Enhanced attention mechanism: Guided feature transfer based on the correspondence between images. In the self-attention module, query features are transformed according to explicit correspondence, and relevant features are borrowed from the source image to form a new query matrix, achieving editing consistency during the denoising process.
- Optimized Classifier Free Guided (CFG) Denoising Strategy (Corr-CFG)By combining pre-computed correspondences, high quality and consistency are maintained during the editing process. The computation method of CFG is modified to manipulate the unconditional branches in the CFG framework under the guidance of correspondences, and unconditional embedding features are fused to further enhance the consistency effect, while preserving the integrity of the strong generative prior of the pre-trained model.
Edicho's project address
- Project official website:https://ezioby.github.io/edicho
- GitHub repository:https://github.com/EzioBy/edicho
- arXiv technical paper:https://arxiv.org/pdf/2412.21079
Application scenarios of Edicho
- Product MarketingSellers on e-commerce platforms edit product images taken from different angles consistently to improve click-through rates and conversion rates.
- Themed Activity: During the Christmas season, users can edit multiple family photos into Christmas-themed images and share them on social media to enjoy with family and friends.
- Personalized content generationUsers can edit multiple personal photos into a profile picture with a unified style according to their own preferences and use it on different social media platforms.
- Virtual RealityIn game development, designers edit multiple design sketches of virtual characters to ensure consistency and enhance the integrity and recognizability of the character's image.
- Augmentation of medical image datasetsIn the field of medical image analysis, consistent editing of images in medical image datasets can improve the training effect of medical image recognition models.