Wear-Any-Way - Alibaba's Taobao Image Search team launches a customizable virtual try-on frame.
Wear-Any-Way is an innovative virtual try-on framework launched by Alibaba's Taobao Image Search team. Through a sparse correspondence alignment mechanism, it achieves high-fidelity and customizable virtual try-on effects. Users can generate realistic try-on images,...
What is Wear-Any-Way?
Wear-Any-Way is an innovative virtual try-on framework launched by Alibaba's Taobao Image Search team. Through a sparse correspondence alignment mechanism, it achieves high-fidelity and customizable virtual try-on effects. Users can generate realistic try-on images and precisely control how clothing is worn through simple clicks and drags, such as rolling up sleeves or adjusting the opening of a coat, bringing a completely new interactive experience to virtual try-on. Wear-Any-Way supports multiple input types, including single-item try-on and multi-item ensemble try-on, and can be applied to various scenarios such as model-to-model and model-to-street.
Wear-Any-Way's main functions
- Free customization and try-onUsers can freely customize the clothing worn by the virtual model according to their preferences and needs, including adjusting the size, color, style, and details such as sleeve length, collar type, and waist. They can try different combinations, such as inner layers, outerwear, and accessories, to create a unique look.
- High-fidelity renderingBased on advanced image rendering technology, the system ensures that the virtual try-on experience closely resembles the real wearing experience. Through high-precision model construction and lighting processing, the system can present the material texture, pattern details, and shape changes of clothing after wearing, allowing users to experience a more realistic and vivid visual effect during the try-on process.
- Try on multiple outfitsIt supports virtual try-on for individual garments, and also supports offering both tops and bottoms simultaneously to achieve the effect of trying on a complete outfit.
- Try on clothes in multiple scenariosIt can support various sub-tasks, such as trying on clothes from model to model, trying on clothes with multiple garments, or complex scenarios such as street photography and sitting poses.
- Intelligent recommendations and matchingBased on Alibaba's big data analytics and AI technology, Wear-Any-Way can intelligently recommend suitable clothing styles and matching schemes based on users' shopping history, browsing records, and personal preferences.
The technical principle of Wear-Any-Way
- Sparse correspondence alignment mechanismBy learning a series of permutable point embeddings and injecting them into the model, precise control over clothing details is achieved. Users can customize the upper body effect of clothing through simple click and drag operations, such as dragging the sleeves to roll them up, dragging the coat to open it, and controlling the style of pleats by clicking.
- Dual U-Net structureThe entire framework consists of two U-Nets. The Reference U-Net takes the clothing image as input and extracts fine-grained features; the Main U-Net is responsible for generating the try-on results, which requires the person image (masked), the clothing mask, and potential noise as input, and is controlled by an additional pose encoder.
- Point-based controlIt is implemented by a point embedding network and a sparse correspondence alignment module. The symbols for flame and snowflake represent trainable parameters and frozen parameters, respectively.
Wear-Any-Way project address
- Project official website:https://mengtingchen.github.io/wear-any-way
- arXiv technical paper:https://arxiv.org/pdf/2403.12965
Application scenarios of Wear-Any-Way
- Virtual fitting roomConsumers can see how clothes look on models in a virtual fitting room and adjust the clothes to their preferred style through interactive operations.
- Fashion Design and DisplayDesigners and brands can more flexibly showcase how different models look wearing the same garment, and better assess the adaptability and versatility of clothing.
- Personalized customizationUsers can adjust how they wear the clothes according to their preferences and needs, such as rolling up the sleeves, opening the coat, adjusting the pleats, etc.
- E-commerce and retailMerchants can more intuitively demonstrate how clothing looks on models, and users can see how clothing looks in different scenarios in a virtual environment, thus enhancing the consumer shopping experience.