DreamVVT - A video virtual try-on technology launched by ByteDance in collaboration with Tsinghua University.

What is DreamVVT?

DreamVVT is a video virtual try-on (VVT) technology jointly developed by ByteDance and Tsinghua University (Shenzhen). Based on the Diffusion Transformer (DiTs) framework, it achieves high-fidelity and temporally coherent virtual try-on effects through a two-stage approach. The first stage samples keyframes from the input video and combines them with a Visual Language Model (VLM) to generate semantically consistent try-on images. The second stage utilizes skeletal maps and motion information, combined with a pre-trained video generation model, to ensure the dynamic coherence of the video. DreamVVT can preserve clothing details under complex actions and scenes, supports trying on entire outfits, and can dress cartoon characters in realistic clothing.

Main functions of DreamVVT

• High-fidelity virtual try-onIt can achieve high-fidelity clothing try-on effects in videos, preserving the details and textures of clothing, and maintaining high-quality visual effects even in complex movements and scenes.
• Temporal coherenceThe two-stage approach ensures the continuity of the video try-on results, avoiding abrupt changes or unnatural transitions, and making the entire try-on process look smooth and natural.
• Multi-scenario adaptationIt supports various scenarios and actions, including complex interactions, dynamic backgrounds, and try-on under different lighting conditions, making it highly adaptable.
• Unpaired data trainingIt can be trained based on unpaired person data, which reduces the difficulty and cost of data preparation and improves the generalization ability of the model.
• Trying on the whole outfitIt supports trying on individual garments and allows users to try on complete outfits, providing a more comprehensive virtual try-on experience.
• Cross-domain applicationsDreamVVT allows cartoon characters to wear real-world clothing, expanding the application scope of virtual try-on.
• Dynamic effects supportIt can generate try-on videos with natural dynamic effects, including the movement of clothing and changes in folds, making the try-on effect more realistic.

The technical principle of DreamVVT

• Two-phase processing frameworkA two-stage approach is adopted. The first stage generates high-fidelity keyframe try-on images, and the second stage uses these keyframes to generate a coherent try-on video.
• Diffusion Transformer (DiTs)By leveraging the diffusion Transformer architecture and combining it with a visual language model (VLM), high-quality image generation and semantic consistency can be achieved.
• Keyframe Sampling and GenerationRepresentative frames are sampled from the input video, and semantically consistent and high-fidelity keyframe try-on images are generated through a multi-frame try-on model, providing a foundation for video generation.
• Skeletal mapping and motion information extractionExtract the skeletal map and motion information from the input video to guide the dynamic changes during video generation and ensure temporal continuity.
• Pre-trained video generation model adaptationEnhance the pre-trained video generation model with a LoRA adapter, and combine keyframe try-on images and motion information to generate temporally coherent try-on videos.

DreamVVT's project address

• Project official websitehttps://virtu-lab.github.io/
• Github repositoryhttps://github.com/Virtu-Lab/DreamVVT
• arXiv technical paperhttps://arxiv.org/pdf/2508.02807v1

Application scenarios of DreamVVT

• Online shopping platformThis provides e-commerce platforms with a virtual try-on feature, allowing consumers to upload their own photos or videos to try on different styles and colors of clothing in real time, improving the shopping experience and reducing return rates.
• Virtual Fashion ShowIn the fashion industry, it is used to create virtual fashion shows, showcasing designers' work and breaking through the limitations of traditional fashion shows in terms of venue and time, thus attracting a wider audience.
• Entertainment and Film ProductionIn film and television special effects production, it can quickly generate different clothing try-on effects for characters, saving on clothing production and shooting costs, and allowing animated characters to try on realistic clothing to enhance visual effects.
• Virtual character customizationIn the fields of gaming and virtual reality (VR), providing personalized clothing customization for virtual characters can enhance users' sense of identification and participation with these characters.
• Social media and content creationUsers can share fashion outfits through virtual try-ons on social media, and creators can use this technology to create interesting content and attract more followers.