DreamActor-H1 - A product demo video generation framework launched by ByteDance

What is DreamActor-H1?

DreamActor-H1, developed by ByteDance, is a framework based on the Diffusion Transformer (DiT) that supports the generation of high-quality human-product demonstration videos from paired human and product images. The framework injects reference information from both humans and products using a masked cross-attention mechanism, while preserving human identity and product details (such as logos and textures). It combines 3D human mesh templates and product bounding boxes to provide precise motion guidance and enhances 3D consistency with structured text encoding. Trained on large-scale mixed datasets, DreamActor-H1 significantly outperforms existing technologies and is suitable for personalized e-commerce advertising and interactive media.

Main functions of DreamActor-H1

• High-fidelity video generationSupports the generation of high-fidelity, realistic demonstration videos from human and product images.
• Identity PreservationDuring the video generation process, human identity features and product details (such as logos, textures, etc.) are preserved.
• Natural Action GenerationIt provides precise motion guidance based on 3D body templates and product bounding boxes, generating natural interactive actions.
• Semantic enhancementBased on structured text encoding, it enhances the visual quality and 3D consistency of videos, especially in the case of small rotational changes.
• Personalized applicationsSuitable for personalized e-commerce advertising and interactive media, supporting diverse human and product inputs.

The technical principles of DreamActor-H1

• Diffusion ModelBased on the generative capabilities of the diffusion model, video content is gradually generated from noise. The diffusion model generates high-quality images or videos by progressively removing noise.
• Masked Cross-Attention MechanismBased on the injected paired human and product reference information, a masked cross-attention mechanism is used to ensure that the details of humans and products in the generated video are accurately preserved.
• 3D motion guidanceCombining 3D body mesh templates and product bounding boxes provides precise motion guidance for video generation, ensuring natural alignment of hand movements with product placement.
• Structured text encodingBased on product descriptions and human attribute information generated by the Visual Language Model (VLM), it enhances semantic consistency in video generation and improves visual quality and 3D stability.
• Multimodal fusionIt integrates human appearance, product appearance, and textual information into a diffusion model, and achieves high-quality video generation based on full attention, reference attention, and object attention mechanisms.

DreamActor-H1 project address

• Project official website:https://submit2025-dream.github.io/DreamActor-H1/
• arXiv technical paper:https://arxiv.org/pdf/2506.10568

Application scenarios of DreamActor-H1

• Personalized product displayBased on the generation of videos of human interaction with products, the product's usage scenarios and functions can be showcased to enhance users' willingness to purchase.
• Virtual trialProvide users with virtual trial experiences, such as virtually trying on clothing or trying on cosmetics, to help users better understand the effects of products.
• Product PromotionGenerate high-quality product demonstration videos for e-commerce platforms, which can be used on product detail pages or in advertising to enhance product appeal and sales conversion rates.
• Social media advertisingGenerate engaging video content for advertising on social media platforms, increasing user engagement and brand exposure.
• Brand promotion: Enhance brand image and user identification by generating videos of brand ambassadors interacting with products.