BAG - A 3D wearable asset generation technology jointly launched by CUHK and Tencent.

What is BAG?

BAG (Body-Aligned 3D Wearable Asset Generation) is an innovative 3D wearable asset generation technology jointly proposed by the Chinese University of Hong Kong and Tencent. By combining a multi-view image diffusion model and a ControlNet, it utilizes human shape and pose information to automatically generate 3D wearable assets, such as clothing and accessories, that perfectly fit the human body. The core of BAG lies in its multi-view generator and 3D diffusion model. First, a multi-view image aligned with the human body is generated using the multi-view image diffusion model. Then, these images are transformed into 3D assets based on the 3D diffusion model. Through physical simulation and optimization, the generated assets naturally conform to the human body, avoiding penetration issues.

BAG's main functions

• Multi-view image generation: Using a multi-view image diffusion model, multi-view images aligned with the human body are generated based on the shape and posture information of the human body.
• 3D asset generationInput multi-view images into a 3D diffusion model to generate high-quality 3D wearable assets.
• Diversity and PersonalizationIt supports the generation of everything from a single asset to combinations of multiple components, meeting diverse needs in different scenarios.
• High-efficiency adaptationIt can quickly generate assets that adapt to different human body models, and is suitable for virtual clothing try-on, game development, VR and AR fields.

BAG's technical principles

• View Image Diffusion ModelBAG first constructs a diffusion model from a single image to a consistent multi-view image, trained on the large-scale Objaverse dataset to achieve diversity and generalization capabilities. The model is guided by a ControlNet to generate multi-view images aligned with the human body. The control signals utilize the multi-view 2D projection of the target human body, where pixel values represent the XYZ coordinates of the human body surface in normalized space.
• 3D shape generationThe generated multi-view images are input into the local 3D diffusion model to generate the 3D shape of the asset. Similarity transformation (Sim3) is recovered under multi-view contour supervision to ensure that the generated 3D asset is spatially aligned with the human model.
• Physics Simulation and OptimizationTo ensure that the generated 3D assets fit the human body naturally, BAG uses physical simulation technology to solve the problem of penetration between the assets and the body. By preserving the basic geometry of the assets through proxy meshes as a representative of the cloth simulation, a non-penetrating alignment is ultimately achieved.

BAG's project address

• Project official website:https://bag-3d.github.io/
• arXiv technical paper:https://arxiv.org/pdf/2501.16177

Application scenarios of BAG

• Virtual clothing try-onBAG can quickly generate suitable 3D clothing and accessories based on the user's body model. Users can try on different styles and designs of clothing in real time in a virtual environment, improving the shopping experience and reducing the return rate due to unsuitable sizes.
• Game developmentIn game development, BAG can be used to generate diverse clothing and accessories for game characters, supporting personalized customization and enriching the player's character experience.
• Virtual Reality (VR) and Augmented Reality (AR)BAG-generated 3D assets can be seamlessly integrated into VR and AR applications, providing users with an immersive clothing try-on and design experience, allowing them to try on different styles of clothing in a virtual showroom.
• Fashion DesignDesigners can use BAG to quickly generate and iterate 3D clothing models, accelerate the design process, preview design effects in a virtual environment, and reduce design costs.