LBM - An AI image transformation framework for enabling controllable shadow generation
LBM (Latent Bridge Matching) is a novel image-to-image transformation framework developed by the Jasper Research team. It achieves fast and efficient image transformation by constructing bridge matching in the latent space. LBM requires only a single step of inference...
What is LBM?
LBM (Latent Bridge Matching) is a novel image-to-image transformation framework developed by the Jasper Research team. It achieves fast and efficient image transformation by constructing bridge matching in a latent space. LBM completes the task in a single inference step and is applicable to various image transformation scenarios, such as target removal, relighting, depth and normal estimation, etc. LBM uses Brownian bridges to establish random paths between the source and target images, increasing sample diversity. The conditional framework enables controllable shadow generation and image relighting. LBM achieves or surpasses the state-of-the-art performance of existing methods in multiple tasks, demonstrating strong versatility and efficiency.
Main functions of LBM
- Target removalRemoves a specified object and its associated shadow from an image while preserving the integrity of the background.
- Image relighting: Relight foreground objects based on given background or lighting conditions, removing existing shadows and reflections.
- Image restoration:Convert degraded images into clean images, restoring the original image quality.
- Depth/Normal Map GenerationConverts input images into depth maps or normal maps for use in tasks such as 3D reconstruction.
- Controllable shadow generationIt generates shadows based on the position, color, and intensity of the light source, enhancing the realism of the image.
LBM Technical Principles
- Latent Space EncodingEncoding the source and target images into a low-dimensional latent space reduces computational costs and improves the scalability of the model.
- Brownian BridgeA random path (Brown bridge) is constructed in the latent space to connect the latent representations of the source and target images. This randomness enables the model to generate diverse samples.
- Stochastic differential equations (SDE)The method is based on solving stochastic differential equations (SDEs) to predict the latent representation on the path, thereby realizing the transformation from source image to target image.
- Conditional FrameworkBy introducing additional conditional variables (such as lightmaps), LBM can achieve controllable image relighting and shadow generation.
- Pixel lossDuring training, the model is optimized using pixel loss (such as LPIPS) to ensure that the generated image is visually consistent with the target image.
LBM's project address
- Project official website:https://gojasper.github.io/latent-bridge-matching/
- GitHub repository:https://github.com/gojasper/LBM
- arXiv technical paper:https://arxiv.org/pdf/2503.07535
- Experience the demo online:https://huggingface.co/spaces/jasperai/LBM
Application scenarios of LBM
- Regular usersDaily photo editing: remove unwanted objects, restore old photos, and adjust lighting.
- Photography enthusiastsPost-processing enhances the realism of photos by adding or adjusting shadows.
- Graphic designerCreative design, generating depth/normal maps, and quickly repairing and adjusting images.
- Video EditorVideo editing, including repairing video frames and adjusting object lighting and shadows.
- 3D modelerGenerate depth/normal maps from photos to aid in 3D modeling.