LuminaBrush - An AI light source drawing tool that allows you to automatically generate lighting effects from hand-drawn light and shadow lines.

What is LuminaBrush?

LuminaBrush is an interactive tool for drawing lighting effects on images. Based on the Flux image processing project, LuminaBrush uses a two-stage approach: the first stage transforms the image into a "uniformly lit" appearance, and the second stage generates specific lighting effects based on the user's drawing. This two-stage approach simplifies the learning process and avoids complex light transport constraints. LuminaBrush is trained using synthetic random normals and relit images, enabling it to handle details such as skin texture and hair.

LuminaBrush's main functions

• Flexible lighting adjustmentUsers can adjust the intensity, direction, and color of the light in real time to achieve a variety of effects, from soft natural light to dramatic stage lighting.
• High-quality image processingIt can handle complex image details, such as skin texture, hair, and fur.
• Interactive User ExperienceIt provides an interactive interface, allowing users to preview and modify lighting effects in real time through simple operations (such as drawing and adjusting with sliders), thereby improving creative efficiency.

The technical principles of LuminaBrush

• Two-phase processing framework:
  • Phase 1 Uniform Illumination ExtractionThis method uses a deep learning model to transform the input image into a "uniformly illuminated" state. The goal is to remove the original lighting information from the image and extract the base image illuminated by uniform lighting.
  • Second stage of lighting effects generationBased on user-provided doodles or markings, the model generates specific lighting effects (such as shadows and highlights). Guided by user input, the model combines deep learning to generate lighting changes that match the user's intent.
• Deep learning modelsThis approach uses deep learning techniques and an architecture based on diffusion models (such as Flux). It is trained on a large amount of image data to learn how to extract the appearance of "uniform lighting" from any input image and generate reasonable lighting effects.
• User interaction mechanismThis model generates lighting effects based on simple doodles or markings. It uses user input as guidance and combines deep learning to generate lighting changes that match the user's intent.

LuminaBrush project address

• GitHub repository:https://github.com/lllyasviel/LuminaBrush
• Experience the demo online:https://huggingface.co/spaces/lllyasviel/LuminaBrush

Application scenarios of LuminaBrush

• Digital art creationEnhance the lighting and shadow effects of the artwork and improve its artistic expression.
• Game DesignAdd dynamic lighting to characters and scenes to enhance immersion.
• Post-production of film and televisionAdjust the lighting and atmosphere of the scene to enhance the visual effect.
• Advertising designOptimize product image lighting and shadows to enhance attractiveness.
• Education and Training: To assist teaching and help students understand lighting and image processing techniques.