InvSR - An open-source image super-resolution model for high-resolution restoration of old photos.
InvSR is an innovative image super-resolution model that recovers high-resolution images based on the inverse process of a diffusion model. It improves super-resolution performance by leveraging rich image priors from large pre-trained diffusion models. The core of InvSR lies in depth...
What is InvSR?
InvSR is an innovative image super-resolution model that recovers high-resolution images based on the inverse process of a diffusion model. It improves super-resolution performance by leveraging rich image priors from a large pre-trained diffusion model. At the heart of InvSR lies a deep noise predictor that estimates the optimal noise map required during the forward diffusion process. This method fully utilizes the prior knowledge of the diffusion model and provides a flexible and efficient sampling mechanism, allowing users to freely adjust the sampling steps according to the type of image degradation or specific needs.
Main functions of InvSR
- Image super-resolution enhancementIts main function is to improve the resolution of images, recovering high-resolution images from low-resolution images.
- Flexible sampling mechanismIt supports any number of sampling steps from one to five, and allows users to adjust the sampling steps according to the type of image degradation or specific needs.
- Noise predictionThe depth noise predictor estimates the optimal noise map and is used in the sampling process to initialize the diffusion model.
- diffusion model utilizes: Fully utilize the prior image knowledge in the pre-trained diffusion model to improve the performance of super-resolution.
- High-efficiency computingEven with fewer sampling steps, it can maintain or surpass the performance of existing methods, thereby improving computational efficiency.
InvSR's technical principles
- Diffusion Model FrameworkBased on diffusion models (such as the Denoising Diffusion Probabilistic Model, DDPM), noise is gradually added to transform high-resolution images into random noise maps of low-resolution images.
- Partial noise prediction strategy: Construct intermediate states of the diffusion model as starting sampling points to simplify the inversion process from low resolution to high resolution.
- Deep noise predictorOne of the core components, used to estimate the optimal noise map in the forward diffusion process, thereby partially initializing the sampling process on the diffusion trajectory.
- Reverse diffusion processStarting with a noisy image, noise is gradually removed through a reverse diffusion process to recover a high-resolution image.
- Flexible sampling steps: I select different starting sampling points and combine them with existing sampling algorithms to achieve different numbers of sampling steps, adapting to different image degradation situations.
- Training and optimizationThe noise predictor is trained by minimizing the difference between the predicted high-resolution image and the real high-resolution image, and then trained with L2 loss, LPIPS loss and GAN loss to optimize performance.
InvSR's project address
- GitHub repository:https://github.com/zsyOAOA/InvSR
- arXiv technical paper:https://arxiv.org/pdf/2412.09013
- Experience the demo online:https://huggingface.co/spaces/OAOA/InvSR
Application scenarios of InvSR
- Digital Media Restoration and EnhancementImprove the resolution of old photos, historical documents, or damaged images to help restore and preserve cultural heritage.
- Video surveillance and securityImprove the clarity of low-resolution videos or images captured by surveillance cameras, and enhance the recognition capabilities of security monitoring systems.
- Medical ImagingIn the field of medical imaging, such as MRI or CT scans, improving image resolution can help doctors diagnose conditions more accurately.
- Satellite and aerial photographyImprove the resolution of satellite imagery for use in Geographic Information Systems (GIS), environmental monitoring, and urban planning.
- Mobile devices and photographyOn smartphones and other mobile devices, improve the resolution of photos taken to enhance the user experience.