Ev-DeblurVSR - A video enhancement model developed by USTC and other institutions.
Ev-DeblurVSR is a video enhancement model jointly developed by the University of Science and Technology of China, the Institute of Artificial Intelligence at the Hefei National Science Center, and the National University of Singapore. It can recover video images from low-resolution and blurry video input...
What is Ev-DeblurVSR?
Ev-DeblurVSR is a video enhancement model jointly developed by the University of Science and Technology of China, the Institute of Artificial Intelligence at the Hefei National Science Center, and the National University of Singapore. It can recover high-resolution, clear video from low-resolution and blurry video input. Ev-DeblurVSR incorporates event signals captured by event cameras, enhancing video deblurring and super-resolution performance with high temporal resolution and rich motion information. The core of Ev-DeblurVSR includes a Reciprocal Feature Deblurring (RFD) module and a Hybrid Deformable Alignment (HDA) module, used for intra-frame deblurring and inter-frame alignment, respectively. Ev-DeblurVSR performs exceptionally well on multiple datasets, outperforming existing methods, especially on real-world data, demonstrating higher accuracy and faster processing speed.
Main functions of Ev-DeblurVSR
- Blurry video super-resolution processingIt restores low-resolution and blurry video frames to high-resolution and clear images.
- Based on event camera data assistanceBy leveraging the high temporal resolution and brightness variation information captured by the event camera, lost motion details and textures can be recovered.
- Anti-motion blur and high dynamic rangeIt effectively reduces motion blur while preserving details in high-contrast areas, making it suitable for both motion scenes and low-light scenes.
- Provide development and testing supportIt is open source code, providing pre-trained models, test datasets, and scripts to help developers get started quickly and verify the results.
The technical principle of Ev-DeblurVSR
- Classification and utilization of event signals:
- Intra-frame events: Records motion information within the exposure time of a video frame, which is used in deblurring frame features.
- Inter-frame eventsRecords the continuous motion trajectory between frames, used for time alignment.
- Reciprocal feature deblurring moduleThis method deblurs video frame features based on motion information from intra-frame events, feeding back global scene context information from the video frame into the event features to enhance their expressive power. It achieves mutual enhancement of frame and event features based on multi-head channel attention and cross-modal attention mechanisms.
- Hybrid deformable alignment moduleBy combining inter-frame events and optical flow information, and using the continuous motion trajectories of events and the spatial information of optical flow, motion estimation in the deformable alignment process is improved. Alignment is achieved based on two branches: Event-Guided Alignment (EGA) and Flow-Guided Alignment (FGA), and the final feature alignment is completed using a deformable convolutional network (DCN).
- Edge enhancement loss functionThe pixel reconstruction error is weighted based on high-frequency information in the event signal, making the model pay more attention to the restoration of edges and high-frequency regions, thereby improving the detail and clarity of the restored video.
Ev-DeblurVSR project address
- Project official website:https://dachunkai.github.io/ev-deblurvsr.github.io/
- GitHub repository:https://github.com/DachunKai/Ev-DeblurVSR
- arXiv technical paper:https://arxiv.org/pdf/2504.13042
Application scenarios of Ev-DeblurVSR
- Video surveillanceImprove the clarity of blurry surveillance videos and help identify details more accurately.
- Sports eventsClearly reproduces fast-paced action scenes, improving referee decisions and the viewing experience for spectators.
- autonomous drivingTo reduce blurry images captured by cameras and provide clearer environmental perception for autonomous driving systems.
- Film and television production: To fix blurry images caused by poor shooting conditions and improve the quality of film and television works.
- Industrial testingImproves the accuracy of industrial inspection by reducing blurry images caused by high-speed movement or equipment vibration.