Deep Learning in Motion Deblurring: Current Status, Benchmarks and Future Prospects

🔥🔥 In this review, we have systematically examined over 150 papers 📃📃📃, summarizing and analyzing 🌟more than 30 blind motion deblurring methods.

🔥🔥🔥 Extensive qualitative and quantitative comparisons have been conducted against the current SOTA methods on four datasets, highlighting their limitations and pointing out future research directions.

🔥🔥🔥🔥 The latest deblurring papers of CVPR 2024 have been included~

Fig 1. Overview of deep learning methods for blind motion deblurring.

Content:

1. Related Reviews and Surveys to Deblurring
2. CNN-based Blind Motion Deblurring Models
3. RNN-based Blind Motion Deblurring Models
4. GAN-based Blind Motion Deblurring Models
5. Transformer-based Blind Motion Deblurring Models
6. Diffusion-based Blind Motion Deblurring Models
7. Motion Deblurring Datasets
8. Evaluation
9. Citation

---

0. Related Reviews and Surveys to Deblurring:

🚀🚀🚀Update (in 2023-12-28) 🎈

No.	Year	Pub.	Title	Links
01	2021	CDS	A Survey on Single Image Deblurring	Paper/Project
02	2021	CVIU	Single-image deblurring with neural networks: A comparative survey	Paper/Project
03	2022	IJCV	Deep Image Deblurring: A Survey	Paper/Project
04	2022	arXiv	Blind Image Deblurring: A Review	Paper/Project
05	2023	CVMJ	A survey on facial image deblurring	Paper/Project
06	2023	arXiv	A Comprehensive Survey on Deep Neural Image Deblurring	Paper/Project

---

1. CNN-based Blind Motion Deblurring Models:

🚀🚀🚀Update (in 2024-05-14) 🎈

No.	Year	Model	Pub.	Title	Links
01	2017	DeepDeblur	CVPR	Deep multi-scale convolutional neural network for dynamic scene deblurring	Paper/Project
02	2019	DMPHN	CVPR	Deep stacked hierarchical multi-patch network for image deblurring	Paper/Project
03	2019	PSS-NSC	CVPR	Dynamic scene deblurring with parameter selective sharing and nested skip connections	Paper/Project
04	2020	DGN	TIP	Dynamic scene deblurring by depth guided model	Paper/Project
05	2020	MSCAN	TCSVT	Deep convolutional-neural-network-based channel attention for single image dynamic scene blind deblurring	Paper/Project
06	2021	SDWNet	ICCVW	Sdwnet: A straight dilated network with wavelet transformation for image deblurring	Paper/Project
07	2021		TIP	Deep Outlier Handling for Image Deblurring	Paper/[Project]
08	2021	MIMOU-Net+	ICCV	Rethinking coarse-to-fine approach in single image deblurring	Paper/Project
09	2021	MPRNet	CVPR	Multi-stage progressive image restoration	Paper/Project
10	2022	MSSNet	ECCVW	Mssnet: Multi-scale-stage network for single image deblurring	Paper/Project
11	2022	HINet	CVPRW	Hinet: Half instance normalization network for image restoration	Paper/Project
12	2022	BANet	TIP	Banet: a blur-aware attention network for dynamic scene deblurring	Paper/Project
13	2022	IRNeXt	ICML	Irnext: Rethinking convolutional network design for image restoration	Paper/Project
14	2023	ReLoBlur	AAAI	Real-World Deep Local Motion Deblurring	Paper/Project
15	2023	MRLPFNet	ICCV	Multi-scale Residual Low-Pass Filter Network for Image Deblurring	Paper/[Project]
16	2023	MSFS-FNet	TCSVT	Multi-Scale Frequency Separation Network for Image Deblurring	Paper/Project

2. RNN-based Blind Motion Deblurring Models:

🚀🚀🚀Update (in 2024-05-14) 🎈

No.	Year	Model	Pub.	Title	Links
01	2018	SVRNN	CVPR	Dynamic scene deblurring using spatially variant recurrent neural networks	Paper/Project
02	2018	SRN	CVPR	Scale-recurrent network for deep image deblurring	Paper/Project
03	2022		TCSVT	Deep Dynamic Scene Deblurring From Optical Flow	Paper/[Project]
04	2023	MT-RNN	ECCV	Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training	Paper/Project

3. GAN-based Blind Motion Deblurring Models:

🚀🚀🚀Update (in 2024-05-14) 🎈

No.	Year	Model	Pub.	Title	Links
01	2018	DeblurGAN	CVPR	Deblurgan: Blind motion deblurring using conditional adversarial networks	Paper/Project
02	2019	DeblurGAN-V2	ICCV	Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better	Paper/Project
03	2020	DBGAN	CVPR	Distribution-induced Bidirectional GAN for Graph Representation Learning	Paper/Project
04	2021	CycleGAN	ICCV	Unpaired image-to-image translation using cycle-consistent adversarial networks	Paper/Project
05	2021		TPAMI	Physics-Based Generative Adversarial Models for Image Restoration and Beyond	Paper/[Project]
06	2022	FCLGAN	ACM	Unpaired image-to-image translation using cycle-consistent adversarial networks	Paper/Project
07	2022	Ghost-DeblurGAN	IROS	Application of Ghost-DeblurGAN to Fiducial Marker Detection	Paper/Project

4. Transformer-based Blind Motion Deblurring Models:

🚀🚀🚀Update (in 2024-05-14) 🎈

No.	Year	Model	Pub.	Title	Links
01	2021	Uformer	CVPR	Uformer: A general u-shaped transformer for image restoration	Paper/Project
02	2022	Restormer	CVPR	Restormer: Efficient transformer for high-resolution image restoration	Paper/Project
03	2022	Stripformer	ECCV	Stripformer: Strip transformer for fast image deblurring	Paper/Project
04	2022	Stoformer	NeurIPS	Stochastic Window Transformer for Image Restoration	Paper/Project
05	2023	Sharpformer	TIP	SharpFormer: Learning Local Feature Preserving Global Representations for Image Deblurring	Paper/Project
06	2023	FFTformer	CVPR	Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring	Paper/Project
07	2023	BiT	CVPR	Blur Interpolation Transformer for Real-World Motion from Blur	Paper/Project
08	2024		CVPR	Efficient Multi-scale Network with Learnable Discrete Wavelet Transform for Blind Motion Debluring	[Paper]/[Project]
09	2024		TNNLS	Image Deblurring by Exploring In-Depth Properties of Transformer	[Paper]/Project

5. Diffusion-based Blind Motion Deblurring Models:

🚀🚀🚀Update (in 2024-05-14) 🎈

No.	Year	Model	Pub.	Title	Links
01	2023		ICCV	Multiscale Structure Guided Diffusion for Image Deblurring	Paper/[Project]
02	2024	ID-Blau	CVPR	ID-Blau: Image Deblurring by Implicit Diffusion-based reBLurring AUgmentation	Paper/[Project]
03	2024		CVPR	Fourier Priors-Guided Diffusion for Zero-Shot Joint Low-Light Enhancement and Deblurring	[Paper]/[Project]

---

6. Motion Deblurring Datasets:

🚀🚀🚀Update (in 2024-01-08) 🎈

No.	Dataset	Year	Pub.	Size	Types	Train/Val/Test	Download
01	Köhler at al.	2012	ECCV	4 sharp, 48 blur	Synthetic	-	link
02	GoPro	2017	CVPR	3214	Synthetic	2103/0/1111	link
03	HIDE	2019	CVPR	8422	Synthetic	6397/0/2025	link
04	Blur-DVS	2020	CVPR	13358	Real	8878/1120/3360	[link]
05	RealBlur	2020	ECCV	4738	Real	3758/0/980	link
06	RsBlur	2022	ECCV	13358	Real	8878/1120/3360	link
07	ReLoBlur	2023	AAAI	2405	Real	2010/0/395	link

---

7. Evaluation:

• For evaluation on GoPro results in MATLAB, modify './out/...' to the corresponding path

evaluation_GoPro.m

• For evaluation on HIDE results in MATLAB, modify './out/...' to the corresponding path

evaluation_HIDE.m

• For evaluation on RealBlur_J results, modify './out/...' to the corresponding path

python evaluate_RealBlur_J.py

• For evaluation on RealBlur_R results, modify './out/...' to the corresponding path

python evaluate_RealBlur_R.py

---

Citation:

If you find our survey paper and evaluation code are useful, please cite the following paper:

@article{xiang2024application,
      title={Application of Deep Learning in Blind Motion Deblurring: Current Status and Future Prospects}, 
      author={Yawen Xiang and Heng Zhou and Chengyang Li and Fangwei Sun and Zhongbo Li and Yongqiang Xie},
      year={2024},
      journal={arXiv preprint arXiv:2401.05055},
}

---

👏👏👏 Thanks to the above authors for their excellent work！