Yixing Ji
ABSTRACT
The non-local regression prior has shown promising results in image deblurring by effectively combining local continuity and non-local self-similarity of images. However, traditional non-local regression priors have limited representation capacity, and existing non-local regression-based methods often require iterative solutions to complex optimization problem. Additionally, these methods involve manually tuning hyper-parameters, which can be difficult and time-consuming. To address these limitations, we propose an adaptive non-local regression prior based on the Transformer model for image deblurring. Different from other learning-based approaches, the self-attention mechanism in the Transformer module better aligns with the modeling process of non-local regression prior, and the global computation capability of Transformer enables effective modeling of global context information. By unfolding the iterative process of the model into a neural network, we introduce a novel deep unfolding network. The proposed network is trained end-to-end, allowing all parameters to be jointly optimized to facilitate image restoration. The extensive experiments show that our proposed model outperforms state-of-the-art model-based and learning-based methods both in terms of PSNR metrics and visual quality.
- Eirikur Agustsson and Radu Timofte. 2017. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 126–135.Google Scholar
Cross Ref
- Antoni Buades, Bartomeu Coll, and J-M Morel. 2005. A non-local algorithm for image denoising. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), Vol. 2. 60–65.Google Scholar
- Pierre Charbonnier, Laure Blanc-Feraud, Gilles Aubert, and Michel Barlaud. 1994. Two deterministic half-quadratic regularization algorithms for computed imaging. In Proceedings of 1st international conference on image processing, Vol. 2. 168–172.Google ScholarCross Ref
- Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. 2007. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on image processing 16, 8 (2007), 2080–2095.Google ScholarCross Ref
- Aram Danielyan, Vladimir Katkovnik, and Karen Egiazarian. 2011. BM3D frames and variational image deblurring. IEEE Transactions on image processing 21, 4 (2011), 1715–1728.Google ScholarDigital Library
- Jiangxin Dong, Stefan Roth, and Bernt Schiele. 2020. Deep wiener deconvolution: Wiener meets deep learning for image deblurring. Advances in Neural Information Processing Systems 33 (2020), 1048–1059.Google Scholar
- Weisheng Dong, Peiyao Wang, Wotao Yin, Guangming Shi, Fangfang Wu, and Xiaotong Lu. 2018. Denoising prior driven deep neural network for image restoration. IEEE transactions on pattern analysis and machine intelligence 41, 10 (2018), 2305–2318.Google Scholar
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).Google Scholar
- Michael Elad and Michal Aharon. 2006. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image processing 15, 12 (2006), 3736–3745.Google ScholarDigital Library
- Dong Gong, Zhen Zhang, Qinfeng Shi, Anton van den Hengel, Chunhua Shen, and Yanning Zhang. 2020. Learning deep gradient descent optimization for image deconvolution. IEEE transactions on neural networks and learning systems 31, 12 (2020), 5468–5482.Google Scholar
- Shuhang Gu, Lei Zhang, Wangmeng Zuo, and Xiangchu Feng. 2014. Weighted nuclear norm minimization with application to image denoising. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2862–2869.Google ScholarDigital Library
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Anat Levin, Yair Weiss, Fredo Durand, and William T Freeman. 2009. Understanding and evaluating blind deconvolution algorithms. In 2009 IEEE conference on computer vision and pattern recognition. 1964–1971.Google ScholarCross Ref
- Anat Levin, Yair Weiss, Fredo Durand, and William T Freeman. 2011. Efficient marginal likelihood optimization in blind deconvolution. In CVPR 2011. 2657–2664.Google ScholarDigital Library
- Yawei Li, Kai Zhang, Jiezhang Cao, Radu Timofte, and Luc Van Gool. 2021. Localvit: Bringing locality to vision transformers. arXiv preprint arXiv:2104.05707 (2021).Google Scholar
- Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision. 1833–1844.Google ScholarCross Ref
- Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision. 1833–1844.Google ScholarCross Ref
- Licheng Liu, CL Philip Chen, Xinge You, Yuan Yan Tang, Yushu Zhang, and Shutao Li. 2017. Mixed noise removal via robust constrained sparse representation. IEEE Transactions on Circuits and Systems for Video Technology 28, 9 (2017), 2177–2189.Google ScholarDigital Library
- Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022.Google ScholarCross Ref
- Julien Mairal, Francis Bach, Jean Ponce, Guillermo Sapiro, and Andrew Zisserman. 2009. Non-local sparse models for image restoration. In 2009 IEEE 12th international conference on computer vision. 2272–2279.Google ScholarCross Ref
- David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, Vol. 2. 416–423.Google ScholarCross Ref
- Max Mignotte. 2008. A non-local regularization strategy for image deconvolution. Pattern recognition letters 29, 16 (2008), 2206–2212.Google Scholar
- Chao Ren, Xiaohai He, and Truong Q Nguyen. 2018. Adjusted non-local regression and directional smoothness for image restoration. IEEE Transactions on Multimedia 21, 3 (2018), 731–745.Google ScholarCross Ref
- Chao Ren, Xiaohai He, Chuncheng Wang, and Zhibo Zhao. 2021. Adaptive consistency prior based deep network for image denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8596–8606.Google ScholarCross Ref
- Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4510–4520.Google ScholarCross Ref
- Uwe Schmidt, Jeremy Jancsary, Sebastian Nowozin, Stefan Roth, and Carsten Rother. 2015. Cascades of regression tree fields for image restoration. IEEE transactions on pattern analysis and machine intelligence 38, 4 (2015), 677–689.Google Scholar
- Radu Timofte, Eirikur Agustsson, Luc Van Gool, Ming-Hsuan Yang, and Lei Zhang. 2017. Ntire 2017 challenge on single image super-resolution: Methods and results. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 114–125.Google ScholarCross Ref
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).Google Scholar
- Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. 2022. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 17683–17693.Google ScholarCross Ref
- Jingke Yan, Xin Wang, Jingye Cai, Qin Qin, Hao Yang, Qin Wang, Yao Cheng, Tian Gan, Hua Jiang, Jianhua Deng, 2022. Medical image segmentation model based on triple gate MultiLayer perceptron. Scientific Reports 12, 1 (2022), 6103.Google ScholarCross Ref
- Kai Zhang, Luc Van Gool, and Radu Timofte. 2020. Deep unfolding network for image super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3217–3226.Google ScholarCross Ref
- Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Luc Van Gool, and Radu Timofte. 2021. Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 10 (2021), 6360–6376.Google ScholarDigital Library
- Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang. 2017. Learning deep CNN denoiser prior for image restoration. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3929–3938.Google ScholarCross Ref
- Junmei Zhong and Huifang Sun. 2018. Edge-preserving image denoising based on orthogonal wavelet transform and level sets. Journal of Image and Graphics 6, 2 (2018), 145–151.Google ScholarCross Ref
- Daniel Zoran and Yair Weiss. 2011. From learning models of natural image patches to whole image restoration. In 2011 international conference on computer vision. 479–486.Google ScholarDigital Library
Index Terms
- Adaptive Non-Local Regression Prior based on Transformer for Image Deblurring
Recommendations
Image region driven prior selection for image deblurring
AbstractDeblurring an image has been a long researched problem. This problem is very complex due to the lack of sufficient information about the blur parameters. Image deblurring is important in applications such as remote sensing where the same image ...
Multi-Scale Image Deblurring Based on Local Region Selection and Image Block Classification
WCNA 2017: Proceedings of the 2017 International Conference on Wireless Communications, Networking and ApplicationsA multi-scale image deblurring method based on local region selection and image block classification is proposed in this paper. Firstly, optimal local region is automatically chosen from the original blurred image and built to pyramid images. Then, the ...
Blind image deblurring using elastic-net based rank prior
We propose a novel image prior by exploiting similar patches for image deblurring.We quantitatively verify that it favors clear images over blurred images.An efficient numerical optimization method is developed to solve the proposed model.The prior is ...
Comments