skip to main content
10.1145/3373509.3373547acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

Image Super-Resolution and Deblurring Using Generative Adversarial Network

Published: 25 March 2020 Publication History

Abstract

Image super-resolution and deblurring are two highly ill-posed problems that are usually dealt separately. However, real-world images are often low-resolution and have complex blurring. This paper focuses on ordinary natural scene images and reconstructs clear high-resolution images directly from blurred low-resolution inputs. Firstly, we propose a model based on generative adversarial network to jointly process image super-resolution and non-uniform motion deblurring. Secondly, we decouple this joint problem into feature extraction module, super-resolution reconstruction module and deblurring module. The modules promote each other and reconstruct clearer high-resolution images. Finally, we use bilinear interpolation followed by a convolutional layer to achieve upsampling instead of using the common deconvolution layer, which effectively suppresses checkerboard artifacts. The experimental results show that the proposed method is efficient and can perform better than the existing advanced algorithms in both quantitative and qualitative performance.

References

[1]
Ledig, C., Theis, L., Husza´r, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super resolution using a generative adversarial network. In: CVPR.(2017)
[2]
Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jiri Matas. DeblurGAN: Blind motion deblurring using conditional adversarial networks. In CVPR, 2018.
[3]
Aly H, Dubois E. Regularized image up-sampling using a new observation model and the level set method. In: Proceedings of the 2003 International Conference on Image Processing. Barcelona, Spain: IEEE, 2003, 2: Article No.III665-8
[4]
Aly H A, Dubois E. Image up-sampling using total-variation regularization with a new observation model. IEEE Transactions on Image Processing, 2005, 14(10): 1647--1659
[5]
Li X G, Lam K M, Qiu G P, Shen L S, Wang S Y. Examplebased image super-resolution with class-specific predictors. Journal of Visual Communication and Image Representation, 2009, 20(5): 312--322
[6]
Su C Y, Zhuang Y T, Li H, Wu F. Steerable pyramidbased face hallucination. Pattern Recognition, 2005, 38(6): 813--824
[7]
Jiji C V, Chaudhuri S. Single-frame image super-resolution through contourlet learning. EURASIP Journal on Advances in Signal Processing, 2006, 2006: Article No.073767
[8]
Chao Dong, Chen Change Loy, Kaiming He, and Xiao ouTang. Image super-resolution using deep convolutional networks. TPAMI, 38(2):295--307, 2016.
[9]
Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. Accurate image super-resolution using very deep convolutional networks. In CVPR, 2016.
[10]
Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR, 2017.
[11]
Uwe Schmidt, Kevin Schelten, and Stefan Roth. Bayesian deblurring with integrated noiseestimation. In CVPR, 2011.
[12]
Uwe Schmidt, Carsten Rother, Sebastian Nowozin, Jeremy Jancsary, and Stefan Roth. Discriminative non-blind deblurring. In CVPR, 2013.
[13]
Li Xu, Shicheng Zheng, and Jiaya Jia. Unnatural l0 sparse representation for natural image deblurring. In CVPR, 2013.
[14]
A. Chakrabarti. A neural approach to blind motion deblurring. In ECCV, 2016. 1, 2
[15]
Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In CVPR, 2017.
[16]
T M Nimisha, Akash K Singh, and Ambasamudram N Rajagopalan. Blur-invariant deep learning for blind-deblurring. In ICCV, 2017.
[17]
C. Liu and D. Sun. A bayesian approach to adaptive video super resolution. In CVPR, 2011. 2
[18]
Xu X, Sun D, Pan J, et al. Learning to Super-Resolve Blurry Face and Text Images[C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, 2017.
[19]
Xinyi Zhang, Fei Wang, Hang Dong, and Yu Guo. A deep encoder-decoder networks for joint deblurring and super-resolution. In ICASSP, 2018
[20]
Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Nets[C]//International Conference on Neural Information Processing Systems. MIT Press, 2014.
[21]
He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition [J]. 2015.
[22]
AUGUSTUS ODENA, VINCENT DUMOULIN, CHRIS OLAH. Deconvolution and Checkerboard Artifacts. https://distill.pub/2016/deconv-checkerboard.
[23]
Wang X, Yu K, Wu S, et al. ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks[C]//In ECCV.2018.
[24]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution[C]//In ECCV.2016.
[25]
He K, Zhang X, Ren S, et al.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp. 1026--1034. (2015)
[26]
BeeLim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. In CVPR Workshops, 2017.
[27]
Tao X, Gao H, Wang Y, et al. Scale-recurrent Network for Deep Image Deblurring[C]//In CVPR.2018.

Cited By

View all
  • (2023)Deep architecture for super-resolution and deblurring of text imagesMultimedia Tools and Applications10.1007/s11042-023-15340-x83:2(3945-3961)Online publication date: 20-May-2023
  • (2022)Research and Application of GAN-GRU in Bridge Monitoring Data Cleaning2022 China Automation Congress (CAC)10.1109/CAC57257.2022.10054752(3245-3250)Online publication date: 25-Nov-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICCPR '19: Proceedings of the 2019 8th International Conference on Computing and Pattern Recognition
October 2019
522 pages
ISBN:9781450376570
DOI:10.1145/3373509
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • Hebei University of Technology
  • Beijing University of Technology

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 March 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Generative Adversarial Network (GAN)
  2. Super-resolution
  3. deblurring
  4. residual learning

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICCPR '19

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)1
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Deep architecture for super-resolution and deblurring of text imagesMultimedia Tools and Applications10.1007/s11042-023-15340-x83:2(3945-3961)Online publication date: 20-May-2023
  • (2022)Research and Application of GAN-GRU in Bridge Monitoring Data Cleaning2022 China Automation Congress (CAC)10.1109/CAC57257.2022.10054752(3245-3250)Online publication date: 25-Nov-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media