Skip to main content
SpringerLink
Log in

A texture detail-oriented generative adversarial network: motion deblurring for multi-textured images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The key to motion deblurring in multi-textured images is to extract more accurate feature information from the original input images. Recent studies on the design of feature extraction models have demonstrated the remarkable effectiveness of hierarchically representing multi-scale features to improve model performance. Nevertheless, these approaches generally neglect the size of the receptive field in each layer, which is essential for ensuring the full exchange of information. This paper proposes two feature extraction methods based on lightweight networks that can be flexibly plugged into networks. In addition, a novel model is formed by embedding lighter and more effective feature extraction methods (e.g., our symmetrical depthwise convolution feature extraction block (SDB) and multi-scale iterative channel feature extraction block (MIB)) into a generative adversarial network (GAN), and we name this model the texture detail-oriented GAN (TDGAN). To make the generated image closer to the target image at the visual level, we also integrate the perceptual style loss and the structural similarity loss into the generator loss function. Extensive experiments conducted on GoPro and AssIMG datasets demonstrate that the proposed model outperforms the state-of-the-art methods in terms of accuracy and computational complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Chen L, Fang F, Lei S, Li F, Zhang G (2020) Enhanced sparse model for blind deblurring. In: European conference on computer vision. Springer, pp 631–646

  2. Liu J, Yan M, Zeng T (2021) Surface-aware blind image deblurring. IEEE Trans Pattern Anal Mach Intell 43(3):1041–1055

    Article  Google Scholar 

  3. Sun J, Cao W, Xu Z, Ponce J (2015) Learning a convolutional neural network for non-uniform motion blur removal. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 769–777

  4. Nah S, Kim TH, Lee KM (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3883–3891

  5. Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8174–8182

  6. Shi X, Chen Z, Wang H, Yeung D-Y, Wong W-K, Woo W-C (2015) Convolutional lstm network: a machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems, pp 802–810

  7. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun ACM 63(11):139–144

    Article  MathSciNet  Google Scholar 

  8. Park S, Shin Y-G (2021) Generative residual block for image generation. Appl Intell 1–10

  9. Ramakrishnan S, Pachori S, Gangopadhyay A, Raman S (2017) Deep generative filter for motion deblurring. In: Proceedings of the IEEE international conference on computer vision workshops, pp 2993–3000

  10. Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8183–8192

  11. Zhou L, Min W, Lin D, Han Q, Liu R (2020) Detecting motion blurred vehicle logo in iov using filter-deblurgan and vl-yolo. IEEE Trans Veh Technol 69(4):3604–3614

    Article  Google Scholar 

  12. Sajjadi MSM, Scholkopf B, Hirsch M (2017) Enhancenet: single image super-resolution through automated texture synthesis. In: Proceedings of the IEEE international conference on computer vision, pp 4491–4500

  13. Shen Z, Wang W, Lu X, Shen J, Ling H, Xu T, Shao L (2019) Human-aware motion deblurring. In: Proceedings of the IEEE international conference on computer vision, pp 5572–5581

  14. Xu Y, Zhu Y, Quan Y, Ji H (2021) Attentive deep network for blind motion deblurring on dynamic scenes. Comput Vis Image Underst 205:103169

    Article  Google Scholar 

  15. Wu J, Yu X, Liu D, Chandraker M, Wang Z (2020) David: dual-attentional video deblurring. In: Proceedings of the IEEE Winter conference on applications of computer vision, pp 2376–2385

  16. Cho S-J, Ji S-W, Hong J-P, Jung S-W, Ko S-J (2021) Rethinking coarse-to-fine approach in single image deblurring. arXiv:2108.05054

  17. Tran HTM, Ho-Phuoc T (2019) Deep laplacian pyramid network for text images super-resolution. In: 2019 IEEE-RIVF International conference on computing and communication technologies (RIVF). IEEE, pp 1–6

  18. Chen J, Li B, Xue X (2021) Scene text telescope: text-focused scene image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 12026–12035

  19. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein gan

  20. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A (2017) Improved training of wasserstein gans. In: Proceedings of the 31st international conference on neural information processing systems, pp 5769–5779

  21. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4401–4410

  22. Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020) Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8110–8119

  23. Hou H, Huo J, Wu J, Lai Y-K, Gao Y (2021) Mw-gan: multi-warping gan for caricature generation with multi-style geometric exaggeration. IEEE Trans Image Process 30:8644–8657

    Article  Google Scholar 

  24. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  25. Kuang P, Ma T, Chen Z, Li F (2019) Image super-resolution with densely connected convolutional networks. Appl Intell 49(1):125–136

    Article  Google Scholar 

  26. Shang T, Dai Q, Zhu S, Yang T, Guo Y (2020) Perceptual extreme super-resolution network with receptive field block. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 440–441

  27. Maeda S (2020) Unpaired image super-resolution using pseudo-supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 291–300

  28. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Loy CC (2018) Esrgan: enhanced super-resolution generative adversarial networks. In: European conference on computer vision. Springer, pp 63–79

  29. Howard A, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861

  30. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520

  31. Haase D, Amthor M (2020) Rethinking depthwise separable convolutions: how intra-kernel correlations lead to improved mobilenets. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 14600–14609

  32. Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6848–6856

  33. Ma N, Zhang X, Zheng H-T, Sun J (2018) Shufflenet v2: practical guidelines for efficient cnn architecture design. In: Proceedings of the European conference on computer vision (ECCV), pp 116–131

  34. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

  35. Ding X, Guo Y, Ding G, Han J (2019) Acnet: strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks. In: Proceedings of the IEEE international conference on computer vision, pp 1911–1920

  36. Yu C, Xiao B, Gao C, Lu Y, Zhang L, Sang N, Wang J (2021) Lite-hrnet: a lightweight high-resolution network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 10440–10450

  37. Wang L, Yin B, Guo A, Ma H, Cao J (2018) Skip-connection convolutional neural network for still image crowd counting. Appl Intell 48(10):3360–3371

    Article  Google Scholar 

  38. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  39. Qiao J, Song H, Zhang K, Zhang X (2021) Conditional generative adversarial network with densely-connected residual learning for single image super-resolution. Multimed Tools Appl 80(3):4383–4397

    Article  Google Scholar 

  40. Mehta S, Rastegari M, Caspi A, Shapiro L, Hajishirzi H (2018) Espnet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: European conference on computer vision. Springer, pp 561–580

  41. Ma L, Li H, Meng F, Wu Q, Ngan KN (2018) Global and local semantics-preserving based deep hashing for cross-modal retrieval. Neurocomputing 312:49–62

    Article  Google Scholar 

  42. Ma L, Li H, Meng F, Wu Q, Ngan KN (2020) Discriminative deep metric learning for asymmetric discrete hashing. Neurocomputing 380:115–124

    Article  Google Scholar 

  43. Ma L, Li X, Yu S, Wu J, Zhang Y (2020) Correlation filtering-based hashing for fine-grained image retrieval. IEEE Signal Process Lett 27:2129–2133

    Article  Google Scholar 

  44. Ma L, Li X, Yu S, Huang L, Huang Z, Wu J (2021) Learning discrete class-specific prototypes for deep semantic hashing. Neurocomputing 443:85–95

    Article  Google Scholar 

  45. He K, Zhang X, Ren S, Sun J. (2016) IEEE 2016 IEEE conference on computer vision and pattern recognition (cvpr)—Las Vegas, NV, USA (2016.6.27–2016.6.30)]. In: 2016 IEEE conference on computer vision and pattern recognition (cvpr)—deep residual learning for image recognition, pp 770–778

  46. Xu X, Sun D, Pan J, Zhang Y, Pfister H, Yang M-H (2017) Learning to super-resolve blurry face and text images. In: Proceedings of the IEEE international conference on computer vision, pp 251–260

  47. Park D, Kang DU, Kim J, Se YC (2020) Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In: European conference on computer vision. Springer, pp 327–343

  48. Zhang H, Dai Y, Li H, Koniusz P (2019) Deep stacked hierarchical multi-patch network for image deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5978–5986

  49. Zamir S W, Arora A, Khan S, Hayat M, Khan FS, Yang M-H, Shao L (2021) Multi-stage progressive image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 14821–14831

Download references

Acknowledgements

The authors of this paper are supported by the funding of Natural Science Foundation of China (No: 61662006, 62062015) and the Innovation Project of School of Computer Science and Information Engineering, Guangxi Normal University under the contract number JXXYYJSCXXM-002 and Guangxi Province 100 oversea talent plan.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Guangxi Normal University, Guilin City, China

    Xiao Zhang, Ming Chen, Zhengqin Zhang & Shenglian Lu

Authors
  1. Xiao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengqin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shenglian Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ming Chen.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Chen, M., Zhang, Z. et al. A texture detail-oriented generative adversarial network: motion deblurring for multi-textured images. Appl Intell 53, 3255–3272 (2023). https://doi.org/10.1007/s10489-022-03628-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03628-8

Keywords

Search

Navigation