Skip to main content
SpringerLink
Log in

Image super-resolution reconstruction based on feature map attention mechanism

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

To improve the issue of low-frequency and high-frequency components from feature maps being treated equally in existing image super-resolution reconstruction methods, the paper proposed an image super-resolution reconstruction method using attention mechanism with feature map to facilitate reconstruction from original low-resolution images to multi-scale super-resolution images. The proposed model consists of a feature extraction block, an information extraction block, and a reconstruction module. Firstly, the extraction block is used to extract useful features from low-resolution images, with multiple information extraction blocks being combined with the feature map attention mechanism and passed between feature channels. Secondly, the interdependence is used to adaptively adjust the channel characteristics to restore more details. Finally, the reconstruction module reforms different scales high-resolution images. The experimental results can demonstrate that the proposed method can effectively improve not only the visual effect of images but also the results on the Set5, Set14, Urban100, and Manga109. The results can demonstrate the proposed method has structurally similarity to the image reconstruction methods. Furthermore, the evaluating indicator of Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) has been improved to a certain degree, while the effectiveness of using feature map attention mechanism in image super-resolution reconstruction applications is useful and effective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238

    Article  Google Scholar 

  2. Chen YT, Tao JJ, Liu LW, Xiong J, Xia RL, Xie JB, Zhang Q, Yang K (2020) Research of improving semantic image segmentation based on a feature fusion model. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-020-02066-z

  3. Chen YT, Wang J, Liu SJ, Chen X, Xiong J, Xie JB, Yang K (2019) The multi-scale fast correlation filtering tracking algorithm based on a features fusion model. Concurrency Computat Pract Exper. https://doi.org/10.1002/cpe.5533

  4. Yang J, Wright J, Huang TS, Yu L (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  5. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. Paper presented at: Proceedings of the 2013 IEEE conference on computer vision (ICCV), Sydney, Australia, pp 1920–1927

    Google Scholar 

  6. Dong C, Loy CC, He KM, Tang XO (2014) Learning a deep convolutional network for image super-resolution. Paper presented at: Proceedings of the European conference on computer vision (ECCV), Zurich, Switzerland, pp 184–199

    Google Scholar 

  7. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. Paper presented at: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, pp 1646–1654

    Google Scholar 

  8. He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. Paper presented at: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, pp 770–778

    Google Scholar 

  9. Chen YT, Xiong J, Xu WH, Zuo JW (2019) A novel online incremental and decremental learning algorithm based on variable support vector machine. Cluster Comput 22(3):7435–7445

  10. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. Paper presented at: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition workshops (CVPR workshops), Honolulu, pp 1132–1140

    Google Scholar 

  11. Tai Y, Yang J, Liu XM (2017) Image super-resolution via deep recursive residual network. Paper presented at: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, pp 2790–2798

    Google Scholar 

  12. Chen YT, Zhang HP, Liu LW, Tao JJ, Zhang Q, Yang K, Xia RL, Xie JB (2020) The image inpainting algorithm of texture decomposition and local variation minimization. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-020-02778-2

  13. Zhang H, Wang XM, Gao XB (2018) Fast and accurate single image super-resolution via information distillation network. Paper presented at: Proceedings of the 2018 IEEE conference on computer vision and pattern recognition (CVPR), Salt Lake City, pp 723–731

    Google Scholar 

  14. Sakai Y, Lu HM, Tan J-K, Kim H (2019) Recognition of surrounding environment from electric wheelchair videos based on modified YOLOv2. Futur Gener Comput Syst 92:157–161

    Article  Google Scholar 

  15. Memisevic R, Konda KR, Krueger D (2015) Zero-bias autoencoders and the benefits of co-adapting features. Paper presented at: Proceedings of the 2015 International Conference on Learning Representations (ICLR), San Diego, CA, USA arXiv:1402.3337

    Google Scholar 

  16. Xiang LY, Yang SH, Liu YH, Li Q, Zhu CZ (2020) Novel linguistic steganography based on character-level text generation. Mathematics. 8:1558

    Article  Google Scholar 

  17. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. Paper presented at: Proceedings of the 2010 International conference on machine learning (ICML), Haifa, pp 807–814

    Google Scholar 

  18. Chen YT, Liu LW, Tao JJ, Xia RL, Zhang Q, Yang K, Xiong J, Chen X (2020) The improved image inpainting algorithm via encoder and similarity constraint. Vis Comput. https://doi.org/10.1007/s00371-020-01932-3

  19. Chen YT, Wang J, Chen X, Sangaiah AK, Yang K, Cao ZH (2019) Image super-resolution algorithm based on dual-channel convolutional neural networks. Appl Sci 9(11):2316

    Article  Google Scholar 

  20. Sun L, Ma C, Chen Y, Zheng Y, Shim HJ, Wu Z, Jeon B (2019) Low rank component induced spatial-spectral kernel method for hyperspectral image classification. IEEE Trans Circ Syst Video T 30:3829–3842. https://doi.org/10.1109/TCSVT.2019.2946723

    Article  Google Scholar 

  21. Chen YT, Liu LW, Tao JJ, Chen X, Xia RL, Zhang Q, Xiong J, Yang K, Xie JB (2020) The image annotation algorithm using convolutional features from intermediate layer of deep learning. Multimed Tools Appl. https://doi.org/10.1007/s11042-020-09887-2

  22. He SM, Li ZZ, Wang J, Xiong NX (2020) Intelligent detection for key performance indicators in industrial-based cyber-physical systems. IEEE Trans Ind Inform:1. https://doi.org/10.1109/TII.2020.3036168

  23. Lu WP, Zhang YT, Wang SJ, Huang HY, Liu Q, Luo S (2020) Concept representation by learning explicit and implicit concept couplings. IEEE Intell Syst. https://doi.org/10.1109/MIS.2020.3021188

  24. Martin D, Fowlkes C, Tal D, Malik J (2002) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Paper presented at: Proceedings of the 2002 International conference on computer vision (ICCV), Vancouver, pp 416–423

    Google Scholar 

  25. Chen YT, Zhang HP, Liu LW, Chen X, Zhang Q, Yang K, Xia RL, Xie JB (2020) Research on image inpainting algorithm of improved GAN based on two-discriminations networks. Appl Intell. https://doi.org/10.1007/s10489-020-01971-2

  26. Chen YT, Phonevilay V, Tao JJ, Chen X, Xia RL, Zhang Q, Yang K, Xiong J, Xie JB (2020) The face image super-resolution algorithm based on combined representation learning. Multimed Tools Appl. https://doi.org/10.1007/s11042-020-09969-1

  27. Shi WZ, Caballero J, Huszar F, Totz J, Altken AP, Bishop R, Rueckert D, Wang ZH (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Paper presented at: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, pp 1874–1883

    Google Scholar 

  28. Luo X, Tu XM, Ding Y, Gao G, Deng MH (2019) Expectation pooling: an effective and interpretable pooling method for predicting DNA–protein binding. Bioinformatics. https://doi.org/10.1093/bioinformatics/btz768

  29. Timofte R, Agustsson E, Van Gool L, Yang MH, Zhang L, Lim B, Son S, Kim H, Nah S, Lee KM et al (2017) Ntire 2017 challenge on single image super-resolution: methods and results. Paper presented at: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition workshops (CVPR workshops), Honolulu, pp 1110–1121

    Google Scholar 

  30. Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests. Paper Presented at: Proceedings of the 2015 IEEE conference on computer vision and pattern recognition (CVPR), Boston, USA, pp 3791–3799

    Google Scholar 

  31. Zhang JM, Sun J, Wang J, Yue XG (2020) Visual object tracking based on residual network and cascaded correlation filters. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-020-02572-0

  32. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. Paper presented at: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, pp 1637–1645

  33. Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K et al (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76(20):21811–21838

  34. Soh JW, Cho S, Cho NI (2020) Meta-transfer learning for zero-shot super-resolution. Paper presented at: Proceedings of the 2020 IEEE conference on computer vision and pattern recognition (CVPR), Seattle, pp 3513–3522

    Google Scholar 

  35. Dai T, Cai JR, Zhang YB, Xia ST, Zhang L (2019) Second-order attention network for single image super-resolution. Paper presented at: Proceedings of the 2019 IEEE conference on computer vision and pattern recognition (CVPR), Long Beach, pp 11065–11074

    Google Scholar 

  36. Guo Y, Chen J, Wang JD, Chen Q, Cao JZ, Deng ZS, Xu YW, Tan MK (2020) Closed-loop matters: dual regression networks for single image super-resolution. Paper presented at: Proceedings of the 2020 IEEE conference on computer vision and pattern recognition (CVPR), Seattle, pp 5406–5415

    Google Scholar 

  37. Li F, Cong RM, Bai HH, He YF (2020) Deep interleaved network for single image super-resolution with asymmetric co-attention. Paper presented at: proceedings of the twenty-ninth international joint conference on artificial intelligence (IJCAI), Yokohama, Japan, pp 537–543

    Google Scholar 

  38. Liu J, Zhang WJ, Tang YT, Tang J, Wu GS (2020) Residual feature aggregation network for image super-resolution. Paper presented at: Proceedings of the 2020 IEEE conference on computer vision and pattern recognition (CVPR), Seattle, pp 2356–2365

    Google Scholar 

  39. Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Process 15(11):3440–3451

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China [61972056, 61772454, 61402053, 61981340416], the Natural Science Foundation of Hunan Province of China [2020JJ4623], the Scientific Research Fund of Hunan Provincial Education Department [17A007, 19C0028, 19B005], the Junior Faculty Development Program Project of Changsha University of Science and Technology [2019QJCZ011], the “Double First-class” International Cooperation and Development Scientific Research Project of Changsha University of Science and Technology [2019IC34], the Practical Innovation and Entrepreneurship Ability Improvement Plan for Professional Degree Postgraduate of Changsha University of Science and Technology [SJCX202072], the Postgraduate Training Innovation Base Construction Project of Hunan Province [2019-248-51, 2020-172-48].

Author information

Authors and Affiliations

  1. School of Computer and Communication Engineering & Hunan Provincial Key Laboratory of Intelligent Processing of Big Data on Transportation, Changsha University of Science and Technology, Changsha, 410114, Hunan, China

    Yuantao Chen, Linwu Liu, Volachith Phonevilay & Ke Gu

  2. Hunan Institute of Scientific and Technical Information, Changsha, 410001, Hunan, China

    Runlong Xia & Jingbo Xie

  3. Department of Electronic Products, Hunan ZOOMLION Intelligent Technology Corporation Limited, Changsha, 410005, Hunan, China

    Qian Zhang & Kai Yang

Authors
  1. Yuantao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linwu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Volachith Phonevilay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ke Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Runlong Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jingbo Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kai Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuantao Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Liu, L., Phonevilay, V. et al. Image super-resolution reconstruction based on feature map attention mechanism. Appl Intell 51, 4367–4380 (2021). https://doi.org/10.1007/s10489-020-02116-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02116-1

Keywords

Search

Navigation