Skip to main content
SpringerLink
Log in

Pulmonary nodule image super-resolution using multi-scale deep residual channel attention network with joint optimization

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

High-resolution medical images can help doctors to find early lesions and provide assistance and support for the diagnosis and treatment of diseases. Super-resolution can obtain a single high-resolution image from a given low-resolution image. It can be divided into three major means roughly: the interpolation-based methods, the reconstruction-based methods, and the learning-based methods. The interpolation-based methods rely on the smoothness assumptions and cannot restore fine textures. The reconstruction-based methods need to find the image degradation model and the optimal blur kernels. In fact, blur kernels are complicated and unknown. Kernel mismatch will fail to produce good results (e.g., over-sharpening or over-smoothing). The learning-based methods pay more attention to the understanding of the image content and structure, and they can establish a mapping function between the high-resolution images and the low-resolution images, which attract the attention of researchers. Deep learning has the strong ability of nonlinear mapping. Therefore, it has been widely used in super-resolution in recent years. In the paper, we propose a multi-scale deep residual channel attention network which consists of six components: joint input of low-resolution image and edge, shallow feature extraction, deep feature extraction, channel attention, high-resolution image reconstruction, and total loss. Edges are the first-order high-frequency details which are very important to super-resolution. The joint input of low-resolution images and edges enhances useful information. The multi-scale deep residual channel attention module can not only acquire structural features but also capture features of different scales and hierarchies. It can also obtain relationships among channel features. In addition, the joint guidance of perceptual loss, content loss, and edge loss is used to improve the visual quality and preserve the spatial structure and high-frequency details of low-resolution images. Experiments have been conducted on the pulmonary nodule image dataset, and the results demonstrate that the proposed method can yield better performance by comparing with the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Choi J, Kim M (2017) Single image super-resolution using global regression based on multiple local linear mappings. IEEE Trans Image Process 26(3):1300–1314

    Article  MathSciNet  Google Scholar 

  2. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  3. Yang J, Wang Z, Lin Z, Cohen S, Huang TS (2012) Coupled dictionary training for image super-resolution. IEEE Trans Image Process 21(8):3467–3478

    Article  MathSciNet  Google Scholar 

  4. Zhang Z, Jiang W, Li F, Zhao M, Li B, Zhang L (2017) Structured latent label consistent dictionary learning for salient machine faults representation-based robust classification. IEEE Trans Ind Inform 13(2):644–656

    Article  Google Scholar 

  5. Zhang Z, Li F, Zhao M, Zhang L, Yan S (2017) Robust neighborhood preserving projection by nuclear/l2, 1-norm regularization for image feature extraction. IEEE Trans Image Process 26(4):1607–1622

    Article  MathSciNet  Google Scholar 

  6. Yuan X, Yuan X (2011) Fusion of multi-planar images for improved three-dimensional object reconstruction. Comput Med Imaging Gr 35(5):373–382

    Article  Google Scholar 

  7. Efrat N, Glasner D, Apartsin A, Nadler B, Levin A (2013) Accurate blur models vs. image priors in single image super-resolution. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, Australia, 1–8 Dec, pp 2832–2839

  8. Yang C, Ma C, Yang M (2014) Single-image super-resolution: a benchmark. In: European Conference on Computer Vision, ECCV 2014, Zurich, Switzerland, 6–12 Sept, pp 372–386

    Chapter  Google Scholar 

  9. Gu S, Sang N, Ma F (2012) Fast image super resolution via local regression. In: International Conference on Pattern Recognition, ICPR 2012, Tsukuba, Japan, 11–15 Nov, pp 3128–3131

  10. Yang C, Yang M (2013) Fast direct super-resolution by simple functions. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, Australia, 1–8 Dec 2013, pp 561–568

  11. Romano Y, Isidoro J, Milanfar P (2017) RAISR: rapid and accurate image super resolution. IEEE Trans Comput Imaging 3(1):110–125

    Article  MathSciNet  Google Scholar 

  12. Gu S, Zuo W, Xie Q, Meng D, Feng X, Zhang L (2015) Convolutional sparse coding for image super-resolution. In: IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, 7–13 Dec 2015, pp 1823–1831

  13. Chang H, Yeung D, Xiong Y (2004) Super-resolution through neighbor embedding. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2004, Washington, DC, USA, 27 June–2 July, pp 275–282

  14. Timofte R, Smet VD, Gool LV (2013) Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision, ICCV 2013, Sydney, Australia, 1–8 Dec, pp 1920–1927

  15. Timofte R, Smet VD, Gool LV (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. In: Asian Conference on Computer Vision, ACCV 2014, Singapore, 1–5 Nov, pp 111–126

    Chapter  Google Scholar 

  16. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017, pp 105–114

  17. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, ECCV 2016, Amsterdam, The Netherlands, 11–14 Oct, pp 694–711

  18. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision, ECCV 2014, Zurich, Switzerland, 6–12 Sept, pp 184–199

    Chapter  Google Scholar 

  19. Liang Y, Wang J, Zhou S, Gong Y, Zheng N (2016) Incorporating image priors with deep convolutional neural networks for image super-resolution. Neurocomputing 194:340–347

    Article  Google Scholar 

  20. Xie J, Feris RS, Sun M (2016) Edge-guided single depth image super resolution. IEEE Trans Image Process 25(1):428–438

    Article  MathSciNet  Google Scholar 

  21. Yang W, Feng J, Yang J, Zhao F, Liu J, Guo Z, Yan S (2017) Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans Image Process 26(12):5895–5907

    Article  MathSciNet  Google Scholar 

  22. Liu H, Fu Z, Han J, Shao L, Hou S, Chu Y (2019) Single image super-resolution using multi-scale deep encoder–decoder with phase congruency edge map guidance. Inf Sci 473:44–58

    Article  MathSciNet  Google Scholar 

  23. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June, pp 1646–1654

  24. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June, pp 1637–1645

  25. Lai W, Huang J, Ahuja N, Yang M (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July, pp 5835–5843

  26. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European Conference on Computer Vision, ECCV 2016, Amsterdam, The Netherlands, 11–14 Oct, pp 391–407

  27. Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June, pp 1874–1883

  28. Hu Y, Gao X, Li J, Huang Y, Wang H (2018) Single image super-resolution via cascaded multi-scale cross network. CoRR arXiv:abs/1802.08808

  29. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: European Conference on Computer Vision, ECCV 2018, Munich, Germany, 8–14 Sept, pp 527–542

  30. Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July, pp 2261–2269

  31. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 Oct, pp 4809–4817

  32. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June, pp 2472–2481

  33. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Loy CC (2018) ESRGAN: enhanced super-resolution generative adversarial networks. In: European Conference on Computer Vision, ECCV 2018, Munich, Germany, 8–14 Sept, pp 63–79

  34. Ahn N, Kang B, Sohn K (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: European Conference on Computer Vision, ECCV 2018, Munich, Germany, 8–14 Sept, pp 256–272

  35. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July, pp 1132–1140

  36. Zhao H, Gallo O, Frosio I, Kautz J (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57

    Article  Google Scholar 

  37. Bruna J, Sprechmann P, LeCun Y (2016) Super-resolution with deep convolutional sufficient statistics. In: International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, 2–4 May

  38. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May

  39. Armato SGI, Mclennan G, Bidaut L, Mcnitt-Gray MF, Clarke LP (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 38(2):915–931

    Article  Google Scholar 

  40. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May

  41. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, 7–13 Dec, pp 1026–1034

  42. Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44(13):800–801

    Article  Google Scholar 

  43. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by the Natural Science Foundation of China under Grant 61702157 and 41804118, in part by Hebei Province Department of Education Fund under Grant QN2018085, and in part by Innovation Capacity Improvement Project of Hebei Province 199676146H.

Author information

Authors and Affiliations

  1. State Key Laboratory of Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Tianjin, China

    Yongjun Qi & Junhua Gu

  2. Laboratory of Electromagnetic Field and Electrical Apparatus Reliability, Hebei University of Technology, Tianjin, China

    Yongjun Qi & Junhua Gu

  3. Information Technology Center, North China Institute of Aerospace Engineering, Langfang, China

    Yongjun Qi & Juanping Geng

  4. School of Artificial Intelligence, Hebei University of Technology, Tianjin, China

    Junhua Gu, Weixun Li, Zepei Tian & Yajuan Zhang

  5. Hebei Province Key Laboratory of Big Data Calculation, Hebei University of Technology, Tianjin, China

    Junhua Gu & Yajuan Zhang

Authors
  1. Yongjun Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junhua Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weixun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zepei Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yajuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Juanping Geng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongjun Qi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qi, Y., Gu, J., Li, W. et al. Pulmonary nodule image super-resolution using multi-scale deep residual channel attention network with joint optimization. J Supercomput 76, 1005–1019 (2020). https://doi.org/10.1007/s11227-019-03066-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-019-03066-3

Keywords

Search

Navigation