Skip to main content
SpringerLink
Log in

Brain MR image super-resolution via a deep convolutional neural network with multi-unit upsampling learning

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Image super-resolution (SR) is a preferred approach to achieving high-resolution MR images because they are typically achieved in real world at the expense of reduced signal-to-noise ratio and/or increased imaging time. A novel deep residual network (DRN) with multi-unit upsampling learning is designed for MR image SR. The multi-unit upsampling learning mechanism involves multi-unit upsampling and adaptive learning. The designed DRN performs the SR task in the LR space to accelerate the network via an upsampling strategy at the late stage of the network architecture. A multi-unit upsampling strategy is proposed to transmit lost information in each residual unit and to accumulate the upscaled feature maps achieved by different residual units. An adaptive learning strategy following multi-unit upsampling is utilized to potentially discover the contributions of these upscaled feature maps to high-resolution MR image reconstruction by adaptively assigning different significance weights to the intermediate predictions. The proposed DRN achieves a fair good reconstruction performance, which is superior to some state-of-the-art deep-learning-based methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Shi, F., Cheng, J., Wang, L., Yap, P.T., Shen, D.G.: LRTV: MR imagesuper-resolution with low-rank and total variation regularizations. IEEETrans. Med. Imag. 34(12), 2459–2466 (2015)

    Article  Google Scholar 

  2. Esben, P., Poot, D.H.J., Monique, B., Gyula, K., Gavin, H., Piotr, W., vanderLouise, W., Wiro, J.N., Erik, M.: Super-resolution methods in MRI: can they improve the trade-off between resolution, signal-to-noise ratio, and acquisition time. Magn. Reson. Med. 68(6), 1983–1993 (2012)

    Article  Google Scholar 

  3. Pham, C.H., Ducournau, A., Fablet, R., Rousseau, F.: Brain MRI super-resolution using deep 3D convolutional networks. In: Proceedings of International Symposium on Biomedical Imaging, pp. 197–200

  4. Oktay, O., Bai, W., Lee, M., Guerrero, R., Kamnitsas, K., Caballero, J., Marvao, A., Cook, S., O’Regan, D., Rueckert, D.: Multi-input cardiac image super-resolution using convolutional neural networks. In: Proceedings of 19th International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 246–254

  5. Giannakidis, A., Oktay, O., Keegan, J., Spadotto, V., Voges, I., Smith, G., Pierce, I., Bai, W., Rueckert, D., Ernst, S., Gatzoulis, M.A., Pennell, D.J., Narayan, S.B., Firmin, D.N.: Super-resolution reconstruction of late gadolinium cardiovascular magnetic resonance images using a residual convolutional neural network. In: Proceedings of 25th Science Meeting International Society for Magnetic Resonance in Medicine

  6. Shi, J., Liu, Q., Wang, C., Zhang, Q., et al.: Super-resolution reconstruction of MR image with a novel residual learning network algorithm. Phys. Med. Biol. 63(8), 085011 (2018)

    Article  Google Scholar 

  7. Shi, J., Li, Z., Ying, S., Wang, C., Zhang, Q., Yan, P.: MR image super-resolution via wide residual networks with fixed skip connection. IEEE J. Biomed. Health Inform 23(3), 1129–1140 (2019)

    Article  Google Scholar 

  8. McDonagh, S., Hou, B., Alansary, A., Oktay, O., Kamnitsas, K., Rutherford, M., Hajnal, J.V., Kainzm, B.: Context-sensitive super-resolution for fast fetal magnetic resonance imaging. In: Proceedings of Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment, pp. 116–126

  9. Mardani, M., Gong, E., Cheng, J.Y. et al.: Deep generative adversarial networks for compressed sensing automates MRI. arXiv:1706.00051

  10. Tanno, R., Worrall, D.E., Ghosh, A., Kaden, E., Sotiropoulos, S.N., Criminisi, A., Alexander, D.C.: Bayesian image quality transfer with CNNs: exploring uncertainty in dMRI super-resolution. In: Proceedings of International Conference on Medical Imaging and Computer-Assisted Intervention, pp. 611–619

  11. Sánchez, I., Vilaplana, V.: Brain MRI super-resolution using 3D generative adversarial networks. arXiv:1812.11440

  12. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Proceedings of European Conference on Computer Vision, pp. 184–199

  13. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  14. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolutionusingvery deep convolutional networks. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1646–1654

  15. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern Recognition, pp. 2790–2798

  16. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883

  17. Haris, M., Shakhnarovich, G., Ukita, D.: Deep back-projection networks for super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1664–1673

  18. Dong,C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: Proceedings of European Conference on Computer Vision, pp. 391–407

  19. https://wiki.canerimagingarchive.net/display/Public/Brain-Tumor-Progression

  20. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034

Download references

Acknowledgments

This work was in part supported by the National Natural Science Foundation of China (Grant Nos. 61001179 and 61471132), Science and Technology Project of Guangdong Province (No. 2017B090911012), Project of Jihua Laboratory, China (No. X190071UZ190), Key Laboratory Construction Projects in Guangdong (No. 2017B030314178), University Innovation and Entrepreneurship Education Major Project of Guangzhou City (No. 201709P05), and the Science and Technology Program of Guangzhou, China (No. 201803010065).

Author information

Authors and Affiliations

  1. School of Information Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Hao Xia, Nian Cai, Huiheng Wang & Yadong Mao

  2. China Electronic Product Reliability and Environment Testing Research Institute, Guangzhou, 510610, China

    Hao Xia

  3. School of Electromechanical Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Han Wang

  4. Department of Diagnostic and Interventional Ultrasound, Sun Yat-sen University Cancer Center, Guangzhou, 510060, China

    Jian Li

  5. Department of Hepatobiliary Surgery, The First Affiliated Hospital of Guangzhou Medical University, Guangzhou, 510120, China

    Ping Wang

Authors
  1. Hao Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nian Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huiheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yadong Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Han Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nian Cai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xia, H., Cai, N., Wang, H. et al. Brain MR image super-resolution via a deep convolutional neural network with multi-unit upsampling learning. SIViP 15, 931–939 (2021). https://doi.org/10.1007/s11760-020-01817-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-020-01817-x

Keywords

Search

Navigation