Skip to main content
SpringerLink
Log in

Unsupervised deformable image registration network for 3D medical images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Image registration aims to establish an active correspondence between a pair of images. Such correspondence is critical for many significant applications, such as image fusion, tumor growth monitoring, and atlas generation. In this study, we propose an unsupervised deformable image registration network (UDIR-Net) for 3D medical images. The proposed UDIR-Net is designed in an encoder-decoder architecture and directly estimates the complex deformation field between input pairwise images without any supervised information. In particular, we recalibrate the feature slice of each feature map that is propagated between the encoder and the decoder in accordance with the importance of each feature slice and the correlation between feature slices. This method enhances the representational power of feature maps. To achieve efficient and robust training, we design a novel hierarchical loss function that evaluates multiscale similarity loss between registered image pairs. The proposed UDIR-Net is tested on different public magnetic resonance image datasets of the human brain. Experimental results show that UDIR-Net exhibits competitive performance against several state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M et al (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:160304467

  2. Avants BB, Epstein CL, Grossman M, Gee JC (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 12(1):26–41

    Article  Google Scholar 

  3. Avants BB, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) A reproducible evaluation of ants similarity metric performance in brain image registration. Neuroimage 54(3):2033–2044

    Article  Google Scholar 

  4. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2018) An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9252–9260

  5. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2019) Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging 38(8):1788–1800

    Article  Google Scholar 

  6. Cao X, Yang J, Zhang J, Wang Q, Yap PT, Shen D (2018) Deformable image registration using a cue-aware deep regression network. IEEE Trans Biomed Eng 65(9):1900–1911

    Article  Google Scholar 

  7. Chee E, Wu J (2018) Airnet: Self-supervised affine registration for 3d medical images using neural networks. arXiv:181002583

  8. Cheng X, Zhang L, Zheng Y (2018) Deep similarity learning for multimodal medical images. Comput Methods Biomech Biomed Eng Imaging Vis 6(3):248–252

    Article  Google Scholar 

  9. Chollet F (2015) Keras. https://keras.io

  10. Dalca AV, Balakrishnan G, Guttag J, Sabuncu MR (2018) Unsupervised learning for fast probabilistic diffeomorphic registration. In: International conference on medical image computing and computer-assisted intervention, pp 729–738

  11. Di Martino A, Yan CG, Li Q, Denio E, Castellanos FX, Alaerts K, Anderson JS, Assaf M, Bookheimer SY, Dapretto M et al (2014) The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol Psychiatry 19(6):659–667

    Article  Google Scholar 

  12. Eppenhof KA, Pluim JP (2018) Error estimation of deformable image registration of pulmonary ct scans using convolutional neural networks. J Med Imaging 5(2):024003

    Article  Google Scholar 

  13. Fan J, Cao X, Yap PT, Shen D (2019) Birnet: Brain image registration using dual-supervised fully convolutional networks. Med Image Anal 54:193–206

    Article  Google Scholar 

  14. Fischl B (2012) Freesurfer. Neuroimage 62(2):774–781

    Article  Google Scholar 

  15. Haskins G, Kruger U, Yan P (2019) Deep learning in medical image registration: A survey. arXiv:190302026

  16. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  17. Jaderberg M, Simonyan K, Zisserman A et al (2015) Spatial transformer networks. Advances in neural information processing systems, 2017–2025

  18. Khader M, Schiavi E, Hamza AB (2017) A multicomponent approach to nonrigid registration of diffusion tensor images. Appl Intell 46(2):241–253

    Article  Google Scholar 

  19. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:14126980

  20. Krebs J, Mansi T, Mailhé B, Ayache N, Delingette H (2018) Unsupervised probabilistic deformation modeling for robust diffeomorphic registration. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 101–109

  21. Li H, Fan Y (2018) Non-rigid image registration using self-supervised fully convolutional networks without training data. IEEE Int Symp Biomed Imaging 2018:1075–1078

    Google Scholar 

  22. Li H, Xiong P, An J, Wang L (2018) Pyramid attention network for semantic segmentation. arXiv:180510180

  23. Liao R, Miao S, de Tournemire P, Grbic S, Kamen A, Mansi t, Comaniciu D (2017) An artificial agent for robust image registration. Thirty-First AAAI Conference on Artificial Intelligence

  24. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  Google Scholar 

  25. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (1997) Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 16(2):187– 198

    Article  Google Scholar 

  26. Marcus DS, Wang TH, Parker J, Csernansky JG, Morris JC, Buckner RL (2007) Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. J Cogn Neurosci 19(9):1498–1507

    Article  Google Scholar 

  27. Miao S, Wang ZJ, Liao R (2016) A cnn regression approach for real-time 2d/3d registration. IEEE Trans Med Imaging 35(5):1352–1363

    Article  Google Scholar 

  28. Milham MP, Fair D, Mennes M, Mostofsky SH, et al. (2012) The adhd-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience. Front Syst Neurosci 6:62

    Google Scholar 

  29. Mueller SG, Weiner MW, Thal LJ, Petersen RC, Jack CR, Jagust W, Trojanowski JQ, Toga AW, Beckett L (2005) Ways toward an early diagnosis in alzheimer’s disease: the alzheimer’s disease neuroimaging initiative (adni). Alzheimer’s & Dementia 1(1):55–66

    Article  Google Scholar 

  30. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241

  31. Sheikhjafari A, Noga M, Punithakumar K, Ray N (2018) Unsupervised deformable image registration with fully connected generative neural network. In: International conference on Medical Imaging with Deep Learning

  32. Shen D, Wu G, Suk HI (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248

    Article  Google Scholar 

  33. Sokooti H, de Vos B, Berendsen F, Lelieveldt BP, Išgum I, Staring M (2017) Nonrigid image registration using multi-scale 3d convolutional neural networks. International Conference on Medical Image Computing and Computer-Assisted Intervention, 232–239

  34. Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: a survey. IEEE Trans Med Imaging 32(7):1153–1190

    Article  Google Scholar 

  35. de Vos BD, Berendsen FF, Viergever MA, Staring M, Išgum I (2017) End-to-end unsupervised deformable image registration with a convolutional neural network. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 204–212

  36. Woo S, Park J, Lee JY, So Kweon I (2018) Cbam: Convolutional block attention module, 3–19

  37. Wu G, Kim M, Wang Q, Gao Y, Liao S, Shen D (2013) Unsupervised deep feature learning for deformable registration of mr brain images. In: International conference on medical image computing and computer-assisted intervention, pp 649–656

  38. Wu G, Kim M, Wang Q, Munsell BC, Shen D (2015) Scalable high-performance image registration framework by unsupervised deep feature representations learning. IEEE Trans Biomed Eng 63(7):1505–1516

    Article  Google Scholar 

  39. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: 2009 IEEE Conference on computer vision and pattern recognition, pp 1794–1801

  40. Zhang Z, Sejdić E (2019) Radiological images and machine learning: trends, perspectives, and prospects. Comput Biol Med 108:354–370

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by Natural Science Foundation of Shandong province (Nos. ZR2019MF013, ZR2019BF026), Project of Jinan Scientific Research Leaderś Laboratory (No. 2018GXRC023).

Author information

Authors and Affiliations

  1. Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan, 250022, People’s Republic of China

    Yingjun Ma, Dongmei Niu, Jinshuo Zhang, Xiuyang Zhao & Bo Yang

  2. College of Cybersecurity, Shandong University of Political Science and Law, Jinan, 250014, People’s Republic of China

    Yingjun Ma

  3. School of Information Science and Engineering, University of Jinan, Jinan, 250022, People’s Republic of China

    Dongmei Niu, Jinshuo Zhang & Xiuyang Zhao

  4. School of Software, Shandong University, Jinan, 250101, People’s Republic of China

    Caiming Zhang

Authors
  1. Yingjun Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongmei Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinshuo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiuyang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bo Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Caiming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiuyang Zhao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Yingjun Ma and Dongmei Niu contributed equally to this work and should be considered co-first authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, Y., Niu, D., Zhang, J. et al. Unsupervised deformable image registration network for 3D medical images. Appl Intell 52, 766–779 (2022). https://doi.org/10.1007/s10489-021-02196-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02196-7

Keywords

Search

Navigation