Skip to main content
SpringerLink
Log in

BIRGU Net: deformable brain magnetic resonance image registration using gyral-net map and 3D Res-Unet

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Deformable image registration is a fundamental procedure in medical imaging. Recently, deep learning-based deformable image registrations have achieved fast registration by learning the spatial correspondence from image pairs. However, it remains challenging in brain image registration due to the structural complexity of individual brains and the lack of ground truth for anatomical correspondences between the brain image pairs. This work devotes to achieving an end-to-end unsupervised brain deformable image registration method using the gyral-net map and 3D Res-Unet (BIRGU Net). Firstly, the gyral-net map was introduced to represent the 3D global cortex complex information of the brain image since it was considered as one of the anatomical landmarks, which can help to extract the salient structural feature of individual brains for registration. Secondly, the variant of 3D U-net architecture involving dual residual strategies was designed to map the image into the deformation field effectively and to prevent the gradient from vanishing as well. Finally, double regularized terms were imposed on the deformation field to guide the network training for leveraging the smoothness and the topology preservation of the deformation field. The registration procedure was trained in an unsupervised manner, which addressed the lack of ground truth for anatomical correspondences between the brain image pairs. The experimental results on four public data sets demonstrate that the extracted gyral-net can be an auxiliary feature for registration and the proposed network with the designed strategies can improve the registration performance since the Dice similarity coefficient (DSC) and normalized mutual information (NMI) are higher and the time consumption is comparable than the state-of-the-art. The code is available at https://github.com/mynameiswode/BIRGU-Net.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Haskins G, Kruger U, Yan P (2020) Deep learning in medical image registration: a survey. Mach Vis Appl 31(1):1–18. https://doi.org/10.1007/s00138-020-01060-x

    Article  Google Scholar 

  2. Li H, Fan Y (2017) Non-rigid image registration using fully convolutional networks with deep self-supervision. arXiv:1709.00799

  3. Cao X, Yang J, Zhang J, Wang Q, Yap P-T, Shen D (2018) Deformable image registration using a cue-aware deep regression network. IEEE Trans Biomed Eng 65(9):1900–1911. https://doi.org/10.1109/TBME.2018.2822826

    Article  Google Scholar 

  4. Sokooti H, De Vos B, Berendsen F, Lelieveldt BP, Išgum I, Staring M (2017) Nonrigid image registration using multi-scale 3D convolutional neural networks. In: International conference on medical image computing and computer-assisted intervention. https://doi.org/10.1007/978-3-319-66182-7_27. Springer, pp 232–239

  5. Miao S, Wang ZJ, Liao R (2016) A CNN regression approach for real-time 2D/3D registration. IEEE Trans Med Imaging 35(5):1352–1363. https://doi.org/10.1109/TMI.2016.2521800

    Article  Google Scholar 

  6. Fan J, Cao X, Yap P-T, Shen D (2019) Birnet: Brain image registration using dual-supervised fully convolutional networks. Med Image Anal 54:193–206. https://doi.org/10.1016/j.media.2019.03.006

    Article  Google Scholar 

  7. Dosovitskiy A, Fischer P, Ilg E, Hausser P, Hazirbas C, Golkov V, Van Der Smagt P, Cremers D, Brox T (2015) Flownet: Learning optical flow with convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 2758–2766, https://doi.org/10.1109/ICCV.2015.316

  8. Cao X, Yang J, Zhang J, Nie D, Kim M, Wang Q, Shen D (2017) Deformable image registration based on similarity-steered CNN regression. In: International conference on medical image computing and computer-assisted intervention, pp 300–308, https://doi.org/10.1007/978-3-319-66182-7_35

  9. Sun Y, Moelker A, Niessen WJ, van Walsum T (2018) Towards robust CT-ultrasound registration using deep learning methods. In: Understanding and interpreting machine learning in medical image computing applications, pp 43–51, https://doi.org/10.1007/978-3-030-02628-8_5

  10. Fu Y, Lei Y, Wang T, Curran WJ, Liu T, Yang X (2020) Deep learning in medical image registration: a review. Phys Med Biol 65(20):20–01. https://doi.org/10.1088/1361-6560/ab843e

    Article  Google Scholar 

  11. Zhang J (2018) Inverse-consistent deep networks for unsupervised deformable image registration. https://doi.org/10.48550/arXiv.1809.03443

  12. Wu G, Kim M, Wang Q, Gao Y, Liao S, Shen D (2013) Unsupervised deep feature learning for deformable registration of MR brain images. In: International conference on medical image computing and computer-assisted intervention, pp 649–656, https://doi.org/10.1007/978-3-642-40763-5_80

  13. Shan S, Yan W, Guo X, Chang EI, Fan Y, Xu Y et al (2017) Unsupervised end-to-end learning for deformable medical image registration. arXiv:1711.08608

  14. Vos B, Berendsen FF, Viergever MA, Staring M, Igum I (2017) End-to-end unsupervised deformable image registration with a convolutional neural network. In: International workshop on deep learning in medical image analysis international workshop on multimodal learning for clinical decision support, https://doi.org/10.1007/978-3-319-67558-9_24

  15. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2018) An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9252–9260, https://doi.org/10.1109/CVPR.2018.00964

  16. Ramalhinho J, Tregidgo HFJ, Gurusamy K, Hawkes DJ, Davidson B, Clarkson MJ (2021) Registration of untracked 2D laparoscopic ultrasound to CT images of the liver using multi-labelled content-based image retrieval. IEEE Trans Med Imaging 40(3):1042–1054. https://doi.org/10.1109/TMI.2020.3045348

    Article  Google Scholar 

  17. Jaderberg M, Simonyan K, Zisserman A et al (2015) Spatial transformer networks. Advances in neural information processing systems 28. arXiv:1506.02025

  18. Mok TC, Chung A (2020) Fast symmetric diffeomorphic image registration with convolutional neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4644–4653, https://doi.org/10.48550/arXiv.2003.09514

  19. Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2019) Voxelmorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging 38(8):1788–1800. https://doi.org/10.1109/TMI.2019.2897538

    Article  Google Scholar 

  20. Mahapatra D, Ge Z, Sedai S, Chakravorty R (2018) Joint registration and segmentation of xray images using generative adversarial networks. In: International workshop on machine learning in medical imaging, pp 73–80, https://doi.org/10.1007/978-3-030-00919-9_9

  21. Li K, Guo L, Li G, Nie J, Faraco C, Cui G, Zhao Q, Miller LS, Liu T (2010) Gyral folding pattern analysis via surface profiling. Neuroimage 52(4):1202–1214. https://doi.org/10.1016/j.neuroimage.2010.04.263

    Article  Google Scholar 

  22. Zhang T, Chen H, Razavi MJ, Li Y, Ge F, Guo L, Wang X, Liu T (2018) Exploring 3-hinge gyral folding patterns among HCP Q3 868 human subjects. Human Brain Mapp 39(10):4134–4149. https://doi.org/10.1002/hbm.24237

    Article  Google Scholar 

  23. Chen H, Li Y, Ge F, Li G, Shen D, Liu T (2017) Gyral net: A new representation of cortical folding organization. Med Image Anal 42:14–25. https://doi.org/10.1016/j.media.2017.07.001

    Article  Google Scholar 

  24. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241, https://doi.org/10.1007/978-3-319-24574-4_28

  25. Shattuck DW, Mirza M, Adisetiyo V, Hojatkashani C, Salamon G, Narr KL, Poldrack RA, Bilder RM, Toga AW (2008) Construction of a 3D probabilistic atlas of human cortical structures. Neuroimage 39(3):1064–1080. https://doi.org/10.1016/j.neuroimage.2007.09.031

    Article  Google Scholar 

  26. Rohlfing T (2011) Image similarity and tissue overlaps as surrogates for image registration accuracy: widely used but unreliable. IEEE Trans Med Imaging 31(2):153–163. https://doi.org/10.1109/TMI.2011.2163944

    Article  Google Scholar 

  27. Klein A, Andersson J, Ardekani BA, Ashburner J, Avants B, Chiang M-C, Christensen GE, Collins DL, Gee J, Hellier P et al (2009) Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. Neuroimage 46(3):786–802. https://doi.org/10.1016/j.neuroimage.2008.12.037

    Article  Google Scholar 

  28. Fischl B (2012) Freesurfer. Neuroimage 62(2):774–781. https://doi.org/10.1016/j.neuroimage.2012.01.021

    Article  Google Scholar 

  29. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D U-net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention, pp 424–432, https://doi.org/10.1016/j.neuroimage.2012.01.021

  30. Jenkinson M, Smith S (2001) A global optimisation method for robust affine registration of brain images. Med Image Anal 5(2):143–156. https://doi.org/10.1016/S1361-8415(01)00036-6

    Article  CAS  Google Scholar 

  31. Dice LR (1945) Measures of the amount of ecologic association between species. Ecology 26 (3):297–302. https://doi.org/10.2307/1932409

    Article  Google Scholar 

  32. Studholme C, Hill DLG, Hawkes DJ (1999) An overlap invariant entropy measure of 3D medical image alignment. Pattern Recogn 32(1):71–86. https://doi.org/10.1016/S0031-3203(98)00091-0

    Article  Google Scholar 

  33. Marcus M (1963) An introduction to information theory. Math Mag 36 (4):207–218. https://doi.org/10.1080/0025570X.1963.11975434

    Article  Google Scholar 

  34. Wu S, Wang Z, Liu C, Zhu C, Wu S, Xiao K (2019) Automatical segmentation of pelvic organs after hysterectomy by using dilated convolution u-net++. In: 2019 IEEE 19th international conference on software quality, reliability and security companion (QRS-C), pp 362–367, https://doi.org/10.1109/QRS-C.2019.00074

  35. Avants BB, Epstein CL, Grossman M, Gee JC (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 12(1):26–41. https://doi.org/10.1016/j.media.2007.06.004

    Article  CAS  Google Scholar 

Download references

Funding

This work is supported by Research Foundation of Education Department of Hunan Province of China (19A496, 21A0109, 21B0172), the Natural Science Foundation of Hunan Province of China (2022JJ30552, 2022JJ30571), the National Natural Science Foundation of China (CN) (61972333) and Open Project of Key Laboratory of Medical Imaging and Artificial Intelligence of Hunan Province (YXZN2022003), Xiangnan University.

Author information

Authors and Affiliations

  1. MOE Key Laboratory of Intelligent Computing and Information Processing, Xiangtan University, Xiangtan, 411100, Hunan, China

    Chunhong Cao, Ling Cao & Gai Li

  2. Key Laboratory of Medical Imaging and Artificial Intelligence of Hunan Province, Xiangnan University, Chenzhou, Hunan Province, 423000, China

    Chunhong Cao

  3. The School of Automation, Northwestern Polytechnical University, Xi’an, 710072, Shaanxi, China

    Tuo Zhang

  4. Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing, Hunan Normal University, Changsha, 410081, Hunan, China

    Xieping Gao

Authors
  1. Chunhong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ling Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tuo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xieping Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Chunhong Cao, and Ling Cao made contributions to design this study and draft the manuscript; Gai Li performed the supplementary experiments; Tuo Zhang and Xieping Gao revised the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Gai Li or Xieping Gao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, C., Cao, L., Li, G. et al. BIRGU Net: deformable brain magnetic resonance image registration using gyral-net map and 3D Res-Unet. Med Biol Eng Comput 61, 579–592 (2023). https://doi.org/10.1007/s11517-022-02725-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02725-7

Keywords

Search

Navigation