Skip to main content
SpringerLink
Log in

Multi-atlas subcortical segmentation: an orchestration of 3D fully convolutional network and generalized mixture function

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

To accurately segment subcortical structures and therefore profit for numerous neuroimaging applications, we proposed a multi-atlas subcortical segmentation method by orchestrating a 3D fully convolutional network and a generalized mixture function. Template atlases were first aligned to the target image. Then, target image patches and several most similar atlas patches were extracted from the transformed template atlases by employing a proposed similar atlas selection network and fed into the proposed multi-atlas driven 3D fully convolutional neural network. To sufficiently extract the subcortical features and improve the segmentation performance, a restricted region thought as a bounding box was utilized to roughly locate the subcortical structures. Additionally, a generalized mixture function was introduced to reduce the impact of the size and stride in 3D patch extraction. Two datasets consisting of 16 and 18 T1-weighted magnetic resonance images images (MRIs) were included to evaluate the proposed method, respectively. The results showed significantly higher segmentation accuracy than several state-of-the-art subcortical segmentation approaches for most subcortical structures. Furthermore, the proposed method achieved notable higher mean Dice similarity coefficients being, respectively, 0.915 and 0.869. The proposed method automatically and accurately segments subcortical structures in MRIs, which may assist the artificial diagnosis of brain disorders.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://www.predict-hd.net/

  2. https://www.nitrc.org/projects/ibsr

  3. https://pytorch.org/

References

  1. Apostolova, L.G., Dinov, I.D., Dutton, R.A., Hayashi, K.M., Toga, A.W., Cummings, J.L., Thompson, P.M.: 3D comparison of hippocampal atrophy in amnestic mild cognitive impairment and Alzheimer’s disease. Brain 129(11), 2867–2873 (2006). https://doi.org/10.1093/brain/awl274

    Article  Google Scholar 

  2. Cobzas, D., Sun, H., Walsh, A.J., Lebel, R.M., Blevins, G., Wilman, A.H.: Subcortical gray matter segmentation and voxel-based analysis using transverse relaxation and quantitative susceptibility mapping with application to multiple sclerosis. J. Magn. Reson. Imaging 42(6), 1601–1610 (2015). https://doi.org/10.1002/jmri.24951

    Article  Google Scholar 

  3. Cerliani, L., Mennes, M., Thomas, R.M., Di Martino, A., Thioux, M., Keysers, C.: Increased functional connectivity between subcortical and cortical resting-state networks in autism spectrum disorder. JAMA Psychiat. 72(8), 767–777 (2015). https://doi.org/10.1001/jamapsychiatry.2015.0101

    Article  Google Scholar 

  4. Geevarghese, R., Lumsden, D.E., Hulse, N., Samuel, M., Ashkan, K.: Subcortical structure volumes and correlation to clinical variables in Parkinson’s disease. J. Neuroimaging 25(2), 275–280 (2015). https://doi.org/10.1111/jon.12095

    Article  Google Scholar 

  5. Tang, X., Qin, Y., Wu, J., Zhang, M., Zhu, W., Miller, M.I.: Shape and diffusion tensor imaging based integrative analysis of the hippocampus and the amygdala in Alzheimer’s disease. Magn. Reson. Imaging 34(8), 1087–1099 (2016). https://doi.org/10.1016/j.mri.2016.05.001

    Article  Google Scholar 

  6. Collins, D.L., Holmes, C.J., Peters, T.M., Evans, A.C.: Automatic 3-d model-based neuroanatomical segmentation. Hum. Brain Mapp. 3(3), 190–208 (1995). https://doi.org/10.1002/hbm.460030304

    Article  Google Scholar 

  7. Aljabar, P., Heckemann, R.A., Hammers, A., Hajnal, J.V., Rueckert, D.: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. Neuroimage 46(3), 726–738 (2009). https://doi.org/10.1016/j.neuroimage.2009.02.018

    Article  Google Scholar 

  8. Wang, H., Yushkevich, P.: Multi-atlas segmentation with joint label fusion and corrective learning-an open source implementation. Front. Neuroinform. 7, 27 (2013). https://doi.org/10.3389/fninf.2013.00027

    Article  Google Scholar 

  9. Wu, G., Kim, M., Sanroma, G., Wang, Q., Munsell, B.C., Shen, D., Initiative, A.D.N.: Hierarchical multi-atlas label fusion with multi-scale feature representation and label-specific patch partition. Neuroimage 106, 34–46 (2015). https://doi.org/10.1016/j.neuroimage.2014.11.025

    Article  Google Scholar 

  10. Giraud, R., Ta, V.T., Papadakis, N., Manjón, J.V., Collins, D.L., Coupé, P., Initiative, A.D.N.: An optimized patchmatch for multi-scale and multi-feature label fusion. Neuroimage 124, 770–782 (2016). https://doi.org/10.1016/j.neuroimage.2015.07.076

    Article  Google Scholar 

  11. van Opbroek, A., van der Lijn, F., de Bruijne, M.: Automated brain-tissue segmentation by multi-feature SVM classification. MIDAS J. (2013). https://doi.org/10.54294/ojfo7q

    Article  Google Scholar 

  12. Moeskops, P., Benders, M.J., Chiţǎ, S.M., Kersbergen, K.J., Groenendaal, F., de Vries, L.S., Viergever, M.A., Išgum, I.: Automatic segmentation of MR brain images of preterm infants using supervised classification. Neuroimage 118, 628–641 (2015). https://doi.org/10.1016/j.neuroimage.2015.06.007

    Article  Google Scholar 

  13. Moeskops, P., Viergever, M.A., Benders, M.J., Išgum, I.: In: Medical Imaging 2015: Image Processing 9413, (SPIE, 2015) 304–309 (2015). https://doi.org/10.1117/12.2081833

  14. Ashburner, J., Friston, K.J.: Unified segmentation. Neuroimage 26(3), 839–851 (2005). https://doi.org/10.1016/j.neuroimage.2005.02.018

    Article  Google Scholar 

  15. Rajchl, M., Baxter, J.S., McLeod, A.J., Yuan, J., Qiu, W., Peters, T.M., Khan, A.R.: Asets: map-based brain tissue segmentation using manifold learning and hierarchical max-flow regularization. In: Proceedings of the MICCAI Grand Challenge on MR Brain Image Segmentation (MRBrainS’13), Nagoya, Japan 26 (2013)

  16. Prakash, R. M., Kumari, R. S. S.: In: Conference: MRBRAINS13, Japan (2018)

  17. Pereira, S., Pinto, A., Oliveira, J., Mendrik, A.M., Correia, J.H., Silva, C.A.: Automatic brain tissue segmentation in MR images using random forests and conditional random fields. J. Neurosci. Methods 270, 111–123 (2016). https://doi.org/10.1016/j.jneumeth.2016.06.017

    Article  Google Scholar 

  18. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: In: International Conference on Medical Image Computing and Ccomputer-Assisted Intervention, pp. 424–432. Springer, (2016)

  19. Kushibar, K., Valverde, S., Gonzalez-Villa, S., Bernal, J., Cabezas, M., Oliver, A., Lladó, X.: Automated sub-cortical brain structure segmentation combining spatial and deep convolutional features. Med. Image Anal. 48, 177–186 (2018). https://doi.org/10.1016/j.media.2018.06.006

    Article  Google Scholar 

  20. Coupeau, P., Fasquel, J.B., Mazerand, E., Menei, P., Montero-Menei, C., Dinomais, M.: Patch-based 3D u-net and transfer learning for longitudinal piglet brain segmentation on MRI. Comput. Methods Progr. Biomed. 214, 106563 (2022). https://doi.org/10.1016/j.cmpb.2021.106563

    Article  Google Scholar 

  21. Yee, E., Ma, D., Popuri, K., Chen, S., Lee, H., Chow, V., Ma, C., Wang, L., Beg, M.F., Initiative, A.D.N.: 3D hemisphere-based convolutional neural network for whole-brain MRI segmentation. Comput. Med. Imaging Gr. 95, 102000 (2022). https://doi.org/10.1016/j.compmedimag.2021.102000

    Article  Google Scholar 

  22. Lee, N., Laine, A.F., Klein, A.: In: 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 321–324. IEEE, (2011). https://doi.org/10.1109/ISBI.2011.5872414

  23. Zhang, W., Li, R., Deng, H., Wang, L., Lin, W., Ji, S., Shen, D.: Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 108, 214–224 (2015). https://doi.org/10.1016/j.neuroimage.2014.12.061

    Article  Google Scholar 

  24. Dou, Q., Chen, H., Yu, L., Zhao, L., Qin, J., Wang, D., Mok, V.C., Shi, L., Heng, P.A.: Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks. IEEE Trans. Med. Imaging 35(5), 1182–1195 (2016). https://doi.org/10.1109/TMI.2016.2528129

    Article  Google Scholar 

  25. Wu, J., Tang, X.: Brain segmentation based on multi-atlas and diffeomorphism guided 3D fully convolutional network ensembles. Pattern Recognit. 115, 107904 (2021). https://doi.org/10.1016/j.patcog.2021.107904

    Article  Google Scholar 

  26. Bernal, J., Kushibar, K., Cabezas, M., Valverde, S., Oliver, A., Lladó, X.: Quantitative analysis of patch-based fully convolutional neural networks for tissue segmentation on brain magnetic resonance imaging. IEEE Access 7, 89986–90002 (2019). https://doi.org/10.1109/ACCESS.2019.2926697

    Article  Google Scholar 

  27. Dolz, J., Desrosiers, C., Ayed, I.B.: 3D fully convolutional networks for subcortical segmentation in MRI: a large-scale study. Neuroimage 170, 456–470 (2018). https://doi.org/10.1016/j.neuroimage.2017.04.039

  28. Huo, Y., Xu, Z., Xiong, Y., Aboud, K., Parvathaneni, P., Bao, S., Bermudez, C., Resnick, S.M., Cutting, L.E., Landman, B.A.: 3D whole brain segmentation using spatially localized atlas network tiles. Neuroimage 194, 105–119 (2019). https://doi.org/10.1016/j.neuroimage.2019.03.041

    Article  Google Scholar 

  29. Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017). https://doi.org/10.1016/j.media.2016.10.004

    Article  Google Scholar 

  30. Wu, J., Tang, X.: A large deformation diffeomorphic framework for fast brain image registration via parallel computing and optimization. Neuroinformatics 18(2), 251–266 (2020). https://doi.org/10.1007/s12021-019-09438-7

    Article  Google Scholar 

  31. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems 30 (2017)

  32. Li, H., Jiang, G., Zhang, J., Wang, R., Wang, Z., Zheng, W.S., Menze, B.: Fully convolutional network ensembles for white matter hyperintensities segmentation in MR images. Neuroimage 183, 650–665 (2018). https://doi.org/10.1016/j.neuroimage.2018.07.005

    Article  Google Scholar 

  33. Costa, V.S., Farias, A.D.S., Bedregal, B., Santiago, R.H., Canuto, A.M.d.P.: Combining multiple algorithms in classifier ensembles using generalized mixture functions. Neurocomputing 313, 402–414 (2018). https://doi.org/10.1016/j.neucom.2018.06.021

  34. Woods, R.P., Mazziotta, J.C., Cherry, S.R.: MRI-PET registration with automated algorithm. J. Comput. Assist. Tomogr. 17, 536–546 (1993). https://doi.org/10.1097/00004728-199307000-00004

    Article  Google Scholar 

  35. Mehta, R., Majumdar, A., Sivaswamy, J.: BrainSegNet: a convolutional neural network architecture for automated segmentation of human brain structures. J. Med. Imaging 4(2), 024003 (2017). https://doi.org/10.1117/1.JMI.4.2.024003

    Article  Google Scholar 

Download references

Acknowledgements

This study was supported by the National Natural Science Foundation of China (62206093), the Natural Science Foundation of Hunan Province (2022JJ40290), the Youth Foundation of Hunan Province Department of Education (21B0619), and the Scientific Research Project of Hunan University of Arts and Science (20ZD01 and 21BSQD31).

Author information

Authors and Affiliations

  1. School of Computer and Electrical Engineering, Hunan University of Arts and Science, Changde, 415000, Hunan, China

    Jiong Wu

  2. Claremont Graduate University, Claremont, CA, 91711, USA

    Shuan He

  3. Furong College, Hunan University of Arts and Science, Changede, 415000, Hunan, China

    Shuang Zhou

Authors
  1. Jiong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuan He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiong Wu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 27 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, J., He, S. & Zhou, S. Multi-atlas subcortical segmentation: an orchestration of 3D fully convolutional network and generalized mixture function. Machine Vision and Applications 34, 64 (2023). https://doi.org/10.1007/s00138-023-01415-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-023-01415-0

Keywords

Search

Navigation