Skip to main content

Advertisement

SpringerLink
Log in

Multi-target Interactive Neural Network for Automated Segmentation of the Hippocampus in Magnetic Resonance Imaging

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The hippocampus has been recognized as an important biomarker for the diagnosis and assessment of neurological diseases. Convenient and accurate automated segmentation of the hippocampus facilitates the analysis of large-scale neuroimaging studies. This work describes a novel technique for hippocampus segmentation in magnetic resonance images, in which interactive neural network (Inter-Net) is based on 3D convolutional operations. Inter-Net achieves the interaction through two aspects: one is the compartments, which builds an exponential ensemble network that integrates numerous short networks together when forward propagation. The other is the pathways, which realizes inter-connection between feature extraction and restoration. In addition, a multi-target architecture is proposed by designing multiple objective functions in terms of evaluation index, information theory, and data distribution. The proposed architecture is validated in fivefold cross-validation on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, where the mean Dice similarity indices of 0.919 (± 0.023) and precision of 0.926 (± 0.032) for the hippocampus segmentation. The running time is approximately 42.1 s from reading the image to outputting the segmentation result in our computer configuration. We compare the experimental results of a variety of methods to prove the effectiveness of the Inter-Net and contrast integrated architectures with different objective functions to illustrate the robustness of the fusion. The proposed framework is general and can be easily extended to numerous tissue segmentation tasks while it is tailored for the hippocampus.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Czepielewski LS, Wang L, Gama CS, et al. The relationship of intellectual functioning and cognitive performance to brain structure in schizophrenia. Schizophr Bull. 2017;43(2):355–64.

    PubMed  Google Scholar 

  2. Steiger VR, Brühl AB, Weidt S, Delsignore A, Rufer M, Jäncke L, et al. Pattern of structural brain changes in social anxiety disorder after cognitive behavioral group therapy: a longitudinal multimodal MRI study. Mol Psychiatry. 2017;22(8):1164–71.

    CAS  PubMed  Google Scholar 

  3. den Heijer T, van der Lijn F, Vernooij MW, et al. Structural and diffusion MRI measures of the hippocampus and memory performance. Neuroimage. 2012;63(4):1782–9.

    Google Scholar 

  4. Wixted JT, Squire LR. The medial temporal lobe and the attributes of memory. Trends Cogn Sci. 2011;15(5):210–7.

    PubMed  PubMed Central  Google Scholar 

  5. Jeneson A, Squire LR. Working memory, long-term memory, and medial temporal lobe function. Learn Mem. 2012;19(1):15–25.

    PubMed  PubMed Central  Google Scholar 

  6. Bobinski M, Wegiel J, Wisniewski HM, Tarnawski M, Bobinski M, Reisberg B, et al. Neurofibrillary pathology—correlation with hippocampal formation atrophy in Alzheimer disease. Neurobiol Aging. 1996;17(6):909–19.

    CAS  PubMed  Google Scholar 

  7. Geuze E, Vermetten E, Bremner JD. MR-based in vivo hippocampal volumetrics: 1. Review of methodologies currently employed. Mol Psychiatry. 2005;10(2):147–59.

    CAS  PubMed  Google Scholar 

  8. Knickmeyer RC, Gouttard S, Kang C, Evans D, Wilber K, Smith JK, et al. A structural MRI study of human brain development from birth to 2 years. J Neurosci. 2008;28(47):12176–82.

    CAS  PubMed  PubMed Central  Google Scholar 

  9. Filippi M, Rocca MA, Ciccarelli O, De Stefano N, Evangelou N, Kappos L, et al. MRI criteria for the diagnosis of multiple sclerosis: MAGNIMS consensus guidelines. Lancet Neurol. 2016;15(3):292–303.

    PubMed  PubMed Central  Google Scholar 

  10. Jacobsen C, Hagemeier J, Myhr KM, Nyland H, Lode K, Bergsland N, et al. Brain atrophy and disability progression in multiple sclerosis patients: a 10-year follow-up study. J Neurol Neurosurg Psychiatry. 2014;85(10):1109–15.

    PubMed  Google Scholar 

  11. Andreasen NC, Liu D, Ziebell S, Vora A, Ho BC. Relapse duration, treatment intensity, and brain tissue loss in schizophrenia: a prospective longitudinal MRI study. Am J Psychiatr. 2013;170(6):609–15.

    PubMed  Google Scholar 

  12. Scheenstra AEH, van de Ven RCG, van der Weerd L, van den Maagdenberg AM, Dijkstra J, Reiber JH. Automated segmentation of in vivo and ex vivo mouse brain magnetic resonance images. Mol Imaging. 2009;8(1):35–44.

    PubMed  Google Scholar 

  13. Carmichael OT, Aizenstein HA, Davis SW, Becker JT, Thompson PM, Meltzer CC, et al. Atlas-based hippocampus segmentation in Alzheimer’s disease and mild cognitive impairment. Neuroimage. 2005;27(4):979–90.

    PubMed  PubMed Central  Google Scholar 

  14. Chupin M, Mukuna-Bantumbakulu AR, Hasboun D, Bardinet E, Baillet S, Kinkingnéhun S, et al. Anatomically constrained region deformation for the automated segmentation of the hippocampus and the amygdala: method and validation on controls and patients with Alzheimer’s disease. Neuroimage. 2007;34(3):996–1019.

    PubMed  Google Scholar 

  15. Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, et al. Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron. 2002;33(3):341–55.

    CAS  Google Scholar 

  16. Zandifar A, Fonov V, Coupé P, Pruessner J, Collins DL, Alzheimer’s Disease Neuroimaging Initiative. A comparison of accurate automatic hippocampal segmentation methods. NeuroImage. 2017;155:383–93.

    PubMed  Google Scholar 

  17. Hosseini MP, Nazem Zadeh MR, Pompili D, Jafari-Khouzani K, Elisevich K, Soleanian-Zadeh H. Comparative performance evaluation of automated segmentation methods of hippocampus from magnetic resonance images of temporal lobe epilepsy patients. Med Phys. 2016;43(1):538–53.

    PubMed  PubMed Central  Google Scholar 

  18. Dill V, Franco AR, Pinho MS. Automated methods for hippocampus segmentation: the evolution and a review of the state of the art. Neuroinformatics. 2015;13(2):133–50.

    PubMed  Google Scholar 

  19. Birenbaum A, Greenspan H. Multi-view longitudinal CNN for multiple sclerosis lesion segmentation. Eng Appl Artif Intell. 2017;65:111–8.

    Google Scholar 

  20. Kwak K, Yoon U, Lee DK, Kim GH, Seo SW, Na DL. Fully-automated approach to hippocampus segmentation using a graph-cuts algorithm combined with atlas-based segmentation and morphological opening. Magn Reson Imaging. 2013;31(7):1190–6.

    PubMed  Google Scholar 

  21. Pipitone J, Park MTM, Winterburn J, Lett TA, Lerch JP, Pruessner JC, et al. Multi-atlas segmentation of the whole hippocampus and subfields using multiple automatically generated templates. Neuroimage. 2014;101:494–512.

    PubMed  Google Scholar 

  22. Sabuncu MR, Yeo BTT, Van Leemput K, Fischl B, Golland P. A generative model for image segmentation based on label fusion. IEEE Trans Med Imaging. 2010;29(10):1714–29.

    PubMed  PubMed Central  Google Scholar 

  23. Van der Lijn F, De Bruijne M, Klein S, Den Heijer T, Hoogendam YY, Van der Lugt A, et al. Automated brain structure segmentation based on atlas registration and appearance models. IEEE Trans Med Imaging. 2012;31(2):276–86.

    PubMed  Google Scholar 

  24. Kim M, Wu G, Li W, Wang L, Son YD, Cho ZH, et al. Automatic hippocampus segmentation of 7.0 Tesla MR images by combining multiple atlases and auto-context models. NeuroImage. 2013;83:335–45.

    PubMed  PubMed Central  Google Scholar 

  25. Hao Y, Wang T, Zhang X, Duan Y, Yu C, Jiang T, et al. Local label learning (LLL) for subcortical structure segmentation: application to hippocampus segmentation. Hum Brain Mapp. 2014;35(6):2674–97.

    PubMed  Google Scholar 

  26. Moghaddam MJ, Soltanian-Zadeh H. Automatic segmentation of brain structures using geometric moment invariants and artificial neural networks//International conference on Information Processing in Medical Imaging. Berlin: Springer; 2009. p. 326–37.

    Google Scholar 

  27. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, 2015. pp. 3431–3440.

  28. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. International conference on medical image computing and computer-assisted intervention. Cham: Springer; 2015. p. 234–41.

    Google Scholar 

  29. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal. 2017;36:61–78.

    PubMed  Google Scholar 

  30. Liu X, Deng Z. Segmentation of drivable road using deep fully convolutional residual network with pyramid pooling. Cogn Comput. 2017:1–10.

  31. Liu W, Tao D. Multiview Hessian regularization for image annotation. IEEE Trans Image Process. 2013;22(7):2676–87.

    PubMed  Google Scholar 

  32. Liu W, Yang X, Tao D, Cheng J, Tang Y. Multiview dimension reduction via Hessian multiset canonical correlations. Information Fusion. 2018;41:119–28.

    Google Scholar 

  33. Yuan Y, Xun G, Ma F, et al. Muvan: a multi-view attention network for multivariate temporal data. 2018 IEEE International Conference on Data Mining (ICDM). Piscataway: IEEE; 2018. p. 717–26.

    Google Scholar 

  34. Kang G, Liu K, Hou B, Zhang N. 3D multi-view convolutional neural networks for lung nodule classification. PloS one, Public Library of Science. 2017;12(11):e0188290.

    Google Scholar 

  35. Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel SJ, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging. 2016;35(5):1160–9.

    PubMed  Google Scholar 

  36. Fischl B. FreeSurfer. Neuroimage. 2012;62(2):774–81.

    PubMed  PubMed Central  Google Scholar 

  37. Chen Y, Shi B, Wang Z, Zhang P, Smith CD, Liu J. Hippocampus segmentation through multi-view ensemble ConvNets[C]//Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium on. IEEE, 2017. pp. 192–196.

  38. Jack CR Jr, Bernstein MA, Fox NC, et al. The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J Magn Reson Imaging. 2008;27(4):685–91.

    PubMed  PubMed Central  Google Scholar 

  39. Wen G, Hou Z, Li H, Li D, Jiang L, Xun E. Ensemble of deep neural networks with probability-based fusion for facial expression recognition. Cogn Comput. 2017;9(5):597–610.

    Google Scholar 

  40. Brosch T, Tang LY, Yoo Y, Li DK. Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans Med Imaging. 2016;35(5):1229–39.

    PubMed  Google Scholar 

  41. Veit A, Wilber M, Belongie S. Residual networks are exponential ensembles of relatively shallow networks. arXiv preprint. arXiv preprint arXiv:1605.06431. 2016;1(2):3.

  42. He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. European Conference on Computer Vision. Cham: Springer; 2016. p. 630–45.

    Google Scholar 

  43. Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th international conference on machine learning (ICML-10). 2010. pp. 807–814.

  44. Zeiler MD. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. 2012.

  45. Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.

  46. Dauphin Y, de Vries H, Bengio Y. Equilibrated adaptive learning rates for non-convex optimization[C]. Adv Neural Inf Proces Syst. 2015:1504–12.

  47. Srivastava N, Hinton G, Krizhevsky A, Stuskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(1):1929–58.

    Google Scholar 

  48. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014.

  49. Zeng D, Zhao F, Shen W, Ge S. Compressing and accelerating neural network for facial point localization. Cogn Comput. 2017:1–9.

  50. Dice LR. Measures of the amount of ecologic association between species. Ecology. 1945;26(3):297–302.

    Google Scholar 

  51. Cabezas M, Oliver A, Lladó X, Freixenet J, Cuadra MB. A review of atlas-based segmentation for magnetic resonance brain images. Comput Methods Prog Biomed. 2011;104(3):e158–77.

    Google Scholar 

  52. Ghanei A, Soltanian-Zadeh H, Windham JP. A 3D deformable surface model for segmentation of objects from volumetric data in medical images. Comput Biol Med. 1998;28(3):239–2.

    CAS  PubMed  Google Scholar 

  53. Lötjönen JMP, Wolz R, Koikkalainen JR, Thurfjell L, Waldemar G, Soininen H, et al. Fast and robust multi-atlas segmentation of brain magnetic resonance images. Neuroimage. 2010;49(3):2352–65.

    PubMed  Google Scholar 

  54. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. pp. 770–778.

  55. Wolz R, Aljabar P, Hajnal JV, Hammers A, Rueckert D. Alzheimer’s Disease Neuroimaging Initiative. LEAP: learning embeddings for atlas propagation. NeuroImage. 2010;49(2):1316–25.

    PubMed  Google Scholar 

Download references

Funding

This work was supported by National Natural Science Foundation of China (61471064), National Science and Technology Major Project of China (No.2017ZX03001022), and BUPT Excellent Ph.D. Students Foundation (No.CX2019309).

Author information

Authors and Affiliations

  1. Beijing University of Posts and Telecommunications, No.10 Xitucheng Road, Haidian District, Beijing, China

    Beibei Hou, Guixia Kang, Ningbo Zhang & Kui Liu

  2. Wuxi BUPT Sensory Technology and Industry Institute CO.LTD, No.97 Linghu Road, Newly District, Wuxi, China

    Beibei Hou, Guixia Kang & Kui Liu

Authors
  1. Beibei Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guixia Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ningbo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guixia Kang.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hou, B., Kang, G., Zhang, N. et al. Multi-target Interactive Neural Network for Automated Segmentation of the Hippocampus in Magnetic Resonance Imaging. Cogn Comput 11, 630–643 (2019). https://doi.org/10.1007/s12559-019-09645-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-019-09645-z

Keywords

Search

Navigation