Skip to main content
Springer Nature Link
Log in

Self-Supervised Pretraining for Cortical Surface Analysis

  • Conference paper
  • First Online:
Medical Image Understanding and Analysis (MIUA 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14859))

Included in the following conference series:

  • 506 Accesses

Abstract

To advance our knowledge in neuroscience it is fundamental to understand the complexities of the human cerebral cortex. The cortex exhibits significant variability across individuals, presenting challenges in identifying patterns at the population level. While supervised learning methods excel at such tasks, they require a large amount of labeled samples to train. This is a serious limitation due to the costly and time-consuming annotation process requiring neuroscience experts. To address this challenge, self-supervised learning (SSL) was introduced in various other domains. By pretraining models on unlabeled data, SSL reduces the dependency on large labeled datasets, as labeled data is only used to fine-tune the models for downstream tasks.

In this paper, we explore the effectiveness of self-supervised pretraining on a large number of cortical surfaces from the Human Connectome Project dataset. Leveraging a masked graph autoencoder, we develop a pretrained model suitable for various downstream tasks. The model’s performance in segmentation (node classification) and age prediction (graph regression) tasks are evaluated by using cortical surfaces from the manually labeled MindBoggle dataset. Our findings demonstrate that SSL with fine-tuning outperforms models trained from sratch across both tasks. Our research contributes to advancing the application of self-supervised learning in cortical surface analysis, with implications for neuroscience research and clinical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/freesurfer/freesurfer.

  2. 2.

    Segmentation and parcellation are used as synonyms throughout the paper.

References

  1. Kandel, E.R., Schwartz, J.H., Jessell, T.M., Siegelbaum, S., Hudspeth, A.J., Mack, S.: Principles of Neural Science. Vol. 4. New York: McGraw-Hill

    Google Scholar 

  2. Amunts, K., et al.: BigBrain: an ultrahigh-resolution 3D human brain model. Science 340(6139), 1472–1475 (2013)

    Article  MATH  Google Scholar 

  3. Van Essen, D.C., et al.: WU-Minn HCP consortium: the human connectome project: a data acquisition perspective. Neuroimage 62(4), 2222–2231 (2012)

    Article  MATH  Google Scholar 

  4. Elam, J.S., et al.: The human connectome project: a retrospective. Neuroimage 244, 118543 (2021)

    Article  MATH  Google Scholar 

  5. Glasser, M.F., et al.: Wu-Minn HCP consortium: the minimal preprocessing pipelines for the human connectome project. Neuroimage 80, 105–124 (2013)

    Article  Google Scholar 

  6. Fischl, B.: FreeSurfer. NeuroImage 62(2), 774–781 (2012)

    Article  Google Scholar 

  7. Ma, Q., Li, L., Robinson, E.C., Kainz, B., Rueckert, D., Alansary, A.: CortexODE: learning cortical surface reconstruction by neural ODEs. IEEE Trans. Med. Imaging 42(2), 430–443 (2022)

    Article  Google Scholar 

  8. Gopinath, K., Desrosiers, C., Lombaert, H.: Learning joint surface reconstruction and segmentation: from brain images to cortical surface parcellation. Med. Image Anal. 90, 102974 (2023)

    Article  Google Scholar 

  9. Yourganov, G., Smith, K.G., Fridriksson, J., Rorden, C.: Predicting aphasia type from brain damage measured with structural MRI. Cortex 73, 203–215 (2015)

    Article  MATH  Google Scholar 

  10. Klein, A., Tourville, J.: 101 labeled brain images and a consistent human cortical labeling protocol. Front. Neurosci. 6, 33392 (2012)

    Article  MATH  Google Scholar 

  11. Gopinath, K., Desrosiers, C., Lombaert, H.: Adaptive graph convolution pooling for brain surface analysis. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 86–98. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_7

    Chapter  MATH  Google Scholar 

  12. Dinsdale, N.K., et al.: Learning patterns of the ageing brain in MRI using deep convolutional networks. Neuroimage 224, 117401 (2021)

    Article  MATH  Google Scholar 

  13. Unyi, D., Gyires-Tóth, B.: Neurodevelopmental phenotype prediction: a state-of-the-art deep learning model. In: Machine Learning for Health, pp. 279–289. PMLR (2022)

    Google Scholar 

  14. Kalapos, A., Gyires-Tóth, B.: Self-supervised pretraining for 2D medical image segmentation. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds.) Computer Vision – ECCV 2022 Workshops. ECCV 2022. LNCS, vol. 13807. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-25082-8_31

  15. Huang, S.C., Pareek, A., Jensen, M., Lungren, M.P., Yeung, S., Chaudhari, A.S.: Self-supervised learning for medical image classification: a systematic review and implementation guidelines. NPJ Digital Med. 6(1), 74 (2023)

    Article  Google Scholar 

  16. Grill, J.B., et al.: Bootstrap your own latent - a new approach to self-supervised learning. Adv. Neural. Inf. Process. Syst. 33, 21271–21284 (2020)

    MATH  Google Scholar 

  17. He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)

    Google Scholar 

  18. Assran, M., et al.: Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15619–15629 (2023)

    Google Scholar 

  19. Gopinath, K., Desrosiers, C., Lombaert, H.: Graph convolutions on spectral embeddings for cortical surface parcellation. Med. Image Anal. 54, 297–305 (2019)

    Article  MATH  Google Scholar 

  20. Ha, S., Lyu, I.: SPHARM-Net: spherical harmonics-based convolution for cortical parcellation. IEEE Trans. Med. Imaging 41(10), 2739–2751 (2022)

    Article  MATH  Google Scholar 

  21. Li, X., Tan, J., Wang, P., Liu, H., Li, Z., Wang, W.: Anatomically constrained squeeze-and-excitation graph attention network for cortical surface parcellation. Comput. Biol. Med. 140, 105113 (2022)

    Article  MATH  Google Scholar 

  22. Besson, P., Parrish, T., Katsaggelos, A.K., Bandt, S.K.: Geometric deep learning on brain shape predicts sex and age. Comput. Med. Imaging Graph. 91, 101939 (2021)

    Article  Google Scholar 

  23. Vosylius, V., et al.: Geometric deep learning for post-menstrual age prediction based on the neonatal white matter cortical surface. In: Sudre, C.H., et al. (eds.) UNSURE/GRAIL -2020. LNCS, vol. 12443, pp. 174–186. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60365-6_17

    Chapter  MATH  Google Scholar 

  24. Zhang, Z., Girdhar, R., Joulin, A., Misra, I.: Self-supervised pretraining of 3D features on any point-cloud. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10252–10263 (2021)

    Google Scholar 

  25. Yu, X., Tang, L., Rao, Y., Huang, T., Zhou, J., Lu, J.: Point-BERT: pre-training 3D point cloud transformers with masked point modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19313–19322 (2022)

    Google Scholar 

  26. Pang, Y., Wang, W., Tay, F.E.H., Liu, W., Tian, Y., Yuan, L.: Masked autoencoders for point cloud self-supervised learning. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022. ECCV 2022. LNCS, vol. 13662. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20086-1_35

  27. Zhang, R., et al.: Point-M2AE: multi-scale masked autoencoders for hierarchical point cloud pre-training. Adv. Neural. Inf. Process. Syst. 35, 27061–27074 (2022)

    Google Scholar 

  28. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  MATH  Google Scholar 

  29. Devlin, J., Chang, M. W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2018). arXiv preprint arXiv:1810.04805

  30. Tian, X., Ran, H., Wang, Y., Zhao, H.: GeoMAE: masked geometric target prediction for self-supervised point cloud pre-training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13570–13580 (2023)

    Google Scholar 

  31. Haque, A., Moon, H., Hao, H., Didari, S., Woo, J.O., Bangert, P.: Unsupervised contrastive representation learning for 3D mesh segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 13, pp. 16222–16223 (2023)

    Google Scholar 

  32. Dosovitskiy, A., et al.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2020). arXiv preprint arXiv:2010.11929

  33. Liang, Y., Zhao, S., Yu, B., Zhang, J., He, F.: MeshMAE: masked autoencoders for 3D mesh data analysis. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) Computer Vision – ECCV 2022. ECCV 2022. LNCS, vol. 13663. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20062-5_3

  34. Dahan, S., et al.: Surface vision transformers: attention-based modelling applied to cortical analysis. In: International Conference on Medical Imaging with Deep Learning, pp. 282–303. PMLR (2022)

    Google Scholar 

  35. Dahan, S., Williams, L.Z.J., Rueckert, D., Robinson, E.C.: The multiscale surface vision transformer. In: Medical Imaging with Deep Learning (2024)

    Google Scholar 

  36. Dahan, S., Williams, L.Z.J., Guo, Y., Rueckert, D., Robinson, E.C.: Spatio-temporal encoding of brain dynamics with surface masked autoencoders. In: Medical Imaging with Deep Learning (2024)

    Google Scholar 

  37. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in Neural Information Processing Systems, vol. 29 (2016)

    Google Scholar 

  38. Loshchilov, I., Hutter, F.: Decoupled Weight Decay Regularization (2017). arXiv preprint arXiv:1711.05101

  39. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456. PMLR (2015)

    Google Scholar 

  40. Kingma, D. P., Ba, J.: Adam: A Method for Stochastic Optimization (2014). arXiv preprint arXiv:1412.6980

Download references

Acknowledgments

The authors are grateful to Petar Veličković for supporting the project with his valuable insights. The work reported in this paper carried out at BME, has been supported by the European Union project RRF-2.3.1-21-2022-00004 within the framework of the Artificial Intelligence National Laboratory. Project no. TKP2021-NVA-02 has been implemented with the support provided by the Ministry of Culture and Innovation of Hungary from the National Research, Development and Innovation Fund, financed under the TKP2021-NVA funding scheme. The presented work of D. Unyi was also supported by the ÚNKP-23-3-II-BME-399 New National Excellence Program of the Ministry for Culture and Innovation from the source of the National Research, Development and Innovation Fund. We thank the Governmental Agency for IT Development (KIFÜ) for the opportunity provided by the Komondor supercomputer, which they operate, and where the computations were performed.

Author information

Authors and Affiliations

  1. Department of Telecommunications and Media Informatics, Faculty of Electrical Engineering and Informatics, Budapest University of Technology and Economics, Műegyetem rkp. 3, 1111, Budapest, Hungary

    Dániel Unyi & Bálint Gyires-Tóth

Authors
  1. Dániel Unyi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bálint Gyires-Tóth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dániel Unyi .

Editor information

Editors and Affiliations

  1. Manchester Metropolitan University, Manchester, Lancashire, UK

    Moi Hoon Yap

  2. Manchester Metropolitan University, Manchester, UK

    Connah Kendrick

  3. Edge Hill University, Ormskirk, UK

    Ardhendu Behera

  4. Aberystwyth University, Aberystwyth, UK

    Timothy Cootes

  5. Aberystwyth University, Aberystwyth, UK

    Reyer Zwiggelaar

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Unyi, D., Gyires-Tóth, B. (2024). Self-Supervised Pretraining for Cortical Surface Analysis. In: Yap, M.H., Kendrick, C., Behera, A., Cootes, T., Zwiggelaar, R. (eds) Medical Image Understanding and Analysis. MIUA 2024. Lecture Notes in Computer Science, vol 14859. Springer, Cham. https://doi.org/10.1007/978-3-031-66955-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-66955-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-66954-5

  • Online ISBN: 978-3-031-66955-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation