Skip to main content
Springer Nature Link
Log in

Utility of Equivariant Message Passing in Cortical Mesh Segmentation

  • Conference paper
  • First Online:
Medical Image Understanding and Analysis (MIUA 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13413))

Included in the following conference series:

  • 2766 Accesses

Abstract

The automated segmentation of cortical areas has been a long-standing challenge in medical image analysis. The complex geometry of the cortex is commonly represented as a polygon mesh, whose segmentation can be addressed by graph-based learning methods. When cortical meshes are misaligned across subjects, current methods produce significantly worse segmentation results, limiting their ability to handle multi-domain data. In this paper, we investigate the utility of E(n)-equivariant graph neural networks (EGNNs), comparing their performance against plain graph neural networks (GNNs). Our evaluation shows that GNNs outperform EGNNs on aligned meshes, due to their ability to leverage the presence of a global coordinate system. On misaligned meshes, the performance of plain GNNs drop considerably, while E(n)-equivariant message passing maintains the same segmentation results. The best results can also be obtained by using plain GNNs on realigned data (co-registered meshes in a global coordinate system).

P. Veličković and B. Gyires-Tóth—Equal contribution

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/daniel-unyi-42/Equivariant-Cortical-Mesh-Segmentation.

References

  1. Liu, X., et al.: A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digital Health 1(6), e271–e297 (2019)

    Google Scholar 

  2. Wen, D., Wei, Z., Zhou, Y., Li, G., Zhang, X., Han, W.: Deep learning methods to process fMRI data and their application in the diagnosis of cognitive impairment: a brief overview and our opinion. Front. Neuroinform. 12, 23 (2018)

    Article  Google Scholar 

  3. Tahmassebi, A., Gandomi, A.H., McCann, I., Schulte, M.H., Goudriaan, A.E., Meyer-Baese, A.: Deep learning in medical imaging: fMRI big data analysis via convolutional neural networks. In: Proceedings of the Practice and Experience on Advanced Research Computing, pp. 1–4 (2018)

    Google Scholar 

  4. Cai, L., Gao, J., Zhao, D.: A review of the application of deep learning in medical image classification and segmentation. Annal. Transl. Med. 8(11) (2020)

    Google Scholar 

  5. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  6. Chen, W., Liu, B., Peng, S., Sun, J., Qiao, X.: S3D-UNet: separable 3D U-Net for brain tumor segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11384, pp. 358–368. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11726-9_32

    Chapter  Google Scholar 

  7. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, 30 (2017)

    Google Scholar 

  8. Chen, J., et al.: TransUNet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)

  9. Cao, H., et al.: Swin-Unet: Unet-like pure transformer for medical image segmentation. arXiv preprint arXiv:2105.05537 (2021)

  10. Zhang, Y., Liu, H., Hu, Q.: TransFuse: fusing transformers and CNNs for medical image segmentation. In: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 14–24. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_2

    Chapter  Google Scholar 

  11. Lorensen, W., Cline, H.: Marching cubes: a high resolution 3D surface construction algorithm. Comput. Graph. 21(4), 163–169 (1987)

    Article  Google Scholar 

  12. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: International Conference on Machine Learning, pp. 1263–1272. PMLR, July 2017

    Google Scholar 

  13. Satorras, V.G., Hoogeboom, E., Welling, M.: E(n) equivariant graph neural networks. In: International Conference on Machine Learning, pp. 9323–9332. PMLR, July 2021

    Google Scholar 

  14. Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013)

  15. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in Neural Information Processing Systems, 29 (2016)

    Google Scholar 

  16. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

  17. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  18. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Networks Learn. Syst. 32(1), 4–24 (2020)

    Article  MathSciNet  Google Scholar 

  19. Bronstein, M.M., Bruna, J., Cohen, T., Veličković, P.: Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478 (2021)

  20. Uy, M.A., Pham, Q.H., Hua, B.S., Nguyen, T., Yeung, S.K.: Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1588–1597 (2019)

    Google Scholar 

  21. Ramakrishnan, R., Dral, P.O., Rupp, M., Von Lilienfeld, O.A.: Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 1(1), 1–7 (2014)

    Article  Google Scholar 

  22. Kipf, T., Fetaya, E., Wang, K.C., Welling, M., Zemel, R.: Neural relational inference for interacting systems. In: International Conference on Machine Learning, pp. 2688–2697. PMLR, July 2018

    Google Scholar 

  23. Köhler, J., Klein, L., & Noé, F.: Equivariant Flows: sampling configurations for multi-body systems with symmetric energies. arXiv preprint arXiv:1910.00753 (2019)

  24. Thomas, N., Smidt, T., Kearnes, S., Yang, L., Li, L., Kohlhoff, K., Riley, P.: Tensor field networks: Rotation-and translation-equivariant neural networks for 3D point clouds. arXiv preprint arXiv:1802.08219 (2018)

  25. Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. 5, 698–700 (1987)

    Article  Google Scholar 

  26. Glasser, M.F., et al.: A multi-modal parcellation of human cerebral cortex. Nature 536(7615), 171–178 (2016)

    Google Scholar 

  27. Cheng, J., Dalca, A.V., Fischl, B., Zöllei, L., Initiative, A.D.N.: Cortical surface registration using unsupervised learning. Neuroimage 221, 117161 (2020)

    Article  Google Scholar 

  28. Seong, S.B., Pae, C., Park, H.J.: Geometric convolutional neural network for analyzing surface-based neuroimaging data. Front. Neuroinform. 12, 42 (2018)

    Article  Google Scholar 

  29. Zhao, F., Xia, S., Wu, Z., Duan, D., Wang, L., Lin, W., Gilmore, J.H., Shen, D., Li, G.: Spherical U-Net on cortical surfaces: methods and applications. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 855–866. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_67

    Chapter  Google Scholar 

  30. Zhao, F., et al.: Spherical deformable U-Net: application to cortical surface parcellation and development prediction. IEEE Trans. Med. Imaging 40(4), 1217–1228 (2021)

    Google Scholar 

  31. Fawaz, A., et al.: Benchmarking geometric deep learning for cortical segmentation and neurodevelopmental phenotype prediction. bioRxiv, 2021.12.01.470730 (2021)

    Google Scholar 

  32. Gopinath, K., Desrosiers, C., Lombaert, H.: Graph convolutions on spectral embeddings for cortical surface parcellation. Med. Image Anal. 54, 297–305 (2019)

    Article  Google Scholar 

  33. Cucurull, G., et al.: Convolutional neural networks for mesh-based parcellation of the cerebral cortex. In: International Conference on Medical Imaging with Deep Learning (2018)

    Google Scholar 

  34. Gopinath, K., Desrosiers, C., Lombaert, H.: Graph domain adaptation for alignment-invariant brain surface segmentation. In: Sudre, C.H., Fehri, H., Arbel, T., Baumgartner, C.F., Dalca, A., Tanno, R., Van Leemput, K., Wells, W.M., Sotiras, A., Papiez, B., Ferrante, E., Parisot, S. (eds.) UNSURE/GRAIL -2020. LNCS, vol. 12443, pp. 152–163. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60365-6_15

    Chapter  Google Scholar 

  35. Jakobsen, E., Liem, F., Klados, M.A., Bayrak, Ş, Petrides, M., Margulies, D.S.: Automated individual-level parcellation of Broca’s region based on functional connectivity. Neuroimage 170, 41–53 (2018)

    Article  Google Scholar 

  36. Jakobsen, E., Böttger, J., Bellec, P., Geyer, S., Rübsamen, R., Petrides, M., Margulies, D.S.: Subdivision of Broca’s region based on individual-level functional connectivity. Eur. J. Neurosci. 43(4), 561–571 (2016)

    Article  Google Scholar 

  37. Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso, M.J., Arbel, T., Carneiro, G., Syeda-Mahmood, T., Tavares, J.M.R.S., Moradi, M., Bradley, A., Greenspan, H., Papa, J.P., Madabhushi, A., Nascimento, J.C., Cardoso, J.S., Belagiannis, V., Lu, Z. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 240–248. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_28

    Chapter  Google Scholar 

  38. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  39. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)

    Article  Google Scholar 

  40. Hendrycks, D., Gimpel, K.: Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415 (2016)

Download references

Acknowledgement

The authors are especially grateful to Konrad Wagstyl for his valuable insights into the data. The research reported in this paper has been partly supported by the Hungarian National Laboratory of Artificial Intelligence funded by the NRDIO under the auspices of the Hungarian Ministry for Innovation and Technology. We thank for the usage of the ELKH Cloud GPU infrastructure (https://science-cloud.hu/) that significantly helped us achieve the results published in this paper. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the NVIDIA GPU also used for this research. The publication of the work reported herein has been supported by ETDB at BME.

Author information

Authors and Affiliations

  1. Department of Telecommunications and Media Informatics, Budapest University of Technology and Economics, Műegyetem rkp. 3., 1111, Budapest, Hungary

    Dániel Unyi & Bálint Gyires-Tóth

  2. Department of Mathematics, Imperial College London, London, SW7 2AZ, UK

    Ferdinando Insalata

  3. DeepMind, London, UK

    Petar Veličković

Authors
  1. Dániel Unyi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ferdinando Insalata
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Petar Veličković
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bálint Gyires-Tóth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dániel Unyi .

Editor information

Editors and Affiliations

  1. Imperial College London, London, UK

    Guang Yang

  2. University of Cambridge, Cambridge, UK

    Angelica Aviles-Rivero

  3. University of Cambridge, Cambridge, UK

    Michael Roberts

  4. University of Cambridge, Cambridge, UK

    Carola-Bibiane Schönlieb

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Unyi, D., Insalata, F., Veličković, P., Gyires-Tóth, B. (2022). Utility of Equivariant Message Passing in Cortical Mesh Segmentation. In: Yang, G., Aviles-Rivero, A., Roberts, M., Schönlieb, CB. (eds) Medical Image Understanding and Analysis. MIUA 2022. Lecture Notes in Computer Science, vol 13413. Springer, Cham. https://doi.org/10.1007/978-3-031-12053-4_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-12053-4_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-12052-7

  • Online ISBN: 978-3-031-12053-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation