Skip to main content
SpringerLink
Log in

A joint parcellation and boundary network with multi-rate-shared dilated graph attention for cortical surface parcellation

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Cortical surface parcellation aims to segment the surface into anatomically and functionally significant regions, which are crucial for diagnosing and treating numerous neurological diseases. However, existing methods generally ignore the difficulty in learning labeling patterns of boundaries, hindering the performance of parcellation. To this end, this paper proposes a joint parcellation and boundary network (JPBNet) to promote the effectiveness of cortical surface parcellation. Its core is developing a multi-rate-shared dilated graph attention (MDGA) module and incorporating boundary learning into the parcellation process. The former, in particular, constructs a dilated graph attention strategy, extending the dilated convolution from regular data to irregular graph data. We fuse it with different dilated rates to extract context information in various scales by devising a shared graph attention layer. After that, a boundary enhancement module and a parcellation enhancement module based on graph attention mechanisms are built in each layer, forcing MDGA to capture informative and valuable features for boundary detection and parcellation tasks. Integrating MDGA, the boundary enhancement module, and the parcellation enhancement module at each layer to supervise boundary and parcellation information, an effective JPBNet is formed by stacking multiple layers. Experiments on the public dataset reveal that the proposed method outperforms comparison methods and performs well on boundaries for cortical surface parcellation.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Messé A (2020) Parcellation influence on the connectivity-based structure-function relationship in the human brain. Hum Brain Mapp 41(5):1167–1180

    Article  PubMed  Google Scholar 

  2. Eickhoff SB, Yeo BT, Genon S (2018) Imaging-based parcellations of the human brain. Nat Rev Neurosci 19(11):672–686

    Article  CAS  PubMed  Google Scholar 

  3. Assem M, Glasser MF, Van Essen DC, Duncan J (2020) A domain-general cognitive core defined in multimodally parcellated human cortex. Cereb Cortex 30(8):4361–4380

    Article  PubMed  PubMed Central  Google Scholar 

  4. Arbabshirani MR, Plis S, Sui J, Calhoun VD (2017) Single subject prediction of brain disorders in neuroimaging: promises and pitfalls. NeuroImage 145:137–165

    Article  PubMed  Google Scholar 

  5. Fischl B, Sereno MI, Dale AM (1999) Cortical surface-based analysis: II. Inflation, flattening, and a surface-based coordinate system. NeuroImage 9(2):195–207

  6. Cheng J, Dalca AV, Fischl B, Zöllei L, Initiative ADN et al (2020) Cortical surface registration using unsupervised learning. NeuroImage 221:117161

    Article  PubMed  Google Scholar 

  7. Kaltenmark I, Deruelle C, Brun L, Lefèvre J, Coulon O, Auzias G (2020) Group-level cortical surface parcellation with sulcal pits labeling. Med Image Anal 66:101749

    Article  PubMed  PubMed Central  Google Scholar 

  8. Silva F, Guevara M, Poupon C, Mangin JF, Hernández C, Guevara P (2019) Cortical surface parcellation based on graph representation of short fiber bundle connections. In: Proceedings of the IEEE international symposium on biomedical imaging, IEEE, Venice, Italy, pp 1479–1482

  9. Vergara C, Silva F, Huerta I, López-López N, Vázquez A, Houenou J, Poupon C, Mangin JF, Hernández C, Guevara P (2021) Group-wise cortical surface parcellation based on inter-subject fiber clustering. In: Proceedings of International Conference of the IEEE Engineering in Medicine & Biology Society, Mexico, pp 2655–2659

  10. Glasser MF, Coalson TS, Robinson EC, Hacker CD, Harwell J, Yacoub E, Ugurbil K, Andersson J, Beckmann CF, Jenkinson M et al (2016) A multi-modal parcellation of human cerebral cortex. Nature 536(7615):171–178

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. López-López N, Vázquez A, Poupon C, Mangin JF, Ladra S, Guevara P (2020) GeoSP: a parallel method for a cortical surface parcellation based on geodesic distance. In: Proceedings of International Conference Of The IEEE Engineering In Medicine & Biology Society, Montreal, Canada, pp 1696–1700

  12. Lefranc S, Roca P, Perrot M, Poupon C, Le Bihan D, Mangin JF, Rivière D (2016) Groupwise connectivity-based parcellation of the whole human cortical surface using watershed-driven dimension reduction. Med Image Anal 30:11–29

    Article  PubMed  Google Scholar 

  13. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Boston, USA, pp 3431–3440

  14. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of international conference on medical image computing and computer assisted intervention. Munich, Germany, pp 234– 241

  15. Wu Z, Li G, Wang L, Shi F, Lin W, Gilmore JH, Shen D (2018) Registration-free infant cortical surface parcellation using deep convolutional neural networks. In: Proceedings of international conference on medical image computing and computer assisted intervention. Granada, Spain, pp 672–680

  16. Parvathaneni P, Bao S, Nath V, Woodward ND, Claassen DO, Cascio CJ, Zald DH, Huo Y, Landman BA, Lyu I (2019) Cortical surface parcellation using spherical convolutional neural networks. In: Proceedings of international conference on medical image computing and computer assisted intervention. Shenzhen, China, pp 501–509

  17. Zhao F, Xia S, Wu Z, Duan D, Wang L, Lin W, Gilmore JH, Shen D, Li G (2019) Spherical U-Net on cortical surfaces: methods and applications. In: Proceedings of information processing in medical imaging. Hong Kong, China, pp 855–866

  18. Zhao F, Wu Z, Wang L, Lin W, Gilmore JH, Xia S, Shen D, Li G (2021) Spherical deformable u-net: application to cortical surface parcellation and development prediction. IEEE Transactions on Medical Imaging 40(4):1217–1228

    Article  PubMed  PubMed Central  Google Scholar 

  19. Ha S, Lyu I (2022) SPHARM-net: spherical harmonics-based convolution for cortical parcellation. IEEE Trans Med Imaging 41(10):2739–2751

    Article  PubMed  Google Scholar 

  20. Cucurull G, Wagstyl K, Casanova A, Veličković P, Jakobsen E, Drozdzal M, Romero A, Evans A, Bengio Y (2018) Convolutional neural networks for mesh-based parcellation of the cerebral cortex. In: Proceedings of international conference on medical imaging with deep learning. Amsterdam, Netherlands

  21. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of international conference on learning representations. Toulon, France

  22. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: Proceedings of international conference on learning representations. Vancouver, Canada

  23. Wu Z, Zhao F, Xia J, Wang L, Lin W, Gilmore JH, Li G, Shen D (2019) Intrinsic patch-based cortical anatomical parcellation using graph convolutional neural network on surface manifold. In: Proceedings of international conference on medical image computing and computer assisted intervention. Shenzhen, China, pp 492–500

  24. Monti F, Boscaini D, Masci J, Rodola E, Svoboda J, Bronstein MM (2017) Geometric deep learning on graphs and manifolds using mixture model CNNs. In: Proceedings of the IEEE conference on computer vision and pattern recognition. Hawaii, USA, pp 5115–5124

  25. Gopinath K, Desrosiers C, Lombaert H (2019) Graph convolutions on spectral embeddings for cortical surface parcellation. Med Image Anal 54:297–305

    Article  PubMed  Google Scholar 

  26. Zhang W, Wang Y (2019) Geometric brain surface network for brain cortical parcellation. In: Proceedings of International workshop on graph learning in medical imaging. Shenzhen, China, pp 120–129

  27. Li X, Tan J, Wang P, Liu H, Li Z, Wang W (2022) Anatomically constrained squeeze-and-excitation graph attention network for cortical surface parcellation. Comput Biol Med 140:105113

    Article  PubMed  Google Scholar 

  28. Eschenburg KM, Grabowski TJ, Haynor DR (2021) Learning cortical parcellations using graph neural networks. Front Neurosci 15:1776

    Article  Google Scholar 

  29. Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions. In: Proceedings of international conference on learning representations. San Juan, Puerto Rico

  30. Cheng H, Wu K, Tian J, Ma K, Gu C, Guan X (2022) Colon tissue image segmentation with MWSI-NET. Med Biol Eng Comput 60:727–737

    Article  PubMed  Google Scholar 

  31. Zuo B, Lee F, Chen Q (2022) An efficient U-shaped network combined with edge attention module and context pyramid fusion for skin lesion segmentation. Med Biol Eng Comput 60:1987–2000

    Article  PubMed  Google Scholar 

  32. Maas AL, Hannun AY, Ng AY, et al (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of international conference on machine learning. Atlanta,USA

  33. Wang K, Shen W, Yang Y, Quan X, Wang R (2020) Relational graph attention network for aspect-based sentiment analysis. In: Proceedings of annual meeting of association for computational linguistics. Online, pp 3229–3238

  34. Brody S, Alon U, Yahav E (2022) How attentive are graph attention networks? In: Proceedings of international conference on learning representations, Online

  35. Chaoyue D, Shiliang S, Jing Z (2023) MST-GAT: a multimodal spatial emporal graph attention network for time series anomaly detection. Inf Fusion 89:527–536

    Article  Google Scholar 

  36. He R, Gopinath K, Desrosiers C, Lombaert H (2020) Spectral graph transformer networks for brain surface parcellation. In: Proceedings of the IEEE international symposium on biomedical imaging. Iowa, USA, pp 372–376

  37. Klein A, Ghosh SS, Bao FS, Giard J, Häme Y, Stavsky E, Lee N, Rossa B, Reuter M, Chaibub Neto E et al (2017) Mindboggling morphometry of human brains. PLoS Comput Biol 13(2):e1005350

    Article  PubMed  PubMed Central  Google Scholar 

  38. Klein A, Tourville J (2012) 101 labeled brain images and a consistent human cortical labeling protocol. Front Neurosci 6:171

    Article  PubMed  PubMed Central  Google Scholar 

  39. Besl P (1992) A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14:239–256

    Article  Google Scholar 

  40. Fischl B (2012) Freesurfer. NeuroImage 62(2):774–781

    Article  PubMed  Google Scholar 

Download references

Funding

This work was supported by the Open Research Fund of Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province, Zhejiang Normal University (Grant No. jykf21001w), supported by the National Natural Science Foundation of China (Grants No. 62006215, 62176244, and 62172370), and also supported by the Zhejiang Provincial Natural Science Foundation (Grant No. LY22F020004).

Author information

Authors and Affiliations

  1. College of Sciences, China Jiliang University, Hangzhou, 310018, China

    Siqi Liu, Hailiang Ye, Bing Yang & Feilong Cao

  2. Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University, Jinhua, 321004, China

    Ming Li

Authors
  1. Siqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hailiang Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Feilong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Hailiang Ye or Feilong Cao.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Ye, H., Yang, B. et al. A joint parcellation and boundary network with multi-rate-shared dilated graph attention for cortical surface parcellation. Med Biol Eng Comput 62, 537–549 (2024). https://doi.org/10.1007/s11517-023-02942-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02942-8

Keywords

Search

Navigation