Skip to main content
SpringerLink
Log in

BrainNet with Connectivity Attention for Individualized Predictions Based on Multi-Facet Connections Extracted from Resting-State fMRI Data

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Resting-state functional magnetic resonance imaging (RS-fMRI) has great potential for clinical applications. This study aimed to promote the performance of RS-fMRI-based individualized predictive models by introducing effective feature extraction and utilization strategies and making better use of information hidden in RS-fMRI data. We proposed a novel framework named multi-facet BrainNet with connectivity attention (MFBCA) to fulfill the purpose, and the framework is characterized by the following three strategies. First, in addition to the overwhelmingly popular functional connectivity, we also used distance correlation and weighted directed connectivity as multi-facet inputs for MFBCA. Second, a connectivity attention layer was proposed to force MFBCA to focus more on connections that are important for predictions. Finally, a BrainNet-based architecture with a feature fusion module was introduced to facilitate final predictions. We evaluated the performance of MFBCA with predictions of individuals' age and intelligence quotient as test cases based on three public RS-fMRI datasets. The results indicate that MFBCA can effectively utilize the information hidden in RS-fMRI data and outperform baselines. The predicted-vs-actual correlations for age predictions were 0.876 (7.314 years), 0.873 (8.121 years), and 0.681 (3.865 years), and for IQ predictions was 0.615 (4.287). The connectivity attention layer made it possible for us to determine the connections important for individualized predictions. MFBCA can be widely applied to predictions based on neuroimaging data from which connectivity maps can be extracted. Furthermore, the explicit physiological basis for predictions provided by the connectivity attention layer makes MFBCA a profitable choice for clinical applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The Cambridge Centre for Aging and Neuroscience (Cam-CAN) dataset analysed during the current study is available in a Linux file system and can be accessed via secure shell (SSH) connection or secure file-transfer (SFTP) protocol, http://www.mrc-cbu.cam.ac.uk/datasets/camcan/. The Enhanced Nathan Kline Institute-Rockland sample (NKI) dataset and the Autism Imaging Data Exchange (ABIDE I) dataset analysed during the current study are available in the FCP-INDI Neuroimaging Data repository, http://fcon_1000.projects.nitrc.org/indi/s3/index.html.

References

  1. Kanavati F, Ichihara S, Rambeau M, Iizuka O, Arihiro K, Tsuneki M. Deep learning models for gastric signet ring cell carcinoma classification in whole slide images. Technol Cancer Res Treat. 2021. https://doi.org/10.1177/15330338211027901.

    Article  Google Scholar 

  2. Xu W, Zhao L, Li J, Shang S, Ding X, Wang T. Detection and classification of tea buds based on deep learning. Comput Electron Agric. 2022;192:106547. https://doi.org/10.1016/j.compag.2021.106547.

    Article  Google Scholar 

  3. Wang Z, Bai L, Song G, Zhang J, Tao J, Mulvenna MD, Bond RR, Chen L. An oil well dataset derived from satellite-based remote sensing. Remote Sens. 2021;13(6):1132. https://doi.org/10.3390/rs13061132.

    Article  Google Scholar 

  4. Cole JH, Franke K. Predicting age using neuroimaging: Innovative brain ageing biomarkers. TINS. 2017;40:681–90. https://doi.org/10.1016/j.tins.2017.10.001.

    Article  Google Scholar 

  5. Kubilius J, Schrimpf M, Kar K, Rajalingham R, Hong H, Majaj N, et al. Brain-like object recognition with high-performing shallow recurrent ANNs. Advances in neural information processing systems, 2019;32. https://dl.acm.org/doi/pdf/10.5555/3454287.3455434.

  6. Zhang C, Baum SA, Adduru VR, Biswal BB, Michael AM. Test-retest reliability of dynamic functional connectivity in resting state fMRI. Neuroimage. 2018;183:907–18. https://doi.org/10.1016/j.neuroimage.2018.08.021.

    Article  Google Scholar 

  7. Gadgil S, Zhao Q, Pfefferbaum A, Sullivan EV, Adeli E, Pohl KM. Spatio-temporal graph convolution for resting-state fmri analysis. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham. 2020;528–538. https://doi.org/10.1007/978-3-030-59728-3_52.

  8. Morris TP, Kucyi A, Anteraper SA, et al. Resting state functional connectivity provides mechanistic predictions of future changes in sedentary behavior. Sci Rep. 2022;12:1–11. https://doi.org/10.1038/s41598-021-04738-y.

    Article  Google Scholar 

  9. Abraham A, Milham MP, Martino AD, Craddock RC, Samaras D, Thiron B, et al. Deriving reproducible biomarkers from multi-site resting-state data: An autism-based example. Neuroimage. 2017;147:736–45. https://doi.org/10.1016/j.neuroimage.2016.10.045.

    Article  Google Scholar 

  10. Yoo K, Rosenberg MD, Noble S, Scheinost D, Constable RT, Chun MM. Multivariate approaches improve the reliability and validity of functional connectivity and prediction of individual behaviors. Neuroimage. 2019;197:212–23. https://doi.org/10.1016/j.neuroimage.2019.04.060.

    Article  Google Scholar 

  11. Geerligs L, Henson RN. Functional connectivity and structural covariance between regions of interest can be measured more accurately using multivariate distance correlation. Neuroimage. 2016;135:16–31. https://doi.org/10.1016/j.neuroimage.2016.04.047.

    Article  Google Scholar 

  12. Radovanovic M, Nanopoulos A, Ivanovic M. Hubs in space: Popular nearest neighbors in high-dimensional data. JMLR. 2010;11: 2487–2531. https://dl.acm.org/doi/pdf/10.5555/1756006.1953015.

  13. Kawahara J, Brown CJ, Miller SP, Booth BG, Chau V, Grunau RE, et al. BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage. 2017;146:1038–49. https://doi.org/10.1016/j.neuroimage.2016.09.046.

    Article  Google Scholar 

  14. Shi W, Yan G, Li Y, Li H, Liu T, Sun C, et al. Fetal brain age estimation and anomaly detection using attention-based deep ensembles with uncertainty. NeuroImage. 2020;223:117316. https://doi.org/10.1016/j.neuroimage.2020.117316.

    Article  Google Scholar 

  15. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv. 2014. https://doi.org/10.48550/arXiv.1409.0473.

  16. Li H, Satterthwaite TD, Fan Y. Brain age prediction based on resting-state functional connectivity patterns using convolutional neural networks. 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018). IEEE. 2018: 101–104. https://doi.org/10.1109/ISBI.2018.8363532.

  17. Xiao L, Cai B, Qu G, Stephen JM, Wilson TW. Distance correlation-based brain functional connectivity estimation and non-convex multi-task learning for developmental fMRI studies. IEEE Trans Biomed Eng. 2022. https://doi.org/10.1109/TBME.2022.3160447.

  18. Hu D, Zhang H, Wu Z, Wang F, Wang L, Smith JK, et al. Disentangled-multimodal adversarial autoencoder: Application to infant age prediction with incomplete multimodal neuroimages. IEEE Trans Med Imag. 2020;39:4137–49. https://doi.org/10.1109/TMI.2020.3013825.

    Article  Google Scholar 

  19. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–44. https://doi.org/10.1038/nature14539.

    Article  Google Scholar 

  20. You Y, Chen T, Wang Z, Shen Y. L2-gcn: Layer-wise and learned efficient training of graph convolutional networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020;2127–2135. https://doi.org/10.1109/CVPR42600.2020.00220.

  21. El-Gazzar A, Thomas RM, Wingen GV. Dynamic adaptive spatio-temporal graph convolution for fMRI modelling. International Workshop on Machine Learning in Clinical Neuroimaging. Springer, Cham. 2021;125–134. https://doi.org/10.1007/978-3-030-87586-2_13.

  22. Taylor JR, Williams N, Cusack R, Auer T, Meredith AS, Dixon M, et al. The Cambridge Centre for Ageing and Neuroscience (Cam-CAN) data repository: Structural and functional MRI, MEG, and cognitive data from a cross-sectional adult lifespan sample. Neuroimage. 2017;144:262–9. https://doi.org/10.1016/j.neuroimage.2015.09.018.

    Article  Google Scholar 

  23. Nooner KB, Colcombe SJ, Tobe RH, Mennes M, Benedict MM, Moreno AL, et al. The NKI-Rockland sample: a model for accelerating the pace of discovery science in psychiatry. Front Neurosci. 2012;6:152. https://doi.org/10.3389/fnins.2012.00152.

    Article  Google Scholar 

  24. Di Martino A, Yan CG, Li Q, Denio E, Castellanos FX, Alaerts K, et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol Psychiatry. 2014;19(6):659–67. https://doi.org/10.1038/mp.2013.78.

    Article  Google Scholar 

  25. Jenkinson M, Beckmann CF, Behrens TEJ, Woolrich MW, Smith SM. Fsl. NeuroImage. 2012;62:782–90. https://doi.org/10.1016/j.neuroimage.2011.09.015.

    Article  Google Scholar 

  26. Power JD, Cohen AL, Nelson SM, Wig SG, Barnes KA, Church JA, et al. Functional network organization of the human brain. Neuron. 2011;72:665–78. https://doi.org/10.1016/j.neuron.2011.09.006.

    Article  Google Scholar 

  27. Seidman LJ, Thermenos HW, Poldrack RA, Peace NK, Koch JK, Faraone SV, et al. Altered brain activation in dorsolateral prefrontal cortex in adolescents and young adults at genetic risk for schizophrenia: An fMRI study of working memory. Schizophr Res. 2006;85:58–72. https://doi.org/10.1016/j.schres.2006.03.019.

    Article  Google Scholar 

  28. Xia M, Wang J, He Y. BrainNet Viewer: a network visualization tool for human brain connectomics. PLoS ONE. 2013;8:e68910. https://doi.org/10.1371/journal.pone.0068910.

    Article  Google Scholar 

  29. van den Bos W, Rodriguez CA, Schweitzer JB, McClure SM. Connectivity strength of dissociable striatal tracts predict individual differences in temporal discounting. J Neurosci. 2014;34:10298–310. https://doi.org/10.1523/JNEUROSCI.4105-13.2014.

    Article  Google Scholar 

  30. Supekar K, Swigart AG, Tenison C, Jolles DD, Rosenberg-Lee M, Fuchs L, et al. Neural predictors of individual differences in response to math tutoring in primary-grade school children. Proc Natl Acad Sci. 2013;110:8230–5. https://doi.org/10.1073/pnas.1222154110.

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China (Grant Nos. 62276021, 61773048).

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing, 100044, China

    Hao Ma, Fan Wu, Yun Guan, Le Xu, Jiangcong Liu & Lixia Tian

  2. Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing, 100044, China

    Hao Ma & Lixia Tian

Authors
  1. Hao Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yun Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Le Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiangcong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lixia Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lixia Tian.

Ethics declarations

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, H., Wu, F., Guan, Y. et al. BrainNet with Connectivity Attention for Individualized Predictions Based on Multi-Facet Connections Extracted from Resting-State fMRI Data. Cogn Comput 15, 1566–1580 (2023). https://doi.org/10.1007/s12559-023-10133-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-023-10133-8

Keywords

Search

Navigation