Abstract
Since the complex brain functions are achieved by the interaction of functional brain networks with the specific spatial distributions and temporal dynamics, modeling the spatial and temporal patterns of functional brain networks based on 4D fMRI data offers a way to understand the brain functional mechanisms. Matrix decomposition methods and deep learning methods have been developed to provide solutions. However, the underlying nature of functional brain networks remains unclear due to underutilizing the spatio-temporal characteristics of 4D fMRI input in previous methods. To address this problem, we propose a novel Guided Attention 4D Convolutional Neural Network (GA-4DCNN) to model spatial and temporal patterns of functional brain networks simultaneously. GA-4DCNN consists of two subnetworks: the spatial 4DCNN and the temporal Guided Attention (GA) network. The 4DCNN firstly extracts spatio-temporal characteristics of fMRI input to model the spatial pattern, while the GA network further models the corresponding temporal pattern guided by the modeled spatial pattern. Based on two task fMRI datasets from the Human Connectome Project, experimental results show that the proposed GA-4DCNN has superior ability and generalizability in modeling spatial and temporal patterns compared to other state-of-the-art methods. This study provides a new useful tool for modeling and understanding brain function.
This is a student paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Naselaris, T., Kay, K.N., Nishimoto, S., Gallant, J.L.: Encoding and decoding in fMRI. Neuroimage 56(2), 400–410 (2011)
Logothetis, N.K.: What we can do and what we cannot do with fMRI. Nature 453(7197), 869–878 (2008)
Andersen, A.H., Gash, D.M., Avison, M.J.: Principal component analysis of the dynamic response measured by fMRI: a generalized linear systems framework. Magn. Reson. Imaging 17(6), 795–815 (1999)
McKeown, M.J., Hansen, L.K., Sejnowski, T.J.: Independent component analysis of functional MRI: what is signal and what is noise? Curr. Opin. Neurobiol. 13(5), 620–629 (2003)
Lv, J.L., et al.: Sparse representation of whole-brain fMRI signals for identification of functional networks. Med. Image Anal. 20(1), 112–134 (2015)
Jiang, X., et al.: Sparse representation of HCP grayordinate data reveals novel functional architecture of cerebral cortex. Hum. Brain Mapp. 36(12), 5301–5319 (2015)
Zhang, W., et al.: Experimental comparisons of sparse dictionary learning and independent component analysis for brain network inference from fMRI data. IEEE Trans. Biomed. Eng. 66(1), 289–299 (2019)
Hjelm, R.D., Plis, S.M., Calhoun, V.: Recurrent neural networks for spatiotemporal dynamics of intrinsic networks from fMRI data. NIPS Brains Bits (2016)
Huang, H., et al.: Modeling task fMRI data via deep convolutional autoencoder. IEEE Trans. Med. Imaging 37(7), 1551–1561 (2018)
Zhang, W., et al.: Identify hierarchical structures from task-based fMRI data via hybrid spatiotemporal neural architecture search net. Med. Image Comput. Comput. Assist. Interv. 2019, 745–753 (2019)
Zhang, W., et al.: Hierarchical organization of functional brain networks revealed by hybrid spatiotemporal deep learning. Brain Connect. 10(2), 72–82 (2020)
Zhao, Y., et al.: Four-dimensional modeling of fMRI data via spatio-temporal convolutional neural networks (ST-CNNs). IEEE Trans. Cognit. Dev. Syst. 12(3), 451–460 (2020)
Zhao, Y., et al.: Modeling 4D fMRI data via spatio-temporal convolutional neural networks (ST-CNNs). In: Medical Image Computing and Computer Assisted Intervention (2018)
Jiang, X., Zhang, T., Zhang, S., Kendrick, K.M., Liu, T.M.: Fundamental functional differences between gyri and sulci: implications for brain function, cognition, and behavior. Psychoradiology 1(1), 23–41 (2021)
Wang, Z.Y., Zou, N., Shen, D.G., Ji, S.W.: Non-local U-nets for biomedical image segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 4, pp. 6315–6322 (2020)
Van Essen, D.C., Smith, S.M., Barch, D.M., Behrens, T.E.J., Yacoub, E., Ugurbil, K.: The WU-Minn human connectome project: an overview. NeuroImage 80, 62–79 (2013)
Woolrich, M.W., Ripley, B.D., Brady, M., Smith, S.M.: Temporal autocorrelation in univariate linear modeling of fMRI data. Neuroimage 14(6), 1370–1386 (2001)
Jenkinson, M., Beckmann, C.F., Behrens, T.E., Woolrich, W.M., Smith, SM.: FSL. Neuroimage 62(2), 782–790 (2012)
Dale, A.M., Fischl, B., Sereno, M.I.: Cortical surface-based analysis. Neuroimage 9(2), 179–194 (1999)
Smith, S.M., et al.: Correspondence of the brain’s functional architecture during activation and rest. Proc. Natl. Acad. Sci. 106(31), 13040–13045 (2009)
Raichle, M.E., MacLeod, A.M., Snyder, A.Z., Powers, W.J., Gusnard, D.A., Shulman, D.L.: A default mode of brain function. Proc. Natl. Acad. Sci. U.S.A. 98(2), 676–682 (2001)
Greicius, M.D., Krasnow, B., Reiss, A.L., Menon, V.: Functional connectivity in the resting brain: a network analysis of the default mode hypothesis. Proc. Natl. Acad. Sci. U.S.A. 100(1), 253–258 (2003)
Vaswani, A., et al.: Attention is all you need. In: 31st Annual Conference on Neural Information Processing Systems, pp. 6000–6010 (2017)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations 2015 (2015)
Acknowledgements
This work was partly supported by the National Natural Science Foundation of China (61976045), Sichuan Science and Technology Program (2021YJ0247), Sichuan Science and Technology Program for Returned Overseas People (2021LXHGKJHD24), Key Scientific and Technological Projects of Guangdong Province Government (2018B030335001), National Natural Science Foundation of China (62006194), the Fundamental Research Funds for the Central Universities (3102019QD005), High-level researcher start-up projects (06100-20GH020161), and National Natural Science Foundation of China (31971288 and U1801265).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yan, J. et al. (2021). A Guided Attention 4D Convolutional Neural Network for Modeling Spatio-Temporal Patterns of Functional Brain Networks. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13021. Springer, Cham. https://doi.org/10.1007/978-3-030-88010-1_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-88010-1_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88009-5
Online ISBN: 978-3-030-88010-1
eBook Packages: Computer ScienceComputer Science (R0)