skip to main content
10.1145/3503161.3548339acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Complementary Graph Representation Learning for Functional Neuroimaging Identification

Published: 10 October 2022 Publication History

Abstract

The functional connectomics study on resting state functional magnetic resonance imaging (rs-fMRI) data has become a popular way for early disease diagnosis. However, previous methods did not jointly consider the global patterns, the local patterns, and the temporal information of the blood-oxygen-level-dependent (BOLD) signals, thereby restricting the model effectiveness for early disease diagnosis. In this paper, we propose a new graph convolutional network (GCN) method to capture local and global patterns for conducting dynamically functional connectivity analysis. Specifically, we first employ the sliding window method to partition the original BOLD signals into multiple segments, aiming at achieving the dynamically functional connectivity analysis, and then design a multi-view node classification and a temporal graph classification to output two kinds of representations, which capture the temporally global patterns and the temporally local patterns, respectively. We further fuse these two kinds of representation by the weighted concatenation method whose effectiveness is experimentally proved as well. Experimental results on real datasets demonstrate the effectiveness of our method, compared to comparison methods on different classification tasks.

Supplementary Material

MP4 File (MM22-fp2657.mp4)
This paper designed a complementary graph representation learning framework by jointly considering the global patterns, the local patterns, and the temporal information, for functional neuroimaging identification.

References

[1]
Amirreza Farnoosh and Sarah Ostadabbas. 2021. Deep Markov Factor Analysis: Towards Concurrent Temporal and Spatial Analysis of fMRI Data. In NeurIPS, Vol. 34. 1--13.
[2]
Soham Gadgil, Qingyu Zhao, Adolf Pfefferbaum, Edith V Sullivan, Ehsan Adeli, and Kilian M Pohl. 2020. Spatio-temporal graph convolution for resting-state fmri analysis. In MICCAI. 528--538.
[3]
Ioannis K Gallos, Evangelos Galaris, and Constantinos I Siettos. 2021. Construction of embedded fMRI resting-state functional connectivity networks using manifold learning. Cognitive neurodynamics, Vol. 15, 4 (2021), 585--608.
[4]
Rikkert Hindriks, Mohit H Adhikari, Yusuke Murayama, Marco Ganzetti, Dante Mantini, Nikos K Logothetis, and Gustavo Deco. 2016. Can sliding-window correlations reveal dynamic functional connectivity in resting-state fMRI? Neuroimage, Vol. 127 (2016), 242--256.
[5]
Zhijian Hou, Chong-Wah Ngo, and Wing-Kwong Chan. 2021. CONQUER: Contextual query-aware ranking for video corpus moment retrieval.(2021). In ACM MM. 20--24.
[6]
Rongyao Hu, Zhenyun Deng, and Xiaofeng Zhu. 2021a. Multi-scale graph fusion for co-saliency detection. In AAAI, Vol. 35. 7789--7796.
[7]
Rongyao Hu, Ziwen Peng, Xiaofeng Zhu, Jiangzhang Gan, Yonghua Zhu, Junbo Ma, and Guorong Wu. 2021b. Multi-band brain network analysis for functional neuroimaging biomarker identification. IEEE transactions on medical imaging, Vol. 40, 12 (2021), 3843--3855.
[8]
Hao Jiang, Peng Cao, MingYi Xu, Jinzhu Yang, and Osmar Zaiane. 2020. Hi-GCN: A hierarchical graph convolution network for graph embedding learning of brain network and brain disorders prediction. Computers in Biology and Medicine, Vol. 127 (2020), 104096.
[9]
Lize C Jiskoot, Jessica L Panman, Lieke H Meeter, Elise GP Dopper, Laura Donker Kaat, Sanne Franzen, Emma L van der Ende, Rick van Minkelen, Serge ARB Rombouts, Janne M Papma, et al. 2019. Longitudinal multimodal MRI as prognostic and diagnostic biomarker in presymptomatic familial frontotemporal dementia. Brain, Vol. 142, 1 (2019), 193--208.
[10]
Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR. 1--15.
[11]
Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. ICLR, 1--14.
[12]
Hongming Li and Yong Fan. 2019. Interpretable, highly accurate brain decoding of subtly distinct brain states from functional MRI using intrinsic functional networks and long short-term memory recurrent neural networks. NeuroImage, Vol. 202 (2019), 116059.
[13]
Jie Li, Yong Fan, Bo Hou, Xinying Huang, Dan Lei, Jie Wang, Chenhui Mao, Liling Dong, Caiyan Liu, Feng Feng, et al. 2021a. A longitudinal observation of brain structure between AD and FTLD. Clinical Neurology and Neurosurgery, Vol. 205 (2021), 106604.
[14]
Xiaoxiao Li, Yuan Zhou, Nicha Dvornek, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence H Staib, Pamela Ventola, and James S Duncan. 2021b. Braingnn: Interpretable brain graph neural network for fmri analysis. Medical Image Analysis, Vol. 74 (2021), 102233.
[15]
Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. 2016. Large-margin softmax loss for convolutional neural networks. In ICML, Vol. 2. 1--10.
[16]
Renaud Lopes, Clément Bournonville, Grégory Kuchcinski, Thibaut Dondaine, Anne-Marie Mendyk, Romain Viard, Jean-Pierre Pruvo, Hilde Hénon, Marios K Georgakis, Marco Duering, et al. 2021. Prediction of long-term cognitive function after minor stroke using functional connectivity. Neurology, Vol. 96, 8 (2021), e1167--e1179.
[17]
Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2016. Hierarchical question-image co-attention for visual question answering. In NIPS, Vol. 29. 1--9.
[18]
Yujie Mo, Liang Peng, Jie Xu, Xiaoshuang Shi, and Xiaofeng Zhu. 2022. Simple Unsupervised Graph Representation Learning. In AAAI. 7797--7805.
[19]
Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML. 1--8.
[20]
Liang Peng, Rongyao Hu, Fei Kong, Jiangzhang Gan, Yujie Mo, Xiaoshuang Shi, and Xiaofeng Zhu. 2022a. Reverse Graph Learning for Graph Neural Network. IEEE Transactions on Neural Networks and Learning Systems (2022), 1--12.
[21]
Liang Peng, Nan Wang, Nicha Dvornek, Xiaofeng Zhu, and Xiaoxiao Li. 2022b. FedNI: Federated Graph Learning with Network Inpainting for Population-Based Disease Prediction. IEEE Transactions on Medical Imaging (2022), 1--12.
[22]
Edmund T Rolls, Chu-Chung Huang, Ching-Po Lin, Jianfeng Feng, and Marc Joliot. 2020. Automated anatomical labelling atlas 3. Neuroimage, Vol. 206 (2020), 116189.
[23]
Jessica Root, Paola Merino, Austin Nuckols, Michelle Johnson, and Thomas Kukar. 2021. Lysosome dysfunction as a cause of neurodegenerative diseases: Lessons from frontotemporal dementia and amyotrophic lateral sclerosis. Neurobiology of disease, Vol. 154 (2021), 105360.
[24]
Caio Pinheiro Santana, Emerson Assis de Carvalho, Igor Duarte Rodrigues, Guilherme Sousa Bastos, Adler Diniz de Souza, and Lucelmo Lacerda de Brito. 2022. rs-fMRI and machine learning for ASD diagnosis: a systematic review and meta-analysis. Scientific reports, Vol. 12, 1 (2022), 1--20.
[25]
Patrick Schober, Christa Boer, and Lothar A Schwarte. 2018. Correlation coefficients: appropriate use and interpretation. Anesthesia & Analgesia, Vol. 126, 5 (2018), 1763--1768.
[26]
Xuegang Song, Feng Zhou, Alejandro F Frangi, Jiuwen Cao, Xiaohua Xiao, Yi Lei, Tianfu Wang, and Baiying Lei. 2021. Graph convolution network with similarity awareness and adaptive calibration for disease-induced deterioration prediction. Medical Image Analysis, Vol. 69 (2021), 101947.
[27]
Petar Velivc ković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. ICLR, 1--12.
[28]
Lebo Wang, Kaiming Li, and Xiaoping P Hu. 2021. Graph convolutional network for fMRI analysis based on connectivity neighborhood. Network Neuroscience, Vol. 5, 1 (2021), 83--95.
[29]
Nan Wang, Dongren Yao, Lizhuang Ma, and Mingxia Liu. 2022. Multi-site clustering and nested feature extraction for identifying autism spectrum disorder with resting-state fMRI. Medical Image Analysis, Vol. 75 (2022), 102279.
[30]
Reagan R Wetherill, Hengyi Rao, Nathan Hager, Jieqiong Wang, Teresa R Franklin, and Yong Fan. 2019. Classifying and characterizing nicotine use disorder with high accuracy using machine learning and resting-state fMRI. Addiction biology, Vol. 24, 4 (2019), 811--821.
[31]
Lennart Wittkuhn and Nicolas W Schuck. 2021. Dynamics of fMRI patterns reflect sub-second activation sequences and reveal replay in human visual cortex. Nature Communications, Vol. 12, 1 (2021), 1--22.
[32]
Xiaodan Xing, Qingfeng Li, Hao Wei, Minqing Zhang, Yiqiang Zhan, Xiang Sean Zhou, Zhong Xue, and Feng Shi. 2019. Dynamic spectral graph convolution networks with assistant task training for early mci diagnosis. In MICCAI. 639--646.
[33]
Jie Xu, Chao Li, Yazhou Ren, Liang Peng, Yujie Mo, Xiaoshuang Shi, and Xiaofeng Zhu. 2022a. Deep Incomplete Multi-View Clustering via Mining Cluster Complementarity. In AAAI. 8761--8769.
[34]
Jie Xu, Yazhou Ren, Huayi Tang, Xiaorong Pu, Xiaofeng Zhu, Ming Zeng, and Lifang He. 2021. Multi-VAE: Learning Disentangled View-Common and View-Peculiar Visual Representations for Multi-View Clustering. In ICCV. 9234--9243.
[35]
Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. 2022b. Multi-Level Feature Learning for Contrastive Multi-View Clustering. In CVPR. 16051--16060.
[36]
Dongren Yao, Jing Sui, Erkun Yang, Pew-Thian Yap, Dinggang Shen, and Mingxia Liu. 2020. Temporal-Adaptive Graph Convolutional Network for Automated Identification of Major Depressive Disorder Using Resting-State fMRI. In MLMI. 1--10.
[37]
Fan Yu, Haonan Wang, Tongwei Ren, Jinhui Tang, and Gangshan Wu. 2020. Visual relation of interest detection. In ACM MM. 1386--1394.
[38]
Jie Zhang, Wei Cheng, Zhaowen Liu, Kai Zhang, Xu Lei, Ye Yao, Benjamin Becker, Yicen Liu, Keith M Kendrick, Guangming Lu, et al. 2016. Neural, electrophysiological and anatomical basis of brain-network variability and its characteristic changes in mental disorders. Brain, Vol. 139, 8 (2016), 2307--2321.
[39]
Xin Zhang, Liangxiu Han, Wenyong Zhu, Liang Sun, and Daoqiang Zhang. 2021a. An Explainable 3D Residual Self-Attention Deep Neural Network For Joint Atrophy Localization and Alzheimer's Disease Diagnosis using Structural MRI. IEEE Journal of Biomedical and Health Informatics, Vol. 4, 8 (2021), 1--10.
[40]
Yu Zhang, Loïc Tetrel, Bertrand Thirion, and Pierre Bellec. 2021b. Functional annotation of human cognitive states using deep graph convolution. NeuroImage, Vol. 231 (2021), 117847.
[41]
Yu Zhang, Han Zhang, Xiaobo Chen, Mingxia Liu, Xiaofeng Zhu, Seong-Whan Lee, and Dinggang Shen. 2019. Strength and similarity guided group-level brain functional network construction for MCI diagnosis. Pattern recognition, Vol. 88 (2019), 421--430.
[42]
Kanhao Zhao, Boris Duka, Hua Xie, Desmond J Oathes, Vince Calhoun, and Yu Zhang. 2022. A dynamic graph convolutional neural network framework reveals new insights into connectome dysfunctions in ADHD. NeuroImage, Vol. 246 (2022), 118774.
[43]
Haiyu Zhou and Daoqiang Zhang. 2021. Graph-In-Graph Convolutional Networks For Brain Disease Diagnosis. In ICIP. 111--115.

Cited By

View all
  • (2025)Semi-supervised multi-view feature selection with adaptive similarity fusion and learningPattern Recognition10.1016/j.patcog.2024.111159159(111159)Online publication date: Mar-2025
  • (2024)Efficient multi-view unsupervised feature selection with adaptive structure learning and inferenceProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/602(5443-5452)Online publication date: 3-Aug-2024
  • (2024)Spatio-Temporal Graph Hubness Propagation Model for Dynamic Brain Network ClassificationIEEE Transactions on Medical Imaging10.1109/TMI.2024.336301443:6(2381-2394)Online publication date: Jun-2024
  • Show More Cited By

Index Terms

  1. Complementary Graph Representation Learning for Functional Neuroimaging Identification

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MM '22: Proceedings of the 30th ACM International Conference on Multimedia
      October 2022
      7537 pages
      ISBN:9781450392037
      DOI:10.1145/3503161
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 10 October 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. functional connectivity networks
      2. graph convolutional networks
      3. rs-fmri data analysis
      4. semi-supervised learning

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      MM '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)83
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 17 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Semi-supervised multi-view feature selection with adaptive similarity fusion and learningPattern Recognition10.1016/j.patcog.2024.111159159(111159)Online publication date: Mar-2025
      • (2024)Efficient multi-view unsupervised feature selection with adaptive structure learning and inferenceProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/602(5443-5452)Online publication date: 3-Aug-2024
      • (2024)Spatio-Temporal Graph Hubness Propagation Model for Dynamic Brain Network ClassificationIEEE Transactions on Medical Imaging10.1109/TMI.2024.336301443:6(2381-2394)Online publication date: Jun-2024
      • (2024)Leveraging Brain Modularity Prior for Interpretable Representation Learning of fMRIIEEE Transactions on Biomedical Engineering10.1109/TBME.2024.337041571:8(2391-2401)Online publication date: Aug-2024
      • (2024)CC-GNN: A Clustering Contrastive Learning Network for Graph Semi-Supervised LearningIEEE Access10.1109/ACCESS.2024.339835612(71956-71969)Online publication date: 2024

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media