Transformer3: A Pure Transformer Framework for fMRI-Based Representations of Human Brain Function | IEEE Journals & Magazine | IEEE Xplore

Transformer3: A Pure Transformer Framework for fMRI-Based Representations of Human Brain Function


We propose a pure transformer-based framework, Transformer3, to extract the sample-wise, spatial, and temporal interdependencies hidden in fMRI time-series for fMRI-based...

Abstract:

Effective representation learning is essential for neuroimage-based individualized predictions. Numerous studies have been performed on fMRI-based individualized predicti...Show More

Abstract:

Effective representation learning is essential for neuroimage-based individualized predictions. Numerous studies have been performed on fMRI-based individualized predictions, leveraging sample-wise, spatial, and temporal interdependencies hidden in fMRI data. However, these studies failed to fully utilize the effective information hidden in fMRI data, as only one or two types of the interdependencies were analyzed. To effectively extract representations of human brain function through fully leveraging the three types of the interdependencies, we establish a pure transformer-based framework, Transformer3, leveraging transformer's strong ability to capture interdependencies within the input data. Transformer3 consists mainly of three transformer modules, with the Batch Transformer module used for addressing sample-wise similarities and differences, the Region Transformer module used for handling complex spatial interdependencies among brain regions, and the Time Transformer module used for capturing temporal interdependencies across time points. Experiments on age, IQ, and sex predictions based on two public datasets demonstrate the effectiveness of the proposed Transformer3. As the only hypothesis is that sample-wise, spatial, and temporal interdependencies extensively exist within the input data, the proposed Transformer3 can be widely used for representation learning based on multivariate time-series. Furthermore, the pure transformer framework makes it quite convenient for understanding the driving factors underlying the predictive models based on Transformer3.
We propose a pure transformer-based framework, Transformer3, to extract the sample-wise, spatial, and temporal interdependencies hidden in fMRI time-series for fMRI-based...
Published in: IEEE Journal of Biomedical and Health Informatics ( Volume: 29, Issue: 1, January 2025)
Page(s): 468 - 481
Date of Publication: 04 October 2024

ISSN Information:

PubMed ID: 39365723

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.