Skip to main content

Advertisement

SpringerLink
Log in

A novel ADHD classification method based on resting state temporal templates (RSTT) using spatiotemporal attention auto-encoder

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

It has been of great interest in the neuroimaging community to model spatiotemporal brain function and related disorders based on resting state functional magnetic resonance imaging (rfMRI). Although a variety of deep learning models have been proposed for modeling rfMRI, the dominant models are limited in capturing the long-distance dependency (LDD) due to their sequential nature. In this work, we propose a spatiotemporal attention auto-encoder (STAAE) to discover global features that address LDDs in volumetric rfMRI. The unsupervised STAAE framework can spatiotemporally model the rfMRI sequence and decompose the rfMRI into spatial and temporal patterns. The spatial patterns have been extensively explored and are also known as resting state networks (RSNs), yet the temporal patterns are underestimated in last decades. To further explore the application of temporal patterns, we developed a resting state temporal template (RSTT)-based classification framework using the STAAE model and tested it with attention-deficit hyperactivity disorder (ADHD) classification. Five datasets from ADHD-200 were used to evaluate the performance of our method. The results showed that the proposed STAAE outperformed three recent methods in deriving ten well-known RSNs. For ADHD classification, the proposed RSTT-based classification framework outperformed methods in recent studies by achieving a high accuracy of 72.5%. Besides, we found that the RSTTs derived from NYU dataset still work on the other four datasets, but the accuracy on different test datasets decreased with the increase in the age gap to NYU dataset, which likely supports the idea of that there exist age differences of brain activity among ADHD patients.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Logothetis NK (2008) What we can do and what we cannot do with fMRI. Nature 453(7197):869

    Article  Google Scholar 

  2. Huettel SA, Song AW, McCarthy G (2004) Functional magnetic resonance imaging. Sinauer Associates Sunderland, MA

  3. Greicius MD, Krasnow B, Reiss AL, Menon V (2003) Functional connectivity in the resting brain: a network analysis of the default mode hypothesis. Proc Natl Acad Sci 100(1):253–258

    Article  Google Scholar 

  4. Pessoa L (2014) Understanding brain networks and brain organization. Phys Life Rev 11(3):400–435

    Article  Google Scholar 

  5. Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME (2005) The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci 102(27):9673–9678

    Article  Google Scholar 

  6. Pessoa L (2012) Beyond brain regions: Network perspective of cognition–emotion interactions. Behavioral and Brain Sciences 35(3):158–159

    Article  Google Scholar 

  7. Beckmann CF, Jenkinson M, Smith SM (2003) General multilevel linear modeling for group analysis in FMRI. Neuroimage 20(2):1052–1063

    Article  Google Scholar 

  8. Barch DM et al (2013) Function in the human connectome: task-fMRI and individual differences in behavior. Neuroimage 80:169–189

    Article  Google Scholar 

  9. Calhoun VD, Liu J, Adalı T (2009) A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. Neuroimage 45(1):S163–S172

    Article  Google Scholar 

  10. Calhoun VD, Adali T (2012) Multisubject independent component analysis of fMRI: a decade of intrinsic networks, default mode, and neurodiagnostic discovery. IEEE Rev Biomed Eng 5:60–73

    Article  Google Scholar 

  11. Calhoun VD, Adali T, Pearlson GD, Pekar J (2001) A method for making group inferences from functional MRI data using independent component analysis. Hum Brain Mapp 14(3):140–151

    Article  Google Scholar 

  12. Beckmann CF, DeLuca M, Devlin JT, Smith SM (2005) Investigations into resting-state connectivity using independent component analysis. Philos Trans R Soc Lond B Biol Sci 360(1457):1001–1013

    Article  Google Scholar 

  13. McKeown MJ (2000) Detection of consistently task-related activations in fMRI data with hybrid independent component analysis. Neuroimage 11(1):24–35

    Article  Google Scholar 

  14. Lv J et al (2015) Sparse representation of whole-brain fMRI signals for identification of functional networks. Med Image Anal 20(1):112–134

    Article  Google Scholar 

  15. Li X, Dong Q, Jiang X, Lv J, Liu T (2016) Multple-demand system identification and characterization via sparse representations of fMRI data. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE, pp 70–73

  16. Ge F, et al (2015) Deriving ADHD biomarkers with sparse coding based network analysis. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI). IEEE, pp 22–25

  17. Lee K, Tak S, Ye JC (2011) A data-driven sparse GLM for fMRI analysis using sparse dictionary learning with MDL criterion. IEEE Trans Med Imaging 30(5):1076–1089

    Article  Google Scholar 

  18. Li Q, et al (2019) Simultaneous spatial-temporal decomposition of connectome-scale brain networks by deep sparse recurrent auto-encoders. In: International conference information processing, pp. 579–591

  19. Cui Y, et al (2018) Identifying brain networks of multiple time scales via deep recurrent neural network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp 284–292

  20. Hjelm RD, Calhoun VD, Salakhutdinov R, Allen EA, Adali T, Plis SM (2014) Restricted Boltzmann machines for neuroimaging: an application in identifying intrinsic networks. Neuroimage 96:245–260

    Article  Google Scholar 

  21. Hu X et al (2018) Latent source mining in FMRI via restricted Boltzmann machine. Hum Brain Mapp 39(6):2368–2380

    Article  Google Scholar 

  22. Huang H et al (2018) Modeling task fMRI data via deep convolutional autoencoder. IEEE Trans Med Imaging 37(7):1551–1561

    Article  Google Scholar 

  23. Li Y, Huang H, Chen H, Liu T (2018) Deep neural networks for exploration of transcriptome of adult mouse brain. IEEE/ACM Trans Comput Biol Bioinf

  24. Plis SM et al (2014) Deep learning for neuroimaging: a validation study. Front Neurosci 8:229

    Article  Google Scholar 

  25. Suk H-I, Wee C-Y, Lee S-W, Shen D (2016) State-space model with deep learning for functional dynamics estimation in resting-state fMRI. Neuroimage 129:292–307

    Article  Google Scholar 

  26. Wang H et al (2018) Recognizing brain states using deep sparse recurrent neural network. IEEE Trans Med Imaging

  27. Zhao Y, et al (2018) Modeling 4D fMRI data via spatio-temporal convolutional neural networks (ST-CNN). In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp 181–189

  28. Qiang N, et al (2020) Modeling task-based fMRI data via deep belief network with neural architecture search. Comput Med Imaging Graph, p 101747

  29. Dong Q, et al (2019) Modeling hierarchical brain networks via volumetric sparse deep belief network (VS-DBN). IEEE Trans Biomed Eng

  30. Qiang N, et al (2020) Deep variational autoencoder for mapping functional brain networks. IEEE Trans Cognit Develop Syst

  31. Piñango MM, Finn E, Lacadie E, Constable RTJFIP (2016) The localization of long-distance dependency components: Integrating the focal-lesion and neuroimaging record. Front Psychol 7:1434

    Article  Google Scholar 

  32. Huang H, et al (2018) Modeling task fMRI data via mixture of deep expert networks. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), IEEE, pp 82–86

  33. Wang L, et al (2017) Decoding dynamic auditory attention during naturalistic experience. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), IEEE, pp 974–977

  34. Li Q, et al (2019) Simultaneous spatial-temporal decomposition of connectome-scale brain networks by deep sparse recurrent auto-encoders. In: International conference on information processing in medical imaging, Springer, pp 579–591

  35. Bahdanau D, Cho K, Bengio YJAPA (2014) Neural machine translation by jointly learning to align and translate

  36. Vaswani A, et al (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  37. Chen Y, Peng G, Zhu Z, Li S (2020) A novel deep learning method based on attention mechanism for bearing remaining useful life prediction. Appl Soft Comput 86:105919

    Article  Google Scholar 

  38. Li X, Zhang W, Ding Q (2019) Understanding and improving deep learning-based rolling bearing fault diagnosis with attention mechanism. Signal Process 161:136–154

    Article  Google Scholar 

  39. Yuan X, Li L, Shardt YA, Wang Y, Yang C (2020) Deep learning with spatiotemporal attention-based LSTM for industrial soft sensor model development. IEEE Trans Industr Electron 68(5):4404–4414

    Article  Google Scholar 

  40. Dai D, Wang J, Hua J, He H (2012) Classification of ADHD children through multimodal magnetic resonance imaging. Front Syst Neurosci 6:63

    Article  Google Scholar 

  41. Zou L, Zheng J, Miao C, Mckeown MJ, Wang ZJ (2017) 3D CNN based automatic diagnosis of attention deficit hyperactivity disorder using functional and structural MRI. IEEE Access 5:23626–23636

    Article  Google Scholar 

  42. Mao Z et al (2019) Spatio-temporal deep learning method for adhd fmri classification. Inf Sci 499:1–11

    Article  Google Scholar 

  43. Dey S, Rao AR, Shah M (2014) Attributed graph distance measure for automatic detection of attention deficit hyperactive disordered subjects. Front Neural Circuits 8:64

    Article  Google Scholar 

  44. Riaz A, Asad M, Alonso E, Slabaugh G (2018) Fusion of fMRI and non-imaging data for ADHD classification. Comput Med Imaging Graph 65:115–128

    Article  Google Scholar 

  45. Itani S, Lecron F, Fortemps P (2018) A multi-level classification framework for multi-site medical data: Application to the ADHD-200 collection. Expert Syst Appl 91:36–45

    Article  Google Scholar 

  46. Bellec P, Chu C, Chouinard-Decorte F, Benhajali Y, Margulies DS, Craddock RC (2017) The neuro bureau ADHD-200 preprocessed repository. Neuroimage 144:275–286

    Article  Google Scholar 

  47. dos Santos Siqueira A, Junior B, Eduardo C, Comfort WE, Rohde LA, Sato JR (2014) Abnormal functional resting-state networks in ADHD: graph theory and pattern recognition analysis of fMRI data. BioMed Res Int, 2014

  48. Smith SM et al (2009) Correspondence of the brain’s functional architecture during activation and rest. Proc Natl Acad Sci 106(31):13040–13045

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the national key R&D program of China (2017YFE0130600), and the Fundamental Research Founds for the Central Universities Grant No. GK202003016 and GK202103014, and the National Natural Science Foundation of China under Grant 61976131, 61703256, 61872007 and 62073133, and Beijing Natural Science Foundation (Z201100008320002).

Author information

Authors and Affiliations

  1. School of Physics and Information Technology, Shaanxi Normal University, No. 620, West Chang’an Avenue, Chang’an District, Xi’an, China

    Ning Qiang, Hongtao Liang, Bao Ge, Jie Gao & Yifei Sun

  2. Center for Brain and Brain-Inspired Computing Research, Department of Computer Science, Northwestern Polytechnical University, Xi’an, China

    Ning Qiang, Bao Ge & Shu Zhang

  3. Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA

    Qinglin Dong

  4. School of Electronics Engineering and Computer Science, Peking University, Beijing, China

    Cheng Zhang

Authors
  1. Ning Qiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qinglin Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongtao Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bao Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jie Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yifei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jie Gao or Yifei Sun.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Ning Qiang, Qinglin Dong, Hongtao Liang these authors are co-first authors

Appendix

Appendix

1.1 Explanations of used abbreviations

To make readers better understand the abbreviations in the paper, explanations of used abbreviations are given in the Table 8.

Table 8 Explanations of used abbreviations
Full size table

1.2 RSNs derived from other four datasets

The RSNs derived by STAAE on PU, KKI, NI, and OHSU are shown in Figs. 12, 13, 14 and 15. For each dataset, 10 RSNs were derived from whole dataset, patient only dataset, and normal control only dataset. As shown in Table 9, we calculated the average ORs between RSNs derived on different datasets and RSN templates from ICA. The results showed that the data size affects the performance of STAAE in deriving RSNs.

Fig. 12
figure 12

RSNs derived by STAAE on PU dataset, including whole dataset, ADHD dataset, and NC (normal control) dataset. Each network is visualized with 10 axial slices

Full size image
Fig. 13
figure 13

RSNs derived by STAAE on KKI dataset, including whole dataset, ADHD dataset, and NC (normal control) dataset. Each network is visualized with 10 axial slices

Full size image
Fig. 14
figure 14

RSNs derived by STAAE on NI dataset, including whole dataset, ADHD dataset, and NC (normal control) dataset. Each network is visualized with 10 axial slices

Full size image
Fig. 15
figure 15

RSNs derived by STAAE on OHSU dataset, including whole dataset, ADHD dataset, and NC (normal control) dataset. Each network is visualized with 10 axial slices

Full size image
Table 9 Average ORs of RSNs to the RSN templates
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiang, N., Dong, Q., Liang, H. et al. A novel ADHD classification method based on resting state temporal templates (RSTT) using spatiotemporal attention auto-encoder. Neural Comput & Applic 34, 7815–7833 (2022). https://doi.org/10.1007/s00521-021-06868-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06868-w

Keywords

Search

Navigation