Skip to main content
Springer Nature Link
Log in

Evaluating the retest reproducibility of intrinsic connectivity network using multivariate correlation coefficient

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Recently, the retest reproducibility of intrinsic connectivity networks (ICNs) has become an increasing concern in the fMRI research community. However, few indexes can be applied to directly quantify the similarity of three or more ICNs for evaluating the retest reproducibility of ICNs. To solve this problem, a multivariable correlation coefficient based on zero-mean normalization and intraclass correlation coefficient (Z-ICC) is proposed. After demonstrating the calculation method and performance analysis on theory, Z-ICC is adopted to evaluate the similarity of three ICNs from three ICN sets, which are inferred from the open retest resting-state fMRI dataset NYU_TRT with dual temporal and spatial sparse representation (DTSSR). The reproducible ICNs and quantization index of retest reproducibility are obtained by the calculated Z-ICC values and the accepted evaluation criterion. The experimental results and visual inspection show that Z-ICC can effectively identify the reproducible ICNs and quantify the retest reproducibility of ICNs. Eighteen (Z-ICC > 0.8) of the inferred twenty ICNs with DTSSR that are found to be reproducible are far more than the seven reproducible ICNs based on temporal concatenation group ICA (TC-GICA). Furthermore, the result of the one-tailed two-sample T test demonstrates that the Z-ICC values of the reproducible ICNs by DTSSR are significantly greater than those of TC-GICA, indicating that more reproducible group-level ICNs with higher retest reproducibility can be achieved with DTSSR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Biswal B, Zerrin Yetkin F, Haughton VM, Hyde JS (1995) Functional connectivity in the motor cortex of resting human brain using echo-planar mri. Magn Reson Med 34(4):537–541

    Google Scholar 

  2. Smyser CD, Inder TE, Shimony JS, Hill JE, Degnan AJ, Snyder AZ, Neil JJ (2010) Longitudinal analysis of neural network development in preterm infants. Cereb Cortex 20(12):2852–2862

    Google Scholar 

  3. Beckmann CF, DeLuca M, Devlin JT, Smith SM (2005) Investigations into resting-state connectivity using independent component analysis. Proc R Soc B Biol Sci 360(1457):1001–1013

    Google Scholar 

  4. Guo CC, Kurth F, Zhou J, Mayer EA, Eickhoff SB, Kramer JH, Seeley WW (2012) One-year test–retest reliability of intrinsic connectivity network fMRI in older adults. Neuroimage 61(4):1471–1483

    Google Scholar 

  5. Biswal B, Mennes M, Zuo XN, Gohel S, Kelly C, Smith SM, Beckmann CF, Adelstein JS, Buckner RL, Colcombe S (2009) Toward discovery science of human brain function. Proc Natl Acad Sci USA 71(10):4734–4739

    Google Scholar 

  6. Smith SM, Fox PT, Miller KL, Glahn DC, Fox PM, Mackay CE, Filippini N, Watkins KE, Toro R, Laird AR (2009) Correspondence of the brain’s functional architecture during activation and rest. Proc Natl Acad Sci USA 106(31):13040–13045

    Google Scholar 

  7. Laird AR, Fox PM, Eickhoff SB, Turner JA, Ray KL, Mckay DR, Glahn DC, Beckmann CF, Smith SM, Fox PT (2011) Behavioral interpretations of intrinsic connectivity networks. J Cogn Neurosci 23(12):4022–4037

    Google Scholar 

  8. Kozák LR, Van LG, Chaudhary UJ, Szabó Á, Lemieux L (2017) ICN_Atlas: automated description and quantification of functional MRI activation patterns in the framework of intrinsic connectivity networks. Neuroimage 163(12):319–341

    Google Scholar 

  9. Adelstein JS, Zarrar S, Maarten M, Deyoung CG, Zuo XN, Clare K, Margulies DS, Aaron B, Gray JR, Xavier CF (2011) Personality is reflected in the brain’s intrinsic functional architecture. PLoS ONE 6(11):e27633

    Google Scholar 

  10. Kunisato Y, Okamoto Y, Okada G, Aoyama S, Nishiyama Y, Onoda K, Yamawaki S (2011) Personality traits and the amplitude of spontaneous low-frequency oscillations during resting state. Neurosci Lett 492(2):109–113

    Google Scholar 

  11. Zhu X, Wang X, Xiao J, Liao J, Zhong M, Wang W, Yao S (2012) Evidence of a dissociation pattern in resting-state default mode network connectivity in first-episode, treatment-naive major depression patients. Biol Psychiatry 71(7):611–617

    Google Scholar 

  12. Smitha KA, Arun KM, Rajesh PG, Thomas B, Kesavadas C (2017) Resting-state seed-based analysis: an alternative to task-based language fMRI and its laterality index. Am J Neuroradiol 38(6):1187–1192

    Google Scholar 

  13. Zhao T, Xu Y, He Y (2018) Graph theoretical modeling of baby brain networks. Neuroimage 185(15):711–727

    Google Scholar 

  14. Bullmore E, Sporns O (2009) Complex brain networks: graph theoretical analysis of structural and functional systems. Nat Rev Neurosci 10(3):186–198

    Google Scholar 

  15. Bi X, Zhao J, Xu Q, Sun Q, Wang Z (2018) Abnormal functional connectivity of resting state network detection based on linear ICA analysis in autism spectrum disorder. Front Physiol 9:475

    Google Scholar 

  16. Lv J, Jiang X, Li X, Zhu D, Chen H, Zhang T, Zhang S, Hu X, Han J, Huang H, Zhang J, Guo L, Liu T (2015) Sparse representation of whole-brain fMRI signals for identification of functional networks. Med Image Anal 20(1):112–134

    Google Scholar 

  17. Zhao S, Han J, Lv J, Jiang X, Hu X, Zhao Y, Ge B, Guo L, Liu T (2015) Supervised dictionary learning for inferring concurrent brain networks. IEEE Trans Med Imaging 34(10):2036–2045

    Google Scholar 

  18. Özmen G, Özşen S (2017) A new denoising method for fMRI based on weighted three-dimensional wavelet transform. Neural Comput and Appl 29(1):1–14

    Google Scholar 

  19. Yan X-D, Kelley S, Goldberg M (2012) Application of connected iterative scan in biological neural network. Neural Comput Appl 21(8):2097–2103

    Google Scholar 

  20. Wisner KM, Atluri G, Lim KO, Rd MA (2013) Neurometrics of intrinsic connectivity networks at rest using fMRI: retest reliability and cross-validation using a meta-level method. Neuroimage 76(8):236–251

    Google Scholar 

  21. Vetter NC, Steding J, Jurk S, Ripke S, Mennigen E, Smolka MN (2017) Reliability in adolescent fMRI within two years—a comparison of three tasks. Sci Rep 7(1):2287

    Google Scholar 

  22. Zuo XN, Kelly C, Adelstein JS, Klein DF, Castellanos FX, Milham MP (2010) Reliable intrinsic connectivity networks: test–retest evaluation using ICA and dual regression approach. Neuroimage 49(3):2163–2177

    Google Scholar 

  23. Wang JH, Zuo XN, Gohel S, Milham MP, Biswal BB, He Y (2011) Graph theoretical analysis of functional brain networks: test–retest evaluation on short- and long-term resting-state functional MRI data. PLoS ONE 6:e21976

    Google Scholar 

  24. Meindl T, Teipel S, Elmouden R, Mueller S, Koch W, Dietrich O, Coates U, Reiser M, Glaser C (2010) Test–retest reproducibility of the default-mode network in healthy individuals. Hum Brain Mapp 31(2):237–246

    Google Scholar 

  25. Fiecas M, Ombao H, Van LD, Baumgartner R, Coimbra A, Feng D (2013) Quantifying temporal correlations: a test–retest evaluation of functional connectivity in resting-state fMRI. Neuroimage 65(2):231–241

    Google Scholar 

  26. Wang N, Zeng W, Chen D, Yin J, Chen L (2016) A novel brain networks enhancement model (BNEM) for BOLD fMRI data analysis with highly spatial reproducibility. IEEE J Biomed Health 20(4):1107–1119

    Google Scholar 

  27. Wang J, Han J, Nguyen VT, Guo L, Guo CC (2017) Improving the test–retest reliability of resting state fMRI by removing the impact of sleep. Front Neurosci 11:249

    Google Scholar 

  28. Golestani AM, Kwinta JB, Khatamian YB, Chen JJ (2017) The effect of low-frequency physiological correction on the reproducibility and specificity of resting-state fMRI metrics: functional connectivity, ALFF, and ReHo. Front Neurosci 11:546

    Google Scholar 

  29. Chen X, Lu B, Yan CG (2017) Reproducibility of R-fMRI metrics on the impact of different strategies for multiple comparison correction and sample sizes. Hum Brain Mapp 39(1):300–318

    Google Scholar 

  30. Termenon M, Jaillard A, Delon-Martin C, Achard S (2016) Reliability of graph analysis of resting state fMRI using test–retest dataset from the Human Connectome Project. Neuroimage 142(11):172–187

    Google Scholar 

  31. Lin IK (1992) Assay validation using the concordance correlation coefficient. Biometrics 48(2):599–604

    MathSciNet  Google Scholar 

  32. Gong J, Liu X, Liu T, Zhou J, Gang S, Tian J (2018) Dual temporal and spatial sparse representation for inferring group-wise brain networks from resting-state fMRI dataset. IEEE Trans Biomed Eng 65(5):1035–1048

    Google Scholar 

  33. Mairal J, Bach F, Ponce J, Sapiro G (2010) Online learning for matrix factorization and sparse coding. J Mach Learn Res 11:19–60

    MathSciNet  MATH  Google Scholar 

  34. Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420

    Google Scholar 

  35. Zuo XN, Martino AD, Kelly C, Shehzad ZE, Gee DG, Klein DF, Castellanos FX, Biswal BB, Milham MP (2010) The oscillating brain: complex and reliable. Neuroimage 49(2):1432–1445

    Google Scholar 

  36. Shehzad Z, Kelly AM, Reiss PT, Gee DG, Gotimer K, Uddin LQ, Lee SH, Margulies DS, Roy AK, Biswal BB (2009) The resting brain: unconstrained yet reliable. Cereb Cortex 19(10):2209–2229

    Google Scholar 

  37. Shutao L, Haitao Y, Leyuan F (2012) Group-sparse representation with dictionary learning for medical image denoising and fusion. Trans Biomed Eng 59(12):3450–3459

    Google Scholar 

  38. Yang X, Sun Q, Wang T (2018) No-reference image quality assessment based on sparse representation. Neural Comput Appl 9:1–16

    Google Scholar 

  39. Zhang H, Wang S, Xu X, Chow TWS, Wu QMJ (2018) Tree2Vector: learning a vectorial representation for tree-structured data. IEEE Trans Neural Netw Learn 29(11):5304–5318

    MathSciNet  Google Scholar 

  40. Yang J, Liu Y (2017) Undersampled face recognition based on virtual samples and representation classification. Neural Comput Appl 1:1–7

    Google Scholar 

  41. Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32:407–499

    MathSciNet  MATH  Google Scholar 

  42. Zhang S, Li X, Lv J, Jiang X, Zhu D, Chen H, Zhang T, Guo L, Liu T (2013) Sparse representation of higher-order functional interaction patterns in task-based fMRI data. In: Medical image computing and computer-assisted intervention–MICCAI 2013. Springer, pp 626–634

  43. Zuo XN, Xing XX (2014) Test–retest reliabilities of resting-state FMRI measurements in human brain functional connectomics: a systems neuroscience perspective. Neurosci Biobehav Rev 45:100–118

    Google Scholar 

  44. Han Z, Chen X, Yu Z, Shen D (2017) Test–retest reliability of “high-order” functional connectivity in young healthy adults. Front Neurosci 11:439

    Google Scholar 

  45. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    MATH  Google Scholar 

  46. Doria V, Beckmann CF, Arichi T, Merchant N, Groppo M, Turkheimer FE, Counsell SJ, Murgasova M, Aljabar P, Nunes RG (2010) Emergence of resting state networks in the preterm human brain. Proc Natl Acad Sci USA 107:20015–20020

    Google Scholar 

  47. Zhang W, Lv J, Li X, Zhu D, Jiang X, Zhang S, Zhao Y, Guo L, Ye J, Hu D (2018) Experimental comparisons of sparse dictionary learning and independent component analysis for brain network inference from fMRI data. IEEE Trans Biomed Eng 65:289–299

    Google Scholar 

Download references

Acknowledgements

Financial support from National Natural Science Foundation of China (No. 61973108), Youth Foundation of Hunan Institute of Engineering (No. XJ1702) and National Natural Science Foundation of China (No. 81571341) is greatly appreciated.

Author information

Authors and Affiliations

  1. College of Electrical and Information Engineering, Hunan University, Changsha 410082, China

    Junhui Gong, Xiaoyan Liu & Gang Sun

  2. College of Electrical and Information Engineering, Hunan Institute of Engineering, Xiangtan 411104, China

    Junhui Gong

  3. Mental Health Institute of the Second Xiangya Hospital, Central South University, Changsha 410011, China

    Jiansong Zhou

Authors
  1. Junhui Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiansong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoyan Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gong, J., Liu, X., Sun, G. et al. Evaluating the retest reproducibility of intrinsic connectivity network using multivariate correlation coefficient. Neural Comput & Applic 32, 14623–14638 (2020). https://doi.org/10.1007/s00521-020-04816-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-04816-8

Keywords