Skip to main content
SpringerLink
Log in

Eyes-Open/Eyes-Closed Dataset Sharing for Reproducibility Evaluation of Resting State fMRI Data Analysis Methods

  • Data Original Article
  • Published:
Neuroinformatics Aims and scope Submit manuscript

Abstract

The multi-scan resting state fMRI (rs-fMRI) dataset was recently released; thus the test-retest (TRT) reliability of rs-fMRI measures can be assessed. However, because this dataset was acquired only from a single group under a single condition, we cannot directly evaluate whether the rs-fMRI measures can generate reproducible between-condition or between-group results. Because the modulation of resting state activity has gained increasing attention, it is important to know whether one rs-fMRI metric can reliably detect the alteration of the resting activity. Here, we shared a public Eyes-Open (EO)/Eyes-Closed (EC) dataset for evaluating the split-half reproducibility of the rs-fMRI measures in detecting changes of the resting state activity between EO and EC. As examples, we assessed the split-half reproducibility of three widely applied rs-fMRI metrics: amplitude of low frequency fluctuation, regional homogeneity, and seed-based correlation analysis. Our results demonstrated that reproducible patterns of EO-EC differences can be detected by all three measures, suggesting the feasibility of the EO/EC dataset for performing reproducibility assessment for other rs-fMRI measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Albert, N. B., Robertson, E. M., & Miall, R. C. (2009). The resting human brain and motor learning. Current Biology, 19(12), 1023–1027.

    Article  PubMed  CAS  Google Scholar 

  • Barnes, A., Bullmore, E. T., & Suckling, J. (2009). Endogenous human brain dynamics recover slowly following cognitive effort. PLoS One, 4(8), e6626.

    Article  PubMed  Google Scholar 

  • Beckmann, C. F., DeLuca, M., Devlin, J. T., & Smith, S. M. (2005). Investigations into resting-state connectivity using independent component analysis. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 360(1457), 1001–1013.

    Article  PubMed  Google Scholar 

  • Biswal, B., Yetkin, F. Z., Haughton, V. M., & Hyde, J. S. (1995). Functional connectivity in the motor cortex of resting human brain using echo-planar MRI. Magnetic Resonance in Medicine, 34(4), 537–541.

    Article  PubMed  CAS  Google Scholar 

  • Castellanos, F. X., Margulies, D. S., Kelly, C., Uddin, L. Q., Ghaffari, M., et al. (2008). Cingulate-precuneus interactions: a new locus of dysfunction in adult attention-deficit/hyperactivity disorder. Biological Psychiatry, 63(3), 332–337.

    Article  PubMed  Google Scholar 

  • Cole, D. M., Smith, S. M., & Beckmann, C. F. (2010). Advances and pitfalls in the analysis and interpretation of resting-state fMRI data. Frontiers in Systems Neuroscience, 4, 8.

  • Cordes, D., Haughton, V. M., Arfanakis, K., Wendt, G. J., Turski, P. A., Moritz, C. H., Quigley, M. A., & Meyerand, M. E. (2000). Mapping functionally related regions of brain with functional connectivity MR imaging. American Journal of Neuroradiology, 21(9), 1636–1644.

    PubMed  CAS  Google Scholar 

  • Cox, R. W. (1996). AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research, 29(3), 162–173.

    Article  PubMed  CAS  Google Scholar 

  • Di Martino, A., Scheres, A., Margulies, D. S., Kelly, A. M., Uddin, L. Q., Shehzad, Z., Biswal, B., Walters, J. R., Castellanos, F. X., & Milham, M. P. (2008). Functional connectivity of human striatum: a resting state FMRI study. Cerebral Cortex, 18(12), 2735–2747.

    Article  PubMed  Google Scholar 

  • Fox, M. D., Snyder, A. Z., Vincent, J. L., Corbetta, M., Van Essen, D. C., & Raichle M. E. (2005). The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proceedings of the National Academy of Sciences in United States of America, 105(27), 9673–9678.

    Google Scholar 

  • Fox, M. D., Corbetta, M., Snyder, A. Z., Vincent, J. L., & Raichle, M. E. (2006). Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. Proceedings of the National Academy of Sciences in United States of America, 103(26), 10046–10051.

    Article  CAS  Google Scholar 

  • Friston, K. J., Holmes, A. P., Price, C. J., Buchel, C., & Worsley, K. J. (1999). Multisubject fMRI studies and conjunction analyses. NeuroImage, 10(4), 385–396.

    Article  PubMed  CAS  Google Scholar 

  • Greicius, M. D., Krasnow, B., Reiss, A. L., & Menon, V. (2003). Functional connectivity in the resting brain: a network analysis of the default mode hypothesis. Proceedings of the National Academy of Sciences in United States of America, 100(1), 253–258.

    Article  CAS  Google Scholar 

  • Grigg, O., & Grady, C. L. (2010). Task-related effects on the temporal and spatial dynamics of resting-state functional connectivity in the default network. PLoS One, 5(10), e13311.

    Article  PubMed  Google Scholar 

  • Hoptman, M. J., Zuo, X. N., Butler, P. D., Javitt, D. C., D’Angelo, D., Mauro, C. J., & Milham, M. P. (2010). Amplitude of low-frequency oscillations in schizophrenia: a resting state fMRI study. Schizophrenia Research, 117(1), 13–20.

    Article  PubMed  Google Scholar 

  • Kendall, M., & Gibbons, J. D. (1990). Rank correlation methods. Oxford: Oxford University Press.

    Google Scholar 

  • Lai, M. C., Lombardo, M. V., Chakrabarti, B., Sadek, S. A., Pasco, G., Wheelwright, S. J., Bullmore, E. T., Baron-Cohen, S., & Suckling, J. (2010). A shift to randomness of brain oscillations in people with autism. Biological Psychiatry, 68(12), 1092–1099.

    Article  PubMed  Google Scholar 

  • Ledberg, A., Akerman, S., & Roland, P. E. (1998). Estimation of the probabilities of 3D clusters in functional brain images. NeuroImage, 8(2), 113–128.

    Article  PubMed  CAS  Google Scholar 

  • Lowe, M. J., Mock, B. J., & Sorenson, J. A. (1998). Functional connectivity in single and multislice echoplanar imaging using resting-state fluctuations. NeuroImage, 7(2), 119–132.

    Article  PubMed  CAS  Google Scholar 

  • Machielsen, W. C., Rombouts, S. A., Barkhof, F., Scheltens, P., & Witter, M. P. (2000). FMRI of visual encoding: reproducibility of activation. Human Brain Mapping, 9(3), 156–164.

    Article  PubMed  CAS  Google Scholar 

  • Margulies, D. S., Bottger, J., Long, X., Lv, Y., Kelly, C., Schafer, A., Goldhahn, D., Abbushi, A., Milham, M. P., Lohmann, G., & Villringer, A. (2010). Resting developments: a review of fMRI post-processing methodologies for spontaneous brain activity. Magma, 23(5–6), 289–307.

    Article  PubMed  Google Scholar 

  • Nichols, T., Brett, M., Andersson, J., Wager, T., & Poline, J. B. (2005). Valid conjunction inference with the minimum statistic. NeuroImage, 25(3), 653–660.

    Article  PubMed  Google Scholar 

  • Paakki, J. J., Rahko, J., Long, X., Moilanen, I., Tervonen, O., Nikkinen, J., Starck, T., Remes, J., Hurtig, T., Haapsamo, H., Jussila, K., Kuusikko-Gauffin, S., Mattila, M. L., Zang, Y., & Kiviniemi, V. (2010). Alterations in regional homogeneity of resting-state brain activity in autism spectrum disorders. Brain Research, 1321, 169–179.

    Article  PubMed  CAS  Google Scholar 

  • Press, W. H. T. S., Vetterling, W. T., & Flannery, B. P. (1992). Numerical Recipes in C (2nd ed.). Cambridge: Cambridge University Press.

    Google Scholar 

  • Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proceedings of the National Academy of Sciences in United States of America, 98(2), 676–682.

    Article  CAS  Google Scholar 

  • Salvador, R., Martinez, A., Pomarol-Clotet, E., Sarro, S., Suckling, J., & Bullmore, E. (2007). Frequency based mutual information measures between clusters of brain regions in functional magnetic resonance imaging. NeuroImage, 35(1), 83–88.

    Article  PubMed  CAS  Google Scholar 

  • Shehzad, Z., Kelly, A. M., Reiss, P. T., Gee, D. G., Gotimer, K., Uddin, L. Q., Lee, S. H., Margulies, D. S., Roy, A. K., Biswal, B. B., Petkova, E., Castellanos, F. X., & Milham, M. P. (2009). The resting brain: unconstrained yet reliable. Cerebral Cortex, 19(10), 2209–2229.

    Article  PubMed  Google Scholar 

  • Sheline, Y. I., Raichle, M. E., Snyder, A. Z., Morris, J. C., Head, D., et al. (2010). Amyloid plaques disrupt resting state default mode network connectivity in cognitively normal elderly. Biological Psychiatry, 67(6), 584–587.

    Article  PubMed  CAS  Google Scholar 

  • Song, X. W., Dong, Z. Y., Long, X. Y., Li, S. F., Zuo, X. N., Zhu, C. Z., He, Y., Yan, C. G., & Zang, Y. F. (2011). REST: a toolkit for resting-state functional magnetic resonance imaging data processing. PLoS One, 6(9), e25031.

    Article  PubMed  CAS  Google Scholar 

  • Stiers, P., Mennes, M., & Sunaert, S. (2010). Distributed task coding throughout the multiple demand network of the human frontal-insular cortex. NeuroImage, 52(1), 252–262.

    Article  PubMed  Google Scholar 

  • Strother, S. C., Anderson, J., Hansen, L. K., Kjems, U., Kustra, R., et al. (2002). The quantitative evaluation of functional neuroimaging experiments: the NPAIRS data analysis framework. NeuroImage, 15(4), 747–771.

    Article  PubMed  Google Scholar 

  • Tambini, A., Ketz, N., & Davachi, L. (2010). Enhanced brain correlations during rest are related to memory for recent experiences. Neuron, 65(2), 280–290.

    Article  PubMed  CAS  Google Scholar 

  • Van Dijk, K. R., Hedden, T., Venkataraman, A., Evans, K. C., Lazar, S. W., & Buckner, R. L. (2010). Intrinsic functional connectivity as a tool for human connectomics: theory, properties, and optimization. Journal of Neurophysiology, 103(1), 297–321.

    Article  PubMed  Google Scholar 

  • Waites, A. B., Stanislavsky, A., Abbott, D. F., & Jackson, G. D. (2005). Effect of prior cognitive state on resting state networks measured with functional connectivity. Human Brain Mapping, 24(1), 59–68.

    Article  PubMed  Google Scholar 

  • Wang, K., Yu, C., Xu, L., Qin, W., Li, K., Xu, L., & Jiang, T. (2009). Offline memory reprocessing: involvement of the brain’s default network in spontaneous thought processes. PLoS One, 4(3), e4867.

    Article  PubMed  Google Scholar 

  • Wang, L., Yu, C., Chen, H., Qin, W., He, Y., Fan, F., Zhang, Y., Wang, M., Li, K., Zang, Y., Woodward, T. S., & Zhu, C. (2010). Dynamic functional reorganization of the motor execution network after stroke. Brain, 133(Pt 4), 1224–1238.

    Article  PubMed  Google Scholar 

  • Wu, T., Long, X., Zang, Y., Wang, L., Hallett, M., Li, K., & Chan, P. (2009). Regional homogeneity changes in patients with Parkinson’s disease. Human Brain Mapping, 30(5), 1502–1510.

    Article  PubMed  Google Scholar 

  • Yan, C., & Zang, Y. (2010). DPARSF: A MATLAB toolbox for “Pipeline” data analysis of resting-state fMRI. Frontiers in Systems Neuroscience, 4, 13.

    Google Scholar 

  • Yan, C., Liu, D., He, Y., Zou, Q., Zhu, C., et al. (2009). Spontaneous brain activity in the default mode network is sensitive to different resting-state conditions with limited cognitive load. PLoS One, 4(5), e5743.

    Article  PubMed  Google Scholar 

  • Yang, H., Long, X. Y., Yang, Y., Yan, H., Zhu, C. Z., Zhou, X. P., Zang, Y. F., & Gong, Q. Y. (2007). Amplitude of low frequency fluctuation within visual areas revealed by resting-state functional MRI. NeuroImage, 36(1), 144–152.

    Article  PubMed  Google Scholar 

  • Zang, Y., Jiang, T., Lu, Y., He, Y., & Tian, L. (2004). Regional homogeneity approach to fMRI data analysis. NeuroImage, 22(1), 394–400.

    Article  PubMed  Google Scholar 

  • Zang, Y. F., He, Y., Zhu, C. Z., Cao, Q. J., Sui, M. Q., Liang, M., Tian, L. X., Jiang, T. Z., & Wang, Y. F. (2007). Altered baseline brain activity in children with ADHD revealed by resting-state functional MRI. Brain & Development, 29(2), 83–91.

    Article  Google Scholar 

  • Zhang, D., & Raichle, M. E. (2010). Disease and the brain’s dark energy. Nature Reviews. Neurology, 6(1), 15–28.

    Article  PubMed  Google Scholar 

  • Zuo, X. N., Di Martino, A., Kelly, C., Shehzad, Z. E., Gee, D. G., Klein, D. F., Castellanos, F. X., Biswal, B. B., & Milham, M. P. (2010a). The oscillating brain: complex and reliable. NeuroImage, 49(2), 1432–1445.

    Article  PubMed  Google Scholar 

  • Zuo, X. N., Kelly, C., Adelstein, J. S., Klein, D. F., Castellanos, F. X., & Milham, M. P. (2010b). Reliable intrinsic connectivity networks: test-retest evaluation using ICA and dual regression approach. NeuroImage, 49(3), 2163–2177.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant No. 30770594) and the National High Technology Program of China (863) (Grant No. 2008AA02Z405).

Conflict of Interest

The authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. Center for Cognition and Brain Disorders, Hangzhou Normal University, Hangzhou, 310015, China

    Dongqiang Liu, Jue Wang & Yufeng Zang

  2. National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China

    Zhangye Dong, Jue Wang & Yufeng Zang

  3. Laboratory for Functional Connectome and Development, Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, China

    Xinian Zuo

Authors
  1. Dongqiang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhangye Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinian Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jue Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yufeng Zang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongqiang Liu.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Supplemental Table 1

(PDF 9 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, D., Dong, Z., Zuo, X. et al. Eyes-Open/Eyes-Closed Dataset Sharing for Reproducibility Evaluation of Resting State fMRI Data Analysis Methods. Neuroinform 11, 469–476 (2013). https://doi.org/10.1007/s12021-013-9187-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12021-013-9187-0

Keywords

Search

Navigation