Abstract
Many content analysis studies involving temporal data are biased by some unknown dose of autocorrelation. The effect of autocorrelation is to inflate or deflate the significant differences that may exist among the different parts of texts being compared. The solution consists in removing effects due to autocorrelation, even if the latter is not statistically significant. Procedures such as Crosbie's (1993) ITSACORR remove the effect of at least first-order autocorrelations and can be used with small samples. The AREG procedure of SPSS (1994) and the AUTOREG procedure of SAS (1993) can be employed to detect and remove first-order autocorrelations, and higher-order ones too in the case of AUTOREG, while several methods specifically intended for small samples (Huitema and McKean, 1991, 1994) have been developed. Four examples of content analysis studies with and without autocorrelation are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Appleby, J., L. Hunt, and M. Jacob. Telling the Truth about History. New York: Norton, 1994.
Baayen, H. “Statistical Models for Word Frequency Distributions: A Linguistic Evaluation.” Computers and the Humanities 26 (1992), 347–363.
Box, G. E. P., G. M. Jenkins, and G. C. Reinsel. Time-series Analysis: Forecasting and Control (3d ed.). Englewood Cliffs, NJ: Prentice-Hall, 1994.
Bratley, P. and P. A. Fortier. “Themes, Statistics and the French Novel.” In Sixth International Conference on Computers in the Humanities. Eds. S. K. Burton and D. D. Short. Rockville, MD: Computer Science Press, 1983, pp. 18–25.
Braudel, F. “Histoire et Sciences Sociales: La Longue Durée” [History and Social Science: The Long Duration]. Annales 13 (1958), 725–753.
Brunet, E. “What Do Statistics Tell Us?.” In Research in Humanities Computing. 1. Selected Papers from the ALLC/ACH Conference, Toronto, June 1989. Ed. I. Lancashire. Oxford, England: Clarendon Press, 1991, pp. 70–92.
Burman, P., E. Chow, and D. A. Noland. “A Cross-validatory Method for Dependent Data.” Biometrika 81 (1994), 351–358.
Cohen, J. and P. Cohen. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates, 1975.
Crosbie, J. “The Inability of the Binomial Test to Control Type I Error with Single-subject Data.” Behavioral Assessment 9 (1987), 141–150.
Crosbie, J. “Interrupted Time-series Analysis with Brief Single-subject Data.” Journal of Consulting and Clinical Psychology 61 (1993), 966–974.
DeCarlo, L. T. and W. W. Tryon. “Estimating and Testing Autocorrelation with Small Samples: A Comparison of the C-statistic to a Modified Estimator.” Behaviour Research & Therapy 3 (1993), 781–788.
Flôres, R. G., Jr. and V. A. Ginsburgh. “The Queen Elisabeth Musical Competition: How Fair is the Final Ranking?.” The Statistician 45 (1996), 97–104.
Gottman, J. M. Time Series Analysis: A Comprehensive Introduction for Social Scientists. Cambridge, England: Cambridge University Press, 1981.
Hayes, A. F. “Permutation Test is not Distribution-free: Testing H0: p = 0.” Psychological Methods 1 (1996), 184–198.
Hogenraad, R., Y. Bestgen, and J. F. Durieux. “Psychology as Literature.” Genetic, Social, and General Psychology Monographs 118 (1992), 455–478.
Hogenraad, R., C. Daubies, and Y. Bestgen. Une Théorie et une Méthode Générale d'Analyse Textuelle Assistée par Ordinateur: Le Système PROTAN (PROTocol ANalyzer), Version du 2 mars 1995 [A General Theory and Method of Computer-aided Text Analysis: The PROTAN System (PROTocol Analyzer), Version of March 2, 1995]. Unpublished document, Psychology Department, Catholic University of Louvain, Louvain-la-Neuve, Belgium, 1995, 265 pages [http://www.psp.ucl.ac.be/~upso/protan/protanae.html].
Hogenraad, R., D. P. McKenzie, J. Morval, and F. A. Ducharme. “Paper Trails of Psychology: The Words that Made Applied Behavioral Sciences.” Journal of Social Behavior and Personality 10 (1995), 491–516.
Huitema, B. E. and J. W. McKean. “Autocorrelation Estimation and Inference with Small Samples.” Psychological Bulletin 110 (1991), 291–304.
Huitema, B. E. and J. W. McKean. “Two Reduced-bias Autocorrelation Estimators: rF1 and rF2.” Perceptual and Motor Skills 78 (1994), 323–330.
Ide, N. M. “A Statistical Measure of Theme and Structure.” Computers and the Humanities 23 (1989), 277–283.
IMSL. User's Manual, STAT/Library: FORTRAN Subroutines for Statistical Analysis, Version 2. Houston, TX: IMSL, 1991.
Judd, C. M., G. H. McLelland, and S. E. Culhane. “Data Analysis: Continuing Issues in the Everyday Analysis of Psychological Data.” Annual Review of Psychology 46 (1995), 433–465.
Kenny, D. A. and C. M. Judd. “Consequences of Violating the Independence Assumption in the Analysis of Variance.” Psychological Bulletin 99 (1986), 422–431.
Martindale, C. The Clockwork Muse: The Predictability of Artistic Change. New York: Basic Books, 1990.
McKenzie, D. P., D. M. Clarke, and C. Martindale. “Autocorrelation and Admission Diversion” [letter and reply]. Psychiatric Services 47 (1996), 91–92.
Moore, K. J., D. W. Osgood, R. E. Larzelere, and P. Chamberlain. “Use of Pooled Time Series in the Study of Naturally Occurring Clinical Events and Problem Behavior in a Foster Care Setting.” Journal of Counseling and Clinical Psychology 62 (1994), 718–728.
Noreen, E. W. Computer-intensive Methods for Testing Hypotheses: An Introduction. New York: Wiley, 1989.
Ostrom, C. W., Jr. Time Series Analysis: Regression Techniques (2nd ed.). Newbury Park, CA: Sage, 1990.
Pollock, D. S. G. “The Methods of Time-series Analysis.” Interdisciplinary Science Reviews 12 (1987), 128–135.
SAS Institute, Inc. SAS User's Guide: Statistics, Version 5 Edition. Cary, NC: SAS Institute Inc., 1985.
SAS Institute, Inc. SAS/ETS User's Guide, Version 6 (2nd ed.). Cary, NC: SAS Institute Inc., 1993.
Sells, S. P., T. E. Smith, and D. H. Sprenkle. “Integrating Qualitative and Quantitative Research Methods: A Research Model.” Family Process 34 (1995), 199–218.
Sigelman, L. “By Their (New) Words Shall Ye Know Them: Edith Wharton, Marion Mainwaring, and The Buccaneers.” Computers and the Humanities 29 (1995), 271–283.
Simonton, D. K. “Cross-sectional Time-series Experiments: Some Suggested Statistical Analyses.” Psychological Bulletin 84 (1977), 489–502.
Simonton, D. K. Psychology, Science, and History: An Introduction to Historiometry. New Haven, CT: Yale University Press, 1990.
Snow, C. P. The Two Cultures. Cambridge, England: Cambridge University Press, 1959.
SPSS, Inc. SPSS Trends 6.1. Chicago: SPSS Inc., 1994.
Stubbs, M. “Collocations and Semantic Profiles: On the Cause of the Trouble with Quantitative Studies.” Functions of Language 2 (1995), 23–55.
West, A. N. “Primary Process Content in the King James Bible: The Five Stages of Christian Mysticism.” Computers and the Humanities 25 (1991), 227–238.
Whissell, C., M. Fournier, R. Pelland, D. Weir, and K. Makarec. “A Dictionary of Affect in Language. IV Reliability, Validity, and Applications.” Perceptual and Motor Skills 62 (1986), 875–888.
Young, G. A. “Bootstrap: More Than a Stab in the Dark.” Statistical Science 9 (1994), 382–415.
Zimmerman, D. W. “A Note on Nonindependence and Nonparametric Tests.” Perceptual and Motor Skills 76 (1993), 407–412.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hogenraad, R., McKenzie, D.P. & Martindale, C. The enemy within: Autocorrelation bias in content analysis of narratives. Comput Hum 30, 433–439 (1996). https://doi.org/10.1007/BF00057939
Issue Date:
DOI: https://doi.org/10.1007/BF00057939