Abstract
We conducted a computer simulation of hundreds of competitions for limited journal space, varying (a) the correlation between the talent of authors and the quality of their manuscripts, (b) the correlation between manuscript quality and quality judged by peer reviewers, (c) the weights reviewers and editors gave judged quality versus number of previous publications (tract record), and (d) the proportion of manuscripts accepted for publication. The results show that even small decreases in the correlations, and small increases in the weight given to track record, quickly skew the outcomes of the peer review process, favouring authors who develop a track record of publications in the first cycles of journal publication while excluding many equally-talented or more-talented authors from publishing (the Matthew Effect; Merton, 1968). Implications for declines in the quality of published manuscripts and for wasting talent are discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allesina, S.: Modelling peer review: An agent-based approach. Ideas in Ecology and Evolution 5 (2012)
APA: Summary report of journal operations (2011), http://www.apa.org/pubs/journals/features/2011-statistics.pdf
Bornmann, L., Daniel, H.D.: Gatekeepers of science – Effect of external reviewers’ attributes on the assessment of fellowship applications. Journal of Informetrics 1, 83–91 (2007)
Bourne, P.E., Barbour, V.: Ten simple rules for building and maintaining a scientific reputation. PLoS Computational Biology: a Peer Reviewed Open-Access Journal 7(6) (2011)
Day, A., Peters, J.: Quality Indicators in Academic Publishing. Library Review 43(7), 4–72 (1994)
Dewett, T., Denisi, A.S.: Exploring scholarly reputation: It’s more than just productivity. Scientometrics 60(2), 249–272 (2004)
Ferber, M.A.: Citations: Are they an objective measure of scholarly merit? Signs 11(2), 381–389 (1986)
Grimaldo, F., Paolucci, M.: A simulation of disagreement for control of rational cheating in peer review. Advances in Complex Systems (2013), doi:10.1142/S0219525913500045
Herron, D.: Is expert peer review obsolete? Surgical Endoscopy 26, 2275–2280 (2012)
Howard, G.S., Cole, D.A., Maxwell, S.E.: Research productivity in psychology based on publication in the journals of the American Psychological Association. American Psychologist 42(11), 975–986 (1987)
Kaufman, A.: IQ Testing 101. Springer Publishing, New York (2009)
Merton, R.K.: The Matthew Effect in Science. Science 159(3810), 56–63 (1968)
Morgan, D.R., Meier, K.J., Kearney, R.C., Hays, S.W., Birch, H.B.: Reputation and productivity among U. S. public administration and public affairs programs. Public Administration Review 41(6), 666–673 (1981)
Natural Sciences and Engineering Research Council: Guidelines for the preparation and review of applications in engineering and the applied sciences (2012), http://www.nserc-crsng.gc.ca/NSERC-CRSNG/Policies-Politiques/prepEngAS-prepGenSA_eng.asp
Ofori-Dankwa, J., Julian, S.: From thought to theory to school: The role of contextual factors in the evolution of schools of management thought. Organization Studies 26(9), 1307–1329 (2005)
Petty, R., Fleming, M., Fabrigar, L.: The review process at PSPB: Correlates of interreviewer agreement and manuscript acceptance. Personality and Social Psychology Bulletin 25(2), 188–203 (1999)
R Core Team: R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2012) ISBN 3-900051-07-0, http://www.R-project.org/
Squazzoni, F., Gandelli, C.: Saint Matthew strikes again: An agent-based model of peer review and he scientific community structure. Journal of Informetrics 6, 265–275 (2012)
Thorngate, W.: On the evolution of adjudicated contests and the principle of invidious selection. Journal of Behavioral Decision Making 1, 5–16 (1988)
Thorngate, W., Dawes, R.M., Foddy, M.: Judging Merit. Psychology Press, Taylor & Francis Group, New York (2009)
Thorngate, W., Hotta, M.: Life and luck: Survival of the fattest. Simulation & Gaming 26, 5–16 (1995)
Thorngate, W., Hotta, M., McClintock, C.: Bingo! The case for cooperation revisited. In: Tolman, C.W., Cherry, F., Van Hezewijk, R., Lubek, I. (eds.) Problems of Theoretical Psychology, Captus Press Inc., Canada (1996)
Thorngate, W., Liu, J., Chowdhury, W.: The Competition for Attention and the Evolution of Science. Journal of Artificial Societies and Social Simulation 14(4) (2011)
Whitehurst, G.J.: Interrater agreement for journal manuscript reviews. American Psychologist 39(1), 22–28 (1984)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Thorngate, W., Chowdhury, W. (2014). By the Numbers: Track Record, Flawed Reviews, Journal Space, and the Fate of Talented Authors. In: Kamiński, B., Koloch, G. (eds) Advances in Social Simulation. Advances in Intelligent Systems and Computing, vol 229. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39829-2_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-39829-2_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39828-5
Online ISBN: 978-3-642-39829-2
eBook Packages: EngineeringEngineering (R0)