Abstract
It has been widely discussed how individuals change the way they act and react in studies just because they are under observation. In this paper, we try to analyse how this so-called Hawthorne effect applies to researchers that are the subject of bibliometric investigations. This encompasses individual assessments as well as international performance comparisons. We test various bibliometric indicators for notable changes in the last decade from a world-wide perspective and deduce explanations for changes from the observations. We then concentrate on the behaviour of German authors in particular, to show national trends. The German publication behaviour is evaluated in regard to citation rates and collaborations in publications and size, publisher country and impact of the journals chosen for publication. We can conclude that authors adapt their publication behaviour to aim for journals that are more internationally known and have a US publisher. Also, a trend from more specialized journals to journals with a broader scope can be observed that raises the question whether the implicit penalization of specialized fields in the bibliometrics leads to undesired shifts in conducted research.
Similar content being viewed by others
Notes
For more methodological details cf. Schmoch et al. (2012).
However, the high standard deviation values indicate that there are also some exceptions to this rule where the citation rate is extremely high in both cases.
I.e. of all volumes that were covered in the Web of Science for the publication year 2008.
The median of the first quintile is 56.42 pages and the median of the fifth quintile is 329.94. The overall median in the set of 66,615 journals is a page size of 152.38.
In particular, in the medical sciences, the impact factor is generally asked, although the impact factor includes no information on the real citation rate of the respective article.
The main Dutch publisher is Elsevier.
References
Adams, J. (2005). Early citation counts correlate with accumulated impact. Scientometrics, 63, 567–581.
Aksnes, D. W. (2003). A macro study of self-citation. Scientometrics, 56, 235–246.
Aksnes, D. W., & Sivertsen, G. (2004). The effect of highly cited papers on national citation indicators. Scientometrics, 59, 213–224.
Bornmann, L. (2011). Mimicry in science? Scientometrics, 86, 173–177.
Butler, L. (2003). Modifying publication practices in response to funding formulas. Research Evaluation, 17(1), 39–46.
Glänzel, W., Thijs, B., & Schlemmer, B. (2004). A bibliometric approach to the role of author self-citations in scientific communication. Scientometrics, 59, 63–77.
Gondal, N. (2011). The local and global structure of knowledge production in an emergent research field: An exponential random graph analysis. Social Networks, 33(1), 20–30.
Grupp, H., Schmoch, U., & Hinze, S. (2001). International alignment and scientific regard as macro-indicators for international comparisons of publications. Scientometrics, 51(2), 359–380.
Jimenez-Contreras, E., Delgado Lopez-Cozar, E., Ruiz-Perez, R., & Fernandez, V. M. (2002). Impact-factor rewards affect Spanish research. Nature, 417(6892), 898.
Jones, S. R. G. (1992). Was there a Hawthorne effect? American Journal of Sociology, 98(3), 451–468.
Ltd, Evidence. (2007). The use of bibliometrics to measure research quality in UK higher education institutions. London: Universities UK.
Michels, C., & Schmoch, U. (2011), Growth of science and database coverage (Full paper) paper presented at the 13th conference of the international society for scientometrics and informetrics (ISSI), 4–7 July 2011, Durban, South Africa.
Michels, C., & Schmoch, U. (2012). The Growth of science and database coverage. Scientometrics, 93(3), 831–846.
Moed, H. (2005). Citation analysis in research evaluation. Dordrecht: Springer. (Information Science and Knowledge Management, 9).
Mueller, P. S., Murali, N. S., Cha, S. S., Erwin, P. J., & Ghosh, A. K. (2006). The association between impact factors and language of general internal medicine journals. Swiss Medical Weekly, 136, 441–443.
Schmoch, U., Michels, C., Neuhäusler, P., & Schulze, N. (2012). Performance and Structure of the German Science System 2011. Germany in an international comparison, China’s profile, behaviour of German authors, comparison of the Web of Science and Scopus. Berlin: Commission of Expert on Research and Innovation (www.e-fi.de).
Schneider, J. W. (2009). An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science, 8(3), 364–378.
Van Leeuwen, T. N., Moed, H. F., Tussen, R. J. W., Visser, M. S., & Van Raan, A. F. J. (2001). Language biases in the coverage of the science citation index and its consequences for international comparisons of national research performance. Scientometrics, 51(1), 335–346.
van Raan, A. J. F. (2004). Measuring science. Capita selecta of current main issues. In W. Glänzel, H. Moed, & U. Schmoch (Eds.), Handbook of quantitative science and technology research. The use of publication and patent statistics in studies on R&D systems (pp. 19–50). Dodrecht: Kluwer Academic Publishers.
Waltman, L, & Van Eck, N.J. (2013): Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison. Scientometrics in press.
Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 63(1), 117–131.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Michels, C., Schmoch, U. Impact of bibliometric studies on the publication behaviour of authors. Scientometrics 98, 369–385 (2014). https://doi.org/10.1007/s11192-013-1015-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-013-1015-7