Skip to main content
Log in

The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators

Scientometrics Aims and scope Submit manuscript

Abstract

The objective of this paper is to explore the impact of preprints in scholarly and broader scientific communication. In particular, the following four indicators are used to examine the 508 arXiv and 5536 non-arXiv papers in three major journals in Library and Information Science: citations from Web of Science (WoS), Scopus and Google Scholar, usage counts in WoS, Mendeley readers and Tweets. The results show that arXiv papers have significant citation advantage across WoS, Scopus and Google Scholar in each year. Google Scholar provides statistically significantly larger number of citations and more ‘early citations’ than Scopus and WoS, but does not reflect greater citation advantage for arXiv papers. The impact advantage of arXiv papers can also be observed in Mendeley readers and in Tweets, but to a much lesser extent in WoS usage counts, indicating that arXiv papers gain broader attention than non-arXiv papers not only from the users of WoS. arXiv papers have higher Altmetric coverage and shorter attention delay on social media compared with non-arXiv papers. Mendeley readership as well as the usage counts in WoS have strong correlations with WoS citations, which are much stronger than those of Tweets. We can also conclude that unlike citations, information derived from statistics on users, readers and social media needs further exploration and in the case of social media also proper context analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  • Abdill, R. J., & Blekhman, R. (2019). Tracking the popularity and outcomes of all bioRxiv preprints. eLife, 8, e45133.

    Google Scholar 

  • Aman, V. (2014). Is there any measurable benefit in publishing preprints in the arXiv section quantitative biology? Retrieved from http://arxiv.org/abs/1411.1955.

  • Bar-Ilan, J. (2014). JASIST@Mendeley revisited. In ACM Web Science Conference 2014 Workshop. Retrieved from http://files.figshare.com/1504021/JASIST_new_revised.pdf.

  • Bornmann, L. (2014). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895–903.

    Google Scholar 

  • Bornmann, L., & Haunschild, R. (2015). Which people use which scientific papers? An evaluation of data from F1000 and Mendeley. Journal of Informetrics, 9(3), 477–487.

    Google Scholar 

  • Brody, T., Harnad, S., & Carr, L. (2006). Earlier web usage statistics as predictors of later citation impact. Journal of the American Society for Information Science and Technology, 57(8), 1060–1072.

    Google Scholar 

  • Brown, C. (2001). The E-volution of preprints in the scholarly communication of physicists and astronomers. Journal of the American Society for Information Science and Technology, 52(3), 187–200.

    Google Scholar 

  • Brown, C. (2003). The role of electronic preprints in chemical communication: Analysis of citation, usage, and acceptance in the journal literature. Journal of the American Society for Information Science and Technology, 54(5), 362–371.

    Google Scholar 

  • Chi, P. S., & Glänzel, W. (2017). An empirical investigation of the associations among usage, scientific collaboration and citation impact. Scientometrics, 112(1), 403–412.

    Google Scholar 

  • Chi, P. S., & Glänzel, W. (2018). Comparison of citation and usage indicators in research assessment in scientific disciplines and journals. Scientometrics, 116(1), 537–554.

    Google Scholar 

  • Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019.

    Google Scholar 

  • Côté, I. M., & Darling, E. S. (2018). Scientists on Twitter: Preaching to the choir or singing from the rooftops? Facets, 3(1), 682–694.

    Google Scholar 

  • Davis, P. M. (2011). Open access, readership, citations: A randomized controlled trial of scientific journal publishing. The FASEB Journal, 25(7), 2129–2134.

    Google Scholar 

  • Davis, P. M., & Fromerth, M. J. (2007). Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles? Scientometrics, 71(2), 203–215.

    Google Scholar 

  • Fang, Z., & Costas, R. (2018). Studying the velocity index for various altmetric. In Proceedings of the 23rd international conference on science and technology indicators (STI2018), Leiden University, pp. 450–456.

  • Feldman, S., Lo, K., & Ammar, W. (2018). Citation count analysis for papers with preprints, pp. 1–7. Retrieved from http://arxiv.org/abs/1805.05238.

  • Fraser, N., Momeni, F., Mayr, P., & Peters, I. (2020). The relationship between bioRxiv preprints, citations and altmetrics. Quantitative Science Studies, 1(2), 618–638.

    Google Scholar 

  • Fu, D. Y., & Hughey, J. J. (2019). Releasing a preprint is associated with more attention and citations for the peer-reviewed article. eLife, 8, e52646.

    Google Scholar 

  • Gargouri, Y., Hajjem, C., Larivière, V., Gingras, Y., Carr, L., Brody, T., et al. (2010). Self-selected or mandated, open access increases citation impact for higher quality research. PLoS ONE, 5(10), e13636.

    Google Scholar 

  • Gentil-Beccot, A., Mele, S., & Brooks, T. C. (2010). Citing and reading behaviours in high-energy physics. Scientometrics, 84(2), 345–355.

    Google Scholar 

  • Glänzel, W., & Chi, P.-S. (2020). The big challenge of Scientometrics 2.0: Exploring the broader impact of scientific research in public health. Scientometrics. https://doi.org/10.1007/s11192-020-03473-x.

    Article  Google Scholar 

  • Glänzel, W., & Gorraiz, J. (2015). Usage metrics versus altmetrics: Confusing terminology? Scientometrics, 102(3), 2161–2164.

    Google Scholar 

  • Glänzel, W., & Heeffer, S. (2014). Cross-national preferences and similarities in downloads and citations of scientific articles: A pilot study. In Proceedings of the 19th international conference on science and technology indicators (STI2014), Leiden University, pp. 207–215.

  • Glänzel, W., & Thijs, B. (2004). The influence of author self-citations on bibliometric macro indicators. Scientometrics, 59(3), 281–310.

    Google Scholar 

  • Glänzel, W., Thijs, B., & Chi, P. S. (2016). The challenges to expand bibliometric studies from periodical literature to monographic literature with a new data source: The book citation index. Scientometrics, 109(3), 2165–2179.

    Google Scholar 

  • Glänzel, W., Thijs, B., & Debackere, K. (2014). The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment. Scientometrics, 101(2), 939–952.

    Google Scholar 

  • Harzing, A. W. (2007). Publish or perish. https://harzing.com/resources/publish-or-perish.

  • Harzing, A. W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison. Scientometrics, 106(2), 787–804.

    Google Scholar 

  • Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2014). Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology, 65(4), 656–669.

    Google Scholar 

  • Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.

    MathSciNet  MATH  Google Scholar 

  • Henneken, E. A., Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Thompson, D., et al. (2006). Effect of e-printing on citation rates in astronomy and physics. Retrieved from http://arxiv.org/abs/cs/0604061. Accessed May 2020.

  • Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, 101(2), 1027–1042.

    Google Scholar 

  • Kim, J. (2010). Faculty self-archiving: Motivations and barriers. Journal of the American Society for Information Science and Technology, 61(9), 1909–1922.

    Google Scholar 

  • Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Henneken, E., et al. (2005). The effect of use and access on citations. Information Processing and Management, 41(6), 1395–1402.

    Google Scholar 

  • Larivière, V., Sugimoto, C. R., Macaluso, B., Milojević, S., Cronin, B., & Thelwall, M. (2014). arXiv E-prints and the journal of record: An analysis of roles and relationships. Journal of the Association for Information Science and Technology, 65(6), 1157–1169.

    Google Scholar 

  • Lawrence, S. (2001). Online or invisible. Nature, 411(6837), 521.

    Google Scholar 

  • Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461–471.

    Google Scholar 

  • Maflahi, N., & Thelwall, M. (2016). When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals. Journal of the Association for Information Science and Technology, 67(1), 191–199.

    Google Scholar 

  • Maflahi, N., & Thelwall, M. (2018). How quickly do publications get read? The evolution of Mendeley reader counts for new articles. Journal of the Association for Information Science and Technology, 69(1), 158–167.

    Google Scholar 

  • Martín-Martín, A., Orduna-Malea, E., Thelwall, M., & Delgado López-Cózar, E. (2018). Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories. Journal of Informetrics, 12(4), 1160–1177.

    Google Scholar 

  • Moed, H. F. (2005). Statistical relationships between downloads and citations at the level of individual documents within a single journal. Journal of the American Society for Information Science and Technology, 56(10), 1088–1097.

    Google Scholar 

  • Moed, H. F. (2007). The effect of “open access” on citation impact: An analysis of ArXiv’s condensed matter section. Journal of the American Society for Information Science and Technology, 58(13), 2047–2054.

    Google Scholar 

  • Moed, H. F., Bar-Ilan, J., & Halevi, G. (2016). A new methodology for comparing Google Scholar and Scopus. Journal of Informetrics, 10(2), 533–551.

    Google Scholar 

  • Mohammadi, E., & Thelwall, M. (2014). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the Association for Information Science and Technology, 65(8), 1627–1638.

    Google Scholar 

  • Mohammadi, E., Thelwall, M., & Kousha, K. (2016). Can Mendeley bookmarks reflect readership? A survey of user motivations. Journal of the Association for Information Science and Technology, 67(5), 1198–1209.

    Google Scholar 

  • Priem, J., & Hemminger, B. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday. https://doi.org/10.5210/fm.v15i7.2874.

    Article  Google Scholar 

  • Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics : A manifesto. Retrieved from http://altmetrics.org/manifesto. Accessed May 2020.

  • Schubert, A., & Glänzel, W. (1986). Mean response time. A new indicator of journal citation speed with application to physics journals. Czechoslovak Journal of Physics, B36, 121–125.

    Google Scholar 

  • Serghiou, S., & Ioannidis, J. P. A. (2018). Altmetric scores, citations, and publication of studies posted as preprints. JAMA, 319(4), 402.

    Google Scholar 

  • Shuai, X., Pepe, A., & Bollen, J. (2012). How the scientific community reacts to newly submitted preprints: Article downloads, Twitter mentions, and citations. PLoS ONE, 7(11), e47523.

    Google Scholar 

  • Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics, 98(2), 1131–1143.

    Google Scholar 

  • Teixeira da Silva, J. A. (2017). Preprints should not be cited. Current Science, 113(6), 1026–1027.

    Google Scholar 

  • Teixeira da Silva, J. A. (2018). The preprint debate: What are the issues? Medical Journal Armed Forces India, 74(2), 162–164.

    Google Scholar 

  • Thelwall, M. (2017). Are Mendeley reader counts useful impact indicators in all fields? Scientometrics, 113(3), 1721–1731.

    Google Scholar 

  • Thelwall, M. (2018). Can Microsoft Academic be used for citation analysis of preprint archives? The case of the social science research network. Scientometrics, 115(2), 913–928.

    Google Scholar 

  • Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), 1–7.

    Google Scholar 

  • Thelwall, M., & Kousha, K. (2015). Web indicators for research evaluation. Part 2: Social media metrics. El Profesional de la Información, 24(5), 607–620.

    Google Scholar 

  • Thelwall, M., & Kousha, K. (2017a). ResearchGate versus Google Scholar: Which finds more early citations? Scientometrics, 112(2), 1125–1131.

    Google Scholar 

  • Thelwall, M., & Kousha, K. (2017b). ResearchGate articles: Age, discipline, audience size, and impact. Journal of the Association for Information Science and Technology, 68(2), 468–479.

    Google Scholar 

  • Thelwall, M., & Wilson, P. (2016). Mendeley readership altmetrics for medical articles: An analysis of 45 fields. Journal of the Association for Information Science and Technology, 67(8), 1962–1972.

    Google Scholar 

  • Wang, X., Fang, Z., & Sun, X. (2016). Usage patterns of scholarly articles on Web of Science: a study on Web of Science usage count. Scientometrics, 109(2), 917–926.

    Google Scholar 

  • Wang, Z., Glänzel, W., & Chen, Y. (2018). How self-archiving influences the citation impact of a paper: A bibliometric analysis of arXiv papers and non-arXiv papers in the field of information science and library science. In Proceedings of the 23rd international conference on science and technology indicators (STI2018), Leiden University, pp. 323–330.

  • Wang, Z., Glänzel, W., & Chen, Y. (2019). The impact of preprints in Library and Information Science: citations, usage, and social attention. In Proceedings of the 17th international conference on scientometrics & informetrics (ISSI2019), Sapienza University, pp. 459–470.

  • Wang, X., Liu, C., Mao, W., & Fang, Z. (2015). The open access advantage considering citation, article usage and social media attention. Scientometrics, 103(2), 555–564.

    Google Scholar 

  • Wang, X., Mao, W., Xu, S., & Zhang, C. (2014). Usage history of scientific literature: Nature metrics and metrics of Nature publications. Scientometrics, 98(3), 1923–1933.

    Google Scholar 

  • Web of Science. (2020). Usage counts. Clarivate analytics. Retrieved from https://images.webofknowledge.com/WOKRS529AR7/help/WOS/hp_usage_score.html. Accessed May 2020.

  • Youngen, G. K. (1998). Citation patterns to traditional and electronic preprints in the published literature. College & Research Libraries, 59(5), 448–456.

    Google Scholar 

  • Zhou, P., & Leydesdorff, L. (2011). Fractional counting of citations in research evaluation: A cross- and interdisciplinary assessment of the Tsinghua University in Beijing. Journal of Informetrics, 5(3), 360–368.

    Google Scholar 

Download references

Acknowledgements

This paper is an extended version of a previous work presented at the 17th International Conference on Scientometrics and Informetrics (ISSI2019) in Rome, Italy (Wang et al. 2019). We thank two anonymous reviewers for their insightful comments and suggestions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhiqi Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Glänzel, W. & Chen, Y. The impact of preprints in Library and Information Science: an analysis of citations, usage and social attention indicators. Scientometrics 125, 1403–1423 (2020). https://doi.org/10.1007/s11192-020-03612-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-020-03612-4

Keywords

Navigation