Skip to main content
SpringerLink
Log in

CiteULike bookmarks are correlated to citations at journal and author levels in library and information science

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Aiming to explore the applicability of bookmarking data in measuring the scientific impact, the present study investigates the correlation between conventional impact indicators (i.e. impact factors and mean citations) and bookmarking metrics (mean bookmarks and percentage of bookmarked articles) at author and journal aggregation levels in library and information science (LIS) field. Applying the citation analysis method, it studies a purposeful sample of LIS articles indexed in SSCI during 2004–2012 and bookmarked in CiteULike. Data are collected via WoS, Journal Citation Report, and CiteULike. There is a positive, though weak, correlation between LIS authors’ mean citations and their mean bookmarks, as well as a moderate to large correlation between LIS journals’ impact factors on the one hand and on the other, their mean bookmarks, and the percentage of their bookmarked articles. Given the correlation between the citation- and bookmark-based indicators at author and journal levels, bookmarking data can be used as a complement to, but not a substitute for, the traditional indicators to get to a more inclusive evaluation of journals and authors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bar-Ilan, J. (2012). JASIST 2001–2010. Bulletin of the American Society for Information Science and Technology, 38(6), 24–28.

    Article  Google Scholar 

  • Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars’ visibility on the social web. arXiv preprint arXiv:1205.5611.

  • Bornmann, L. (2014). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895–903.

    Article  Google Scholar 

  • Bornmann, L. (2015). Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics, 103(3), 1123–1144.

    Article  Google Scholar 

  • Borrego, Á., & Fry, J. (2012). Measuring researchers’ use of scholarly information through social bookmarking data: A case study of BibSonomy. Journal of Information Science, 38(3), 297–308.

    Article  Google Scholar 

  • Butler, L. (2008). Using a balanced approach to bibliometrics: Quantitative performance measures in the Australian research quality framework. Ethics in Science and Environmental Politics, 8(1), 83–92.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.

    MATH  Google Scholar 

  • Costas, R., Zahedi, Z., & Wouters, P. (2014). Do altmetrics correlate with citations? Extensivecomparison of altmetric indicators with citations from a multidisciplinary perspective. Arxiv preprint arXiv: 1401.4321.

  • Garfield, E. (1986). Which medical journals have the greatest impact? Annals of Internal Medicine, 105(2), 313–320.

    Article  Google Scholar 

  • Harnad, S. (2008).Validating research performance metrics against peer rankings. Ethics in Science and Environmental Politics, 8(11). Retrieved July 10, 2010, from http://eprints.ecs.soton.ac.uk/15619/1/esep-harnad.html.

  • Haustein, S., Golov, E., Luckanus, K., Reher, S., & Terliesner, J. (2010). Journal evaluation and science 2.0.Using social bookmarks to analyze reader perception. “In book of abstracts of the 11 th International Conference on Science and Technology Indicators”, (pp. 117–119). Leiden, The Netherlands.

  • Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (2014). Tweets versus Mendeley readers: How do these two social media metrics differ? IT-Information Technology, 56(5), 207–215.

    Article  Google Scholar 

  • Li, X., & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. In E. Archambault, Y. Gingras & V. Lariviere (Eds.), The 17 th International Conference on Science and Technology Indicators, (pp. 541–551). Montreal, Canada: Repro-UQAM.

  • Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461–471.

    Article  Google Scholar 

  • Maflahi, N. & Thelwall, M. (in press).When are readers as good as citers for bibliometrics? Scopus versus Mendeley for LIS journals. Journal of the Association for Information Science and Technology.

  • Mohammadi, E., & Thelwall, M. (2014). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the Association for Information Science and Technology, 65(8), 1627–1638.

    Article  Google Scholar 

  • Mohammadi, E., Thelwall, M., Haustein, S., & Larivière, V. (in press).Who reads research articles? An altmetrics analysis of Mendeley user categories. Journal of the Association for Information Science and Technology.

  • Mohammadi, E., Thelwall, M., & Kousha, K. (in press). Can Mendeley bookmarks reflect readership? A survey of user motivations. Journal of the Association for Information Science and Technology.

  • Ogden, T.L. & Bartley, D.L. (2008). “The ups and downs of journal impact factors.” Annals of Occupational Hygiene, 52(2):73–82. Retrieved July 10, 2010, from http://annhyg.oxfordjournals.org/cgi/reprint/52/2/73.

  • Pallant, J. (2013). SPSS survival manual. UK: McGraw-Hill Education.

    Google Scholar 

  • Schlögl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2013). Download versus citation versus readership data: The case of an information systems journals. In Proceedings of the 14 th International Society of Scientometrics and Informetrics Conference, (pp. 626–634). Vienna, Austria.

  • Sotudeh, H. (2010). A review of the journal impact factor and its deficiencies in research evaluation in different disciplines. Rahyaft, 47, 33–44. [in Persian].

    Google Scholar 

  • Sotudeh, H., Mazare’i, Z., & Mirza-beighi, M. (2015). The relationship between citation indicators and CiteULike bookmarks: The case of LIS field articles during 2004–2012. Information Processing and Management [in Persian], 30(4), 939–963.

    Google Scholar 

  • Taraborelli, D. (2008). Soft peer review: Social software and distributed scientific evaluation. In Proceedings of the eighth international conference on the design of cooperative systems (COOP ’08; Carry–Le–Rouet, 20–23 May). Available at http://nitens.org/docs/spr_coop08.pdf.

  • Thelwall, M. (2012). Journal impact evaluation: A webometric perspective. Scientometrics, 92(2), 429–441.

    Article  Google Scholar 

  • Wang, X., Wang, Z., & Xu, S. (2012). Tracing scientist’s research trends realtimely. Scientometrics, 95(2), 717–729.

    Article  Google Scholar 

  • Zahedi, Z., Costas, R., & Wouters, P. (2014a). How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics, 101(2), 1491–1513.

    Article  Google Scholar 

  • Zahedi, Z., Fenner, M., & Costas, R. (2014). How consistent are altmetrics providers? Study of 1000 PLOS ONE publications using the PLOS ALM, Mendeley and Altmetric.com APIs. In altmetrics 14. Workshop at the Web Science Conference, Bloomington, USA.

  • Zahedi, Z., & Van Eck, N.J. (2014). Visualizing readership activity of Mendeley users using VOSviewer. In altmetrics14: Expanding impacts and metrics, Workshop at Web Science Conference 2014, Bloomington, IN, doi: 10.6084/m9. figshare (Vol. 1041819).

  • Zitt, M., & Bassecoulard, E. (2008). Challenges for scientometric indicators: Data demining, knowledge-flow measurements and diversity issues. Ethics in Science and Environmental Politics, 8, 49–60.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Knowledge and Information Sciences, Faculty of Education and Psychology, Shiraz University, Eram Campus, Shiraz, Iran

    Hajar Sotudeh, Zahra Mazarei & Mahdieh Mirzabeigi

Authors
  1. Hajar Sotudeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zahra Mazarei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahdieh Mirzabeigi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hajar Sotudeh.

Appendix

Appendix

See Table 3.

Table 3 Top ten journals in terms of their JIF, JB % and JMB
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sotudeh, H., Mazarei, Z. & Mirzabeigi, M. CiteULike bookmarks are correlated to citations at journal and author levels in library and information science. Scientometrics 105, 2237–2248 (2015). https://doi.org/10.1007/s11192-015-1745-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-015-1745-9

Keywords

Search

Navigation