Skip to main content
SpringerLink
Log in

Impact evaluation of open source software: an Altmetrics perspective

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

With the emergence of Web 2.0, an online platform which encourages online creation of next generation tools, communication has become a nigh-indispensable tool for researchers. Allowing them to acquire, spread, and share research achievements, with a free flow of ideas online. At present, there are a growing number of studies on non-traditional evaluation indicators, but there is much fewer research focused on the software evaluation, especially for open source research software. What this research focuses on is; with the use of the open source project ‘Depsy’, this research evaluated and analyzed data collected from downloads made online through these open source software. Altmetrics cannot be confined to traditional measurable indicators. That the importance of the open source software used, and its position in the online community is itself a strong measure for academic impact and success, which is all too often overlooked in research. The research can also conclude that the multiplexing of software online, through the citation of a citation, ultimately leads to an online peered review system within the community, effectively developing and maintaining through open use software itself. Moreover, the benefits of such a system has only just begun to come to fruition, having a strong impact on academic research, and predicting research impact.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article level discussion and metrics. Learned Publishing, 26(1), 11–17.

    Article  Google Scholar 

  • Asaishi, T. (2014). Bibliometric data of school library and their applications (beyond bibliometrics). Journal of Information Science & Technology Association, 64, 514–519.

    Google Scholar 

  • Barnes, C. (2015). The use of Altmetrics as a tool for measuring research impact. Australian Academic & Research Libraries, 46(2), 1–14.

    Article  Google Scholar 

  • Bauckhage, C. (2015). NumPy/SciPy Recipes for Image Processing: Creating Fractal Images[J]. researchgate.net, Feb 2015.

  • Bonnici, L. J. (2015). Altmetrics in library and information science: Trickle or Tsunami? 210–218.

  • Borchardt, R., & Roemer, R. C. (2013). Institutional altmetrics and academic libraries. Information Standards Quarterly, 25(2), 14–19.

    Article  Google Scholar 

  • Bornmann, L. (2014). Usefulness of altmetrics for measuring the broader impact of research: A case study using data from PLOS (altmetrics) and F1000Prime (paper tags).

  • Careless, J. Altmetrics 101: A Primer (2013). Information Today, 30(2).

  • Chamberlain, S. (2013). Consuming article-level metrics: Observations and lessons from comparing aggregator provider data. Information Standards Quarterly, 25(2):4–13.

    Article  Google Scholar 

  • Collection, T. A. (2012). The altmetrics collection. PLoS ONE, 7(11), e48753.

    Article  Google Scholar 

  • Cronin, B., & Meho, L. I. (2008). The shifting balance of intellectual trade in information studies. Journal of the American Society for Information Science and Technology, 59(4), 551–564.

    Article  Google Scholar 

  • Fenner, M. (2013). What can article-level metrics do for you? PLoS Biology, 11(10), 363–367.

    Article  Google Scholar 

  • Galloway, L. M., Pease, J. L., et al. (2013). Introduction to altmetrics for science, technology, engineering, and mathematics (STEM) librarians. Science & Technology Libraries, 32(4), 335–345.

    Article  Google Scholar 

  • Hammarfelt, B. (2014). Using altmetrics for assessing research impact in the humanities. Scientometrics, 101(2), 1419–1430.

    Article  Google Scholar 

  • Holbrook, B. (2012). Peer review, altmetrics, and ex ante broader impacts assessment—A proposal.

  • Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, 101(2), 1027–1042.

    Article  Google Scholar 

  • Keener, M., & Jeong S. (2013). Altmetrics and scholarly impact. Scholarly Publishing.

  • Kurtz, M. J., & Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science & Technology, 44(1), 1–64.

    Article  Google Scholar 

  • Kwak, H., Lee, J. G. (2014). Has much potential but biased: exploring the scholarly landscape in twitter. Proceedings of the 23rd International Conference on World Wide Web. ACM, 563–564.

  • Lin, J., & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards Quarterly, 25(2), 20–26.

    Article  Google Scholar 

  • Liu, C. L., Liu, C. L., et al. (2013). Correlation and interaction visualization of altmetric indicators extracted from scholarly social network activities: Dimensions and structure. Journal of Medical Internet Research, 15(11), 281–298.

    Article  Google Scholar 

  • Mohammadi, E., & Thelwall, M. (2014). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the American Society for Information Science and Technology, 65(8), 1627–1638.

    Article  Google Scholar 

  • Neylon, C., Wu, S. (2009) Article level metrics and the evalution of scientific impact. PLOS Biology, 7(11):e1000242. doi:10.1371/journal.pbio.1000242.

    Article  Google Scholar 

  • Pearson, W. H. (1966). Estimation of a correlation coefficient from an uncertainty measure. Psychometrika, 31(3), 421–433.

    Article  Google Scholar 

  • Peters, I., Kraker, P., et al. (2016). Research data explored: An extended analysis of citations and altmetrics. Scientometrics, 107, 1–22.

    Article  Google Scholar 

  • Piwowar, H. (2013a). Altmetrics: Value all research products. Nature, 493(7431), 159.

    Google Scholar 

  • Piwowar, H. (2013b). Altmetrics: Value all research products. Nature, 493(7431), 159.

    Google Scholar 

  • Piwowar, H., & Priem, J. (2013). The power of altmetrics on a CV. Bulletin of the American Society for Information Science & Technology, 39(39), 10–13.

    Article  Google Scholar 

  • Priem, J., Piwowar, H. A., Hemminger, B. M. (2012) Altmetrics in the wild: Using social media to explore scholarly impact.

  • Priem, J., Taborelli, D., Groth P, Nylon, C. (2010). Alt-metrics. A manifesto. http://altmetrics.org/manifesto/. Accessed 15 Jun 2014.

  • Roemer, R. C., & Borchardt, R. (2012). From bibliometrics to altmetrics: A changing scholarly landscape. College & Research Libraries News, 73(10), 596–600.

    Google Scholar 

  • Shema, H., Bar-Ilan, J., et al. (2015). How is research blogged? A content analysis approach. Journal of the Association for Information Science & Technology, 66(6), 1136–1149.

    Article  Google Scholar 

  • Sowe, S. K., Stamelos, I., et al. (2008). Understanding knowledge sharing activities in free/open source software projects: An empirical study. Journal of Systems and Software, 81(3), 431–446.

    Article  Google Scholar 

  • Tang, M. C., Wang, C. M., et al. (2012). Exploring alternative cyberbibliometrics for evaluation of scholarly performance in the social sciences and humanities in Taiwan. Proceedings of the American Society for Information Science & Technology, 49(1), 1.

    Article  Google Scholar 

  • Taylor, M. (2013). Exploring the boundaries: How altmetrics can expand our vision of scholarly communication and social impact. Information Standards Quarterly, 25(2), S213.

    Article  Google Scholar 

  • Thelwall, M., Buckley, K., et al. (2011). Sentiment in Twitter events. Journal of the American Society for Information Science and Technology, 62(2), 406–418.

    Article  Google Scholar 

  • Thelwall, M., Haustein, S., et al. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.

    Article  Google Scholar 

  • Rui, W., Wenjing, H., Wei, G. (2014). Research on academic influence of high Altmetrics sci-tech papers[J]. Library and Information Service, 58(21):92–98.

    Google Scholar 

  • Yuhong C. (2013). From Bibliometrics to Altmetrics: Research on Academic Influence Evaluation Based on Social Network. Information Studies: Theory & Application, 12:004.

    Google Scholar 

  • Wang, R., & Hu, W., et al. (2014). Research on academic influence of high altmetrics sci-tech papers. Library & Information Service.

  • Zahedi, Z., Costas, R., et al. (2014). How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics, 101(2), 1491–1513.

    Article  Google Scholar 

Download references

Acknowledgments

This paper is supported by National Social Science Foundation in China (Grant No.16BTQ055).

Author information

Authors and Affiliations

  1. Research Center for Chinese Science Evaluation, Wuhan University, Wuhan, 430072, China

    Rongying Zhao & Mingkun Wei

  2. School of Information Management, Wuhan University, Wuhan, 430072, China

    Rongying Zhao & Mingkun Wei

Authors
  1. Rongying Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingkun Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingkun Wei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, R., Wei, M. Impact evaluation of open source software: an Altmetrics perspective. Scientometrics 110, 1017–1033 (2017). https://doi.org/10.1007/s11192-016-2204-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-2204-y

Keywords

Search

Navigation