Skip to main content
SpringerLink
Log in

Online Indicators for Non-Standard Academic Outputs

  • Chapter
Book cover Springer Handbook of Science and Technology Indicators

Part of the book series: Springer Handbooks ((SHB))

Abstract

This chapter reviews webometric, altmetric, and other online indicators for the impact of nonstandard academic outputs, such as software, data, presentations, images, videos, blogs, and grey literature. Although the main outputs of academics are journal articles in science and the social sciences, and monographs, chapters, or edited books to some extent in the arts and humanities, many scholars also produce other primary research outputs. For nonstandard outputs, it is important to provide evidence to justify a claim for a type of impact and online indicators may help with this. Using the web, academics may obtain data to present as evidence for a specific impact claim. The research reviewed in this chapter describes the types of evidence that can be gathered, the nature of the claims that can be made, and methods to collect and process the raw data. The chapter concludes by discussing the limitations of online data and summarizing recommendations for interpreting impact evidence.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 299.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 379.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • T.C. Engels, T.L. Ossenblok, E.H. Spruyt: Changing publication patterns in the social sciences and humanities, Scientometrics 93(2), 373–390 (2012)

    Article  Google Scholar 

  • D. Torres-Salinas, N. Robinson-Garcia, Á. Cabezas-Clavijo, E. Jiménez-Contreras: Analyzing the citation characteristics of books: Edited books, book series and publisher types in the book citation index, Scientometrics 98(3), 2113–2127 (2014)

    Article  Google Scholar 

  • ACUMEN: Portfolio, http://research-acumen.eu/portfolio (2015)

  • ACUMEN: Portfolio, http://research-acumen.eu/wp-content/uploads/Blank_AcumenPortfolio.v13x.pdf (2015)

  • T. Greenhalgh, N. Fahy: Research impact in the community-based health sciences: An analysis of 162 case studies from the 2014 UK research excellence framework, BMC Med. 13(1), 232 (2015)

    Article  Google Scholar 

  • G.N. Samuel, G.E. Derrick: Societal impact evaluation: Exploring evaluator perceptions of the characterization of impact under the REF2014, Res. Evaluation 24(3), 229–241 (2015)

    Article  Google Scholar 

  • Research Excellence Framework: Impact case studies, http://impact.ref.ac.uk/CaseStudies (2014)

  • Research Excellence Framework: Impact case study “Rublack”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=23345 (2014)

  • Research Excellence Framework: Impact case study “Million-pound donors: Shaping policy and professional practice in philanthropy and fundraising from high net worth individuals”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=4607 (2014)

  • Research Excellence Framework: Impact case study “10 minute puzzle podcasts”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=43372 (2014)

  • Research Excellence Framework: Impact case study “Impact of machine-learning based visual analytics”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=37006 (2014)

  • Research Excellence Framework: Impact case study “Mrs. Peabody investigates: Enhancing public understanding of German, European and international crime fiction”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=5904 (2014)

  • Research Excellence Framework: Impact case study “Communicating research to the public through YouTube”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=32739 (2014)

  • Research Excellence Framework: Impact case study “Preventing disease through promotion of handwashing with soap”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=41463 (2014)

  • M. Thelwall: Weak benchmarking indicators for formative and semi-evaluative assessment of research, Res. Evaluation 13(1), 63–68 (2004)

    Article  Google Scholar 

  • L. Waltman, N.J. van Eck, T.N. van Leeuwen, M.S. Visser, A.F. van Raan: Towards a new crown indicator: Some theoretical considerations, J. Informetr. 5(1), 37–47 (2011)

    Article  Google Scholar 

  • L. Björneborn, P. Ingwersen: Toward a basic framework for webometrics, J. Am. Soc. Inf. Sci. Technol. 55(14), 1216–1227 (2004)

    Article  Google Scholar 

  • K.J. Holmberg: Altmetrics for Information Professionals: Past, Present and Future (Chandos, Oxford 2015)

    Google Scholar 

  • J. Priem, D. Taraborelli, P. Groth, C. Neylon: Altmetrics: A manifesto, http://altmetrics.org/manifesto/ (2010)

  • K. Kousha, M. Thelwall: Web indicators for research evaluation, part 3: Books and non-standard outputs, Profesion. Inf. 24(6), 724–736 (2015)

    Article  Google Scholar 

  • M. Thelwall, D. Stuart: Web crawling ethics revisited: Cost, privacy and denial of service, J. Am. Soc. Inf. Sci. Technol. 57(13), 1771–1779 (2006)

    Article  Google Scholar 

  • J. Bar-Ilan: Methods for measuring search engine performance over time, J. Am. Soc. Inf. Sci. Technol. 53(4), 308–319 (2002)

    Article  Google Scholar 

  • T. Greenhalgh, J. Raftery, S. Hanney, M. Glover: Research impact: A narrative review, BMC Med. 14(1), 78 (2016)

    Article  Google Scholar 

  • M. Thelwall, P. Sud: Webometric research with the Bing Search API 2.0, J. Inf. 6(1), 44–52 (2012)

    Google Scholar 

  • A. Uyar: Google stemming mechanisms, J. Inf. Sci. 35(5), 499–514 (2009)

    Article  Google Scholar 

  • D. Lewandowski, H. Wahlig, G. Meyer-Bautor: The freshness of web search engine databases, J. Inf. Sci. 32(2), 131–148 (2006)

    Article  Google Scholar 

  • L. Vaughan, M. Thelwall: Search engine coverage bias: Evidence and possible causes, Inf. Process. Manag. 40(4), 693–707 (2004)

    Article  Google Scholar 

  • M. Thelwall: Web Indicators for Research Evaluation: A Practical Guide (Morgan Claypool, San Rafael 2017)

    Google Scholar 

  • M. Thelwall: Are Mendeley reader counts high enough for research evaluations when articles are published?, Aslib J. Inf. Manag. 69(2), 174–183 (2017)

    Article  Google Scholar 

  • A.W. Harzing, S. Alakangas: Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison, Scientometrics 106(2), 787–804 (2016)

    Article  Google Scholar 

  • M. Thelwall: Microsoft Academic automatic document searches: Accuracy for journal articles and suitability for citation analysis, J. Inf. 12(1), 1–9 (2018)

    Google Scholar 

  • M. Thelwall: Does Microsoft Academic find early citations?, Scientometrics 114(1), 325–334 (2018), https://doi.org/10.1007/s11192-017-2558-9

    Article  Google Scholar 

  • M. Thelwall, K. Kousha: ResearchGate versus Google Scholar: Which finds more early citations?, Scientometrics 112(2), 1125–1131 (2017)

    Article  Google Scholar 

  • P. Wouters, R. Costas: Users, narcissism and control: Tracking the impact of scholarly publications in the 21st century. Utrecht: SURF foundation, http://www.surffoundation.nl/nl/publicaties/Documents/Users%20narcissism%20and%20control.pdf (2012)

  • G. Abramo, C.A. D'Angelo, F. Di Costa: National research assessment exercises: A comparison of peer review and bibliometrics rankings, Scientometrics 89(3), 929 (2011)

    Article  Google Scholar 

  • J. Wilsdon, L. Allen, E. Belfiore, P. Campbell, S. Curry, S. Hill, J. Hill: The metric tide: Report of the independent review of the role of metrics in research assessment and management, http://www.hefce.ac.uk/pubs/rereports/Year/2015/metrictide/Title,104463,en.html (2015)

  • E. Mohammadi, M. Thelwall: Assessing non-standard article impact using F1000 labels, Scientometrics 97(2), 383–395 (2013)

    Article  Google Scholar 

  • P. Sud, M. Thelwall: Evaluating altmetrics, Scientometrics 98(2), 1131–1143 (2014)

    Article  Google Scholar 

  • K.A. Neuendorf: The Content Analysis Guidebook (SAGE, Oxford 2016)

    Google Scholar 

  • M. Thelwall, A. Tsou, S. Weingart, K. Holmberg, S. Haustein: Tweeting links to academic articles, Cybermetrics 17(1) (2013), http://cybermetrics.cindoc.csic.es/articles/v17i1p1.html

  • A. Tsou, T. Bowman, A. Ghazinejad, C.R. Sugimoto: Who tweets about science? In: Proc. 15th Int. Soc. Sci. Inf. Conf. (ISSI2015), ed. by A.A. Salah, Y. Tonta, A.A.A. Salah, C. Sugimoto, U. Al (Boğaziçi Univ. Printhouse, Istanbul 2015) pp. 28–36

    Google Scholar 

  • O.J. Reichman, M.B. Jones, M.P. Schildhauer: Challenges and opportunities of open data in ecology, Science 331(6018), 703–705 (2011)

    Article  Google Scholar 

  • P. Vandewalle, J. Kovacevic, M. Vetterli: Reproducible research in signal processing, IEEE Signal Process. Mag. 26(3), 37–47 (2009)

    Article  Google Scholar 

  • D.S. Caetano, A. Aisenberg: Forgotten treasures: The fate of data in animal behaviour studies, Animal Behav 98(1), 1–5 (2014)

    Article  Google Scholar 

  • A. Dinsmore, L. Allen, K. Dolby: Alternative perspectives on impact: The potential of ALMs and altmetrics to inform funders about research impact, PLoS Biol. 12(11), e1002003 (2014)

    Article  Google Scholar 

  • J. Brase, I. Sens, M. Lautenschlager: The tenth anniversary of assigning DOI names to scientific data and a five year history of data cite, D-Lib Mag. (2015), https://doi.org/10.1045/january2015-brase

    Article  Google Scholar 

  • N. Robinson-Garcia, P. Mongeon, W. Jeng, R. Costas: Evaluating the possibilities of DataCite for developing ‘Open Data metrics' on the production and usage of datasets worldwide. In: 3AM Conf (2016), http://wdb.ugr.es/~elrobin/wp-content/uploads/3AM_summary_final.pdf

    Google Scholar 

  • C.L. Borgman: The conundrum of sharing research data, J. Am. Soc. Inf. Sci. Technol. 63(6), 1059–1078 (2012)

    Article  Google Scholar 

  • X. Huang, B.A. Hawkins, F. Lei, G.L. Miller, C. Favret, R. Zhang, G. Qiao: Willing or unwilling to share primary biodiversity data: Results and implications of an international survey, Conserv. Lett. 5(5), 399–406 (2012)

    Article  Google Scholar 

  • C. Tenopir, S. Allard, K. Douglass, A.U. Aydinoglu, L. Wu, E. Read, M. Frame: Data sharing by scientists: Practices and perceptions, PLoS One 6(6), e21101 (2011)

    Article  Google Scholar 

  • H.A. Piwowar, R.S. Day, D.B. Fridsma: Sharing detailed research data is associated with increased citation rate, PLoS One 2(3), e308 (2007)

    Article  Google Scholar 

  • Clarivate Analytics: The data citation index, http://wokinfo.com/products_tools/multidisciplinary/dci/ (2015)

  • L.L. Pavlech: Data citation index, J. Med. Libr. Assoc. 104(1), 88–90 (2016)

    Article  Google Scholar 

  • D. Torres-Salinas, A. Martín-Martín, E. Fuente-Gutiérrez: An introduction to the coverage of the Data Citation Index (Thomson-Reuters): Disciplines, document types and repositories, Rev. Esp. Doc. Cient. 37(1), e036 (2014)

    Article  Google Scholar 

  • D. Torres-Salinas, E. Jiménez-Contreras, N. Robinson-García: How many citations are there in the Data Citation Index? Proceedings of the STI Conference, Leiden, The Netherlands. Online at http://arxiv.org/abs/1409.0753. Accessed on 1 January 2015

  • I. Peters, P. Kraker, E. Lex, C. Gumpenberger, J. Gorraiz: Research data explored: An extended analysis of citations and altmetrics, Scientometrics 107(2), 723–744 (2016)

    Article  Google Scholar 

  • N. Robinson-García, E. Jiménez-Contreras, D. Torres-Salinas: Analyzing data citation practices using the data citation index, J. Assoc. Inf. Sci. Technol. 67(12), 2964–2975 (2016)

    Article  Google Scholar 

  • P. Anagnostou, M. Capocasa, N. Milia, G.D. Bisol: Research data sharing: Lessons from forensic genetics, Forensic Sci. Int. Genet. 7(6), e117–e119 (2013)

    Article  Google Scholar 

  • L. He, V. Nahar: Reuse of scientific data in academic publications: An investigation of Dryad Digital Repository, Aslib J. Inf. Manag. 68(4), 478–494 (2016)

    Article  Google Scholar 

  • M. Thelwall, K. Kousha: Do journal data sharing mandates work? Life sciences evidence from dryad, Aslib J. Inf. Manag. 69(1), 36–45 (2016)

    Article  Google Scholar 

  • M. Thelwall, K. Kousha: Figshare: A universal repository for academic resource sharing?, Online Inf. Rev. 40(3), 333–346 (2016)

    Article  Google Scholar 

  • J.C. Wallis, E. Rolando, C.L. Borgman: If we share data, will anyone use them? Data sharing and reuse in the long tail of science and technology, PLoS One 8(7), e67332 (2013)

    Article  Google Scholar 

  • P. Ingwersen, V. Chavan: Indicators for the data usage index (DUI): An incentive for publishing primary biodiversity data through global information infrastructure, BMC Bioinform 12(Suppl 15), S3 (2011)

    Article  Google Scholar 

  • K.R. Lakhani, E.V. Hippel: How open source software works: “Free” user-to-user assistance, Res. Policy 32(6), 923–943 (2003)

    Article  Google Scholar 

  • A. Mockus, R.T. Fielding, J.D. Herbsleb: Two case studies of open source software development: Apache and mozilla, ACM Trans. Softw. Eng. Methodol. 11(3), 309–346 (2002)

    Article  Google Scholar 

  • C. Calenge: The package adehabitat for the R software: A tool for the analysis of space and habitat use by animals, Ecol. Model. 197(3), 516–519 (2006)

    Article  Google Scholar 

  • M. Thelwall, K. Kousha: Academic software downloads from Google Code: Useful usage indicators?, Inf. Res. 21(1), 709 (2016)

    Google Scholar 

  • P.Y.H. Lee: Inverting the logic of scientific discovery: Applying common law patentable subject matter doctrine to constrain patents on biotechnology research tools, Harvard J. Law Technol. 19(1), 79–109 (2005)

    Google Scholar 

  • Research England: Research Excellence Framework, https://www.ref.ac.uk/guidance/

  • M. Meyer: Academic patents as an indicator of useful research? A new approach to measure academic inventiveness, Res. Evaluation 12(1), 17–27 (2003)

    Article  Google Scholar 

  • S. Xu, X. Wang, Z. Liu, C. Luan: Network structural analysis of technology: A study from patent perspective, J. Sci. Technol. Policy China 4(3), 214–235 (2013)

    Article  Google Scholar 

  • Z. Qu, X. Shen, K. Ding: Comparative analysis on technologies between Chinese and American large-sized oil companies based on patentometrics, J. Scientometr. Res. 3(2), 68–74 (2014)

    Article  Google Scholar 

  • S.B. Johnson: On-line pesticide training with narrated PowerPoint presentations, J. Ext. 53(3) (2015), http://www.joe.org/joe/2015june/tt8.php

  • C. Lisée, V. Larivière, É. Archambault: Conference proceedings as a source of scientific information: A bibliometric analysis, J. Am. Soc. Inf. Sci. Technol. 59(11), 1779–1784 (2008)

    Article  Google Scholar 

  • M. Oppermann: Tourism conferences—academic titillation, social interactions or job market?, Tour. Manag. 18(5), 255–257 (1997)

    Article  Google Scholar 

  • M. Petticrew, M. Egan, H. Thomson, V. Hamilton, R. Kunkler, H. Roberts: Publication bias in qualitative research: What becomes of qualitative research presented at conferences?, J. Epidemiol. Community Health 62(6), 552–554 (2008)

    Article  Google Scholar 

  • D. Schoeneborn: The pervasive power of Power Point: How a genre of professional communication permeates organizational communication, Organ. Stud. 34(12), 1777–1801 (2015)

    Article  Google Scholar 

  • P. Eveillard: Social networks in 10 lessons: No. 3 entrust your presentations with slide share!, Rev. Prat. 63(3), 384 (2013)

    Google Scholar 

  • M. Thelwall, K. Kousha: Online presentations as a source of scientific impact? An analysis of power point files citing academic journals, J. Am. Soc. Inf. Sci. Technol. 59(5), 805–815 (2008)

    Article  Google Scholar 

  • M. Thelwall, K. Kousha: SlideShare presentations, citations, users and trends: A professional site with academic and educational uses, J. Assoc. Inf. Sci. Technol. 68(8), 1989–2003 (2017)

    Article  Google Scholar 

  • K. Kousha, K. Thelwall, M. Abdoli: The role of online videos in research communication: A content analysis of YouTube videos cited in academic publications, J. Am. Soc. Inf. Sci. Technol. 63(9), 1710–1727 (2012)

    Article  Google Scholar 

  • B. Haran, M. Poliakoff: The periodic table of videos, Science 332(6033), 1046–1047 (2011)

    Article  Google Scholar 

  • R. Hovden: Bibliometrics for internet media: Applying the h-index to YouTube, J. Am. Soc. Inf. Sci. Technol. 64(11), 2326–2331 (2013)

    Article  Google Scholar 

  • C. Xiao, Y. Xue, Z. Li, X. Luo, Z. Qin: Measuring user influence based on multiple metrics on YouTube. In: 7th Int. Symp. Parallel Archit. Algorithms Program. (PAAP2015) (IEEE, Menlo Park 2015) pp. 177–182

    Google Scholar 

  • C.R. Sugimoto, M. Thelwall: Scholars on soap boxes: Science communication and dissemination in TED videos, J. Am. Soc. Inf. Sci. Technol. 64(4), 663–674 (2013)

    Article  Google Scholar 

  • H.F. Chan, B.S. Frey, J. Gallus, M. Schaffner, B. Torgler, S. Whyte: Do the best scholars attract the highest speaking fees? An exploration of internal and external influence, Scientometrics 101(1), 793–817 (2014)

    Article  Google Scholar 

  • C.R. Sugimoto, M. Thelwall, V. Larivière, A. Tsou, P. Mongeon, B. Macaluso: Scientists popularizing science: Characteristics and impact of TED talk presenters, PLoS One 8(4), e62403 (2013)

    Article  Google Scholar 

  • A. Tsou, M. Thelwall, P. Mongeon, C. Sugimoto: A community of curious souls: An analysis of commenting behavior on TED talks videos, PLoS One 9(4), e93609 (2014)

    Article  Google Scholar 

  • The New England Journal of Medicine: Author center, images in clinical medicine, https://www.nejm.org/author-center/images-in-clinical-medicine (2018)

  • K. Schröder, H. Lüthen: Astrophotography. In: Handbook of Practical Astronomy, ed. by G.D. Roth (Springer, Berlin 2009) pp. 133–173

    Chapter  Google Scholar 

  • E. Angus, M. Thelwall, D. Stuart: Flickr's potential as an academic image resource: An exploratory study, J. Librariansh. Inf. Sci. 42(4), 268–278 (2010)

    Article  Google Scholar 

  • I. Kirton, M. Terras: Digitisation and dissemination: A reverse image lookup study to assess the reuse of images of paintings from the National Gallery's website, J. Digit. Humanit. 3(1) (2014), http://journalofdigitalhumanities.org/3-1/reverse-image-lookup-paintings-digitisation-reuse/

  • K. Kousha, M. Thelwall, S. Rezaie: Can the impact of scholarly images be assessed online? An exploratory study using image identification technology, J. Am. Soc. Inf. Sci. Technol. 61(9), 1734–1744 (2010)

    Article  Google Scholar 

  • M. Reilly, S. Thompson: Reverse image lookup: Assessing digital library users and reuses, J. Web Librariansh. 11(1), 56–68 (2017)

    Article  Google Scholar 

  • S. Riley, L. Hunter (Eds.): Mapping Landscapes for Performance as Research: Scholarly Acts and Creative Cartographies (Springer, Berlin 2009)

    Google Scholar 

  • C.D. Brown: Straddling the humanities and social sciences: The research process of music scholars, Libr. Inf. Sci. Res. 24(1), 73–94 (2002)

    Article  Google Scholar 

  • Research Excellence Framework: Output details “Reflecting on environmental change through site-based performance”, http://results.ref.ac.uk/Submissions/Output/2863017?searchId=437searchPageNumber=1 (2014)

  • Research Excellence Framework: Output details ‘Devotion by design: Italian Altarpieces before 1500 (National gallery company)', http://results.ref.ac.uk/Submissions/Output/74860?searchId=438searchPageNumber=1 (2014)

  • M. Thelwall, M. Delgado: Arts and humanities research evaluation: No metrics please, just data, J. Doc. 71(4), 817–833 (2015)

    Article  Google Scholar 

  • AHRC: Understanding your project: A guide to self-evaluation, http://www.ahrc.ac.uk/documents/guides/understanding-your-project-a-guide-to-self-evaluation/ (2017)

  • E. Belfiore, O. Bennett: Beyond the “Toolkit Approach”: Arts impact evaluation research and the realities of cultural policy-making, J. Cult. Res. 14(2), 121–142 (2010)

    Article  Google Scholar 

  • P. Ingwersen: The calculation of web impact factors, J. Doc. 54(2), 236–243 (1998)

    Article  Google Scholar 

  • A.G. Smith: A tale of two web spaces: Comparing sites using web impact factors, J. Doc. 55(5), 577–592 (1999)

    Google Scholar 

  • J. Bar-Ilan: A microscopic link analysis of academic institutions within a country—the case of Israel, Scientometrics 59(3), 391–403 (2004)

    Article  Google Scholar 

  • J. Bar-Ilan: What do we know about links and linking? A framework for studying links in academic environments, Inf. Process. Manag. 41(4), 973–986 (2005)

    Article  Google Scholar 

  • D. Wilkinson, G. Harries, M. Thelwall, L. Price: Motivations for academic Web site interlinking: Evidence for the Web as a novel source of information on informal scholarly communication, J. Inf. Sci. 29(1), 49–56 (2003)

    Article  Google Scholar 

  • M. Thelwall, G. Harries: Do better scholars' web publications have significantly higher online impact?, J. Am. Soc. Inf. Sci. Technol. 55(2), 149–159 (2004)

    Article  Google Scholar 

  • M. Thelwall: Interpreting social science link analysis research: A theoretical framework, J. Am. Soc. Inf. Sci. Technol. 57(1), 60–68 (2006)

    Article  Google Scholar 

  • L. Vaughan, R. Yang: Web data as academic and business quality estimates: A comparison of three data sources, J. Am. Soc. Inf. Sci. Technol. 63(10), 1960–1972 (2012)

    Article  Google Scholar 

  • K. Kousha, M. Thelwall: Motivations for URL citations to open access library and information science articles, Scientometrics 68(3), 501–517 (2006)

    Article  Google Scholar 

  • D. Stuart, M. Thelwall: Investigating triple helix relationships using URL citations: A case study of the UK West Midlands automobile industry, Res. Evaluation 15(2), 97–106 (2006)

    Article  Google Scholar 

  • D. Wilkinson, M. Thelwall: Search markets and search results: The case of Bing, Libr. Inf. Sci. Res. 35(4), 318–325 (2013)

    Article  Google Scholar 

  • M. Thelwall: Extracting accurate and complete results from search engines: Case study windows live, J. Am. Soc. Inf. Sci. Technol. 59(1), 38–50 (2008)

    Article  Google Scholar 

  • P. Sud, M. Thelwall: Linked title mentions: A new automated link search candidate, Scientometrics 101(3), 1831–1849 (2014)

    Article  Google Scholar 

  • F. Barjak, M. Thelwall: A statistical analysis of the web presences of European life sciences research teams, J. Am. Soc. Inf. Sci. Technol. 59(4), 628–643 (2008)

    Article  Google Scholar 

  • F. Barjak, X. Li, M. Thelwall: Which factors explain the web impact of scientists personal homepages?, J. Am. Soc. Inf. Sci. Technol. 58(2), 200–211 (2007)

    Article  Google Scholar 

  • A. Mas-Bleda, M. Thelwall, K. Kousha, I.F. Aguillo: Do highly cited researchers successfully use the social web?, Scientometrics 101(1), 337–356 (2014)

    Article  Google Scholar 

  • A. Mas-Bleda, M. Thelwall, K. Kousha, I. Aguillo: Successful researchers publicizing research online: An outlink analysis of European highly cited scientists personal websites, J. Doc. 70(1), 148–172 (2014)

    Article  Google Scholar 

  • S. Meece, A. Robinson, M.T. Gramstadt: Engaging researchers with the world's first scholarly arts repositories: Ten years after the UK's Kultur project, New Rev. Acad. Librariansh. 23(2/3), 209–232 (2017)

    Article  Google Scholar 

  • M.S. Mayernik, D.L. Hart, K.E. Maull, N.M. Weber: Assessing and tracing the outcomes and impact of research infrastructures, J. Assoc. Inf. Sci. Technol. 68(6), 1341–1359 (2017)

    Article  Google Scholar 

  • K.E. Eccles, M. Thelwall, E.T. Meyer: Measuring the web impact of digitised scholarly resources, J. Doc. 68(4), 512–526 (2012)

    Article  Google Scholar 

  • C. Warwick, M. Terras, P. Huntington, N. Pappa: If you build it will they come? The LAIRAH study: Quantifying the use of online resources in the arts and humanities through statistical analysis of user log data, Lit. Linguist. Comput. 23(1), 85–102 (2008)

    Article  Google Scholar 

  • A. Zuccala, M. Thelwall, C. Oppenheim, R. Dhiensa: Web intelligence analyses of digital libraries: A case study of the national electronic library for health (NeLH), J. Doc. 63(4), 558–589 (2007)

    Article  Google Scholar 

  • D. Duin, D. King, P. Van Den Besselaar: Identifying audiences of e-infrastructures-tools for measuring impact, PLoS One 7(12), e50943 (2012)

    Article  Google Scholar 

  • K. Jonkers, G.E. Derrick, C. Lopez-Illescas, P. Van den Besselaar: Measuring the scientific impact of e-research infrastructures: A citation based approach?, Scientometrics 101(2), 1179–1194 (2014)

    Article  Google Scholar 

  • The London School of Economics and Political Science: LSE impact blog, http://blogs.lse.ac.uk/impactofsocialsciences/

  • DC's Improbable Science: Why you should ignore altmetrics and other bibliometric nightmares, http://www.dcscience.net/2014/01/16/why-you-should-ignore-altmetrics-and-other-bibliometric-nightmares/ (2014)

  • H. Shema, J. Bar-Ilan, M. Thelwall: Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics, J. Assoc. Inf. Sci. Technol. 65(5), 1018–1027 (2014)

    Article  Google Scholar 

  • H.R. Jamali, D. Alimohammadi: Blog citations as indicators of the societal impact of research: Content analysis of social sciences blogs, Int. J. Knowl. Content Dev. Technol. 5(1), 15–32 (2015)

    Article  Google Scholar 

  • S. Kjellberg: Researchers' blogging practices in two epistemic cultures: The scholarly blog as a situated genre, Hum. IT 12(3), 36–77 (2014)

    Google Scholar 

  • H. Shema, J. Bar-Ilan, M. Thelwall: Research Blogs and the discussion of scholarly information, PLoS One 7(5), e35869 (2012)

    Article  Google Scholar 

  • H. Shema, J. Bar-Ilan, M. Thelwall: How is research blogged? A content analysis approach, J. Assoc. Inf. Sci. Technol. 66(6), 1136–1149 (2015)

    Article  Google Scholar 

  • J.K. Hoang, J. McCall, A.F. Dixon, R.T. Fitzgerald, F. Gaillard: Using social media to share your radiology research: How effective is a blog post?, J. Am. Coll. Radiol. 12(7), 760–765 (2015)

    Article  Google Scholar 

  • N. Hookway, H. Snee: The Blogosphere. In: The SAGE Handbook of Online Research Methods, ed. by N.G. Fielding, R.M. Lee, G. Blank (SAGE, Oxford 2017) pp. 380–398

    Chapter  Google Scholar 

  • A.T. Weinstein, D.A. McFarlane: Case study–how a business school blog can build stakeholder relationships and create added value in an MBA marketing program, J. Strateg. Mark. 25(2), 1–13 (2017)

    Article  Google Scholar 

  • Research Excellence Framework: Impact case study “China economic policy”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=40126 (2014)

  • Research Excellence Framework: Impact case study “Stimulating public engagement with physics through the LHC and discovery of the Higgs boson”, http://impact.ref.ac.uk/CaseStudies/CaseStudy.aspx?Id=34170 (2014)

  • V. Alberani, P.D.C. Pietrangeli, A.M. Mazza: The use of grey literature in health sciences: A preliminary survey, Bull. Med. Libr. Assoc. 78(4), 358–363 (1990)

    Google Scholar 

  • E. Orduna-Malea, A. Martin-Martin, E.D. Lopez-Cozar: Google Scholar and the gray literature: A reply to Bonato's review, ArXiv Preprint ArXiv:1702.03991

    Google Scholar 

  • N.R. Haddaway, A.M. Collins, D. Coughlin, S. Kirk: The role of Google Scholar in evidence reviews and its applicability to grey literature searching, PLoS One 10(9), e0138237 (2015)

    Article  Google Scholar 

  • S.E. Hug, M. Ochsner, M.P. Brändle: Citation analysis with Microsoft Academic, Scientometrics 111(1), 371–378 (2017)

    Article  Google Scholar 

  • N.L. Pelzer, W.H. Wiese: Bibliometric study of grey literature in core veterinary medical journals, J. Med. Libr. Assoc. 91(4), 434 (2003)

    Google Scholar 

  • C. Talbot, C. Talbot: Sir Humphrey and the Professors: What Does Whitehall Want from Academics? (Univ. Manchester, Manchester 2014)

    Google Scholar 

  • R. Cordes: Is grey literature ever used? Using citation analysis to measure the impact of GESAMP, an international marine scientific advisory body, Can. J. Inf. Libr. Sci. 28(1), 49–69 (2013)

    Google Scholar 

  • Q. Mahood, D.V. Eerd, E. Irvin: Searching for grey literature for systematic reviews: Challenges and benefits, Res. Synt. Methods 5(3), 221–234 (2014)

    Article  Google Scholar 

  • A.M. Kalamar, S. Lee-Rife, M.J. Hindin: Interventions to prevent child marriage among young people in low-and middle-income countries: A systematic review of the published and gray literature, J. Adolesc. Health 59(3), S16–S21 (2016)

    Article  Google Scholar 

  • G. McKiernan: ArXiv.org: The Los Alamos National Laboratory e-print server, Int. J. Grey Lit. 1(3), 127–138 (2000)

    Article  Google Scholar 

  • X. Li, M. Thelwall, K. Kousha: The role of arXiv, RePEc, SSRN and PMC in formal scholarly communication, Aslib J. Inf. Manag. 67(6), 614–635 (2015)

    Article  Google Scholar 

  • C. Seiler, K. Wohlrabe: Ranking economists on the basis of many indicators: An alternative approach using RePEc data, J. Informetr. 6(3), 389–402 (2012)

    Article  Google Scholar 

  • C. Zimmermann: Academic rankings with RePEc, Econometrics 1(3), 249–280 (2013)

    Article  Google Scholar 

  • Z. Kakushadze: An index for SSRN downloads, J. Informetr. 10(1), 9–28 (2016)

    Article  Google Scholar 

  • D. Wilkinson, P. Sud, M. Thelwall: Substance without citation: Evaluating the online impact of grey literature, Scientometrics 98(2), 797–806 (2014)

    Article  Google Scholar 

  • S.K. Panda: Shodhganga–a national level open access ETD repository of Indian electronic theses: Current status and discussions, Libr. Hi Tech News 33(1), 23–26 (2016)

    Article  Google Scholar 

  • G. Marchionini, P. Solomon, C. Davis, T. Russell: Information and library science MPACT: A preliminary analysis, Libr. Inf. Sci. Res. 28(4), 480–500 (2007)

    Article  Google Scholar 

  • C.R. Sugimoto, T.G. Russell, L.I. Meho, G. Marchionini: MPACT and citation impact: Two sides of the same scholarly coin?, Libr. Inf. Sci. Res. 30(4), 273–281 (2008)

    Article  Google Scholar 

  • J. Kayongo, C. Helm: Relevance of library collections for graduate student research: A citation analysis study of doctoral dissertations at Notre Dame, Coll. Res. Libr. 73(1), 47–67 (2011)

    Article  Google Scholar 

  • Y. Tonta, U. Al: Scatter and obsolescence of journals cited in theses and dissertations of librarianship, Libr. Inf. Sci. Res. 28(2), 281–296 (2006)

    Article  Google Scholar 

  • M. Kettler: Ways of disseminating, tracking usage and impact of electronic theses and dissertations (ETDs), Grey J. TGJ 13(1), 27–31 (2017)

    Google Scholar 

  • T. Dewett, A. Denisi: Exploring scholarly reputation: It's more than just productivity, Scientometrics 60(2), 249–272 (2004)

    Article  Google Scholar 

  • S. Jeong, S. Lee, H.G. Kim: Are you an invited speaker? A bibliometric analysis of elite groups for scholarly events in bioinformatics, J. Am. Soc. Inf. Sci. Technol. 60(6), 1118–1131 (2009)

    Article  Google Scholar 

  • J. Yoo, S. Choi, M. Choi, J. Rho: Why people use Twitter: Social conformity and social value perspectives, Online Inf. Rev. 38(2), 265–283 (2014)

    Article  Google Scholar 

  • M.A. Camilleri: Utilising content marketing metrics and social networks for academic visibility: Content marketing for academic impact. In: Digital Tools for Academic Branding and Self-Promotion, ed. by M. Cabrera, N. Lloret (IGI Global, Hershey 2016) pp. 109–126

    Google Scholar 

  • M. Thelwall, K. Kousha: ResearchGate articles: Age, discipline, audience size and impact, J. Assoc. Inf. Sci. Technol. 68(2), 468–479 (2017)

    Article  Google Scholar 

  • N. Busom, R. Petrlic, F. Sebé, C. Sorge, M. Valls: A privacy-preserving reputation system with user rewards, J. Netw. Comput. Appl. 80(1), 58–66 (2017)

    Article  Google Scholar 

  • K. Jordan: Understanding the structure and role of academics ego-networks on social networking sites (The Open Univ, Milton Keynes 2017), Ph.D. Thesis

    Google Scholar 

  • E. Orduna-Malea, A. Martín-Martín, M. Thelwall, E.D. López-Cózar: Do ResearchGate scores create ghost academic reputations?, Scientometrics 112(1), 443–460 (2017)

    Article  Google Scholar 

  • S. Copiello, P. Bonifaci: A few remarks on ResearchGate score and academic reputation, Scientometrics (2018), https://doi.org/10.1007/s11192-017-2582-9

    Article  Google Scholar 

  • J. Poza-Lugan, A. Calduch-Losa: Social media as a tool, and tools offered by to social media to teachers and researchers: Personal or professional use? In: Digital Tools for Academic Branding and Self-Promotion, ed. by M. Cabrera, N. Lloret (IGI Global, Hershey 2016) pp. 66–80

    Google Scholar 

  • M. Thelwall, K. Kousha: Academia.edu: Social network or academic network?, J. Assoc. Inf. Sci. Technol. 65(4), 721–731 (2014)

    Article  Google Scholar 

  • M. Thelwall, K. Kousha: ResearchGate: Disseminating, communicating and measuring scholarship?, J. Assoc. Inf. Sci. Technol. 66(5), 876–889 (2015)

    Article  Google Scholar 

  • F. Riquelme, P. González-Cantergiani: Measuring user influence on Twitter: A survey, Inf. Process. Manag. 52(5), 949–975 (2016)

    Article  Google Scholar 

  • M. Bouguessa, L.B. Romdhane: Identifying authorities in online communities, ACM Trans. Intell. Syst. Technol. (TIST) 6(3), 30 (2015)

    Google Scholar 

  • B. Cronin, H.W. Snyder, H. Rosenbaum, A. Martinson, E. Callahan: Invoked on the web, J. Am. Soc. Inf. Sci. 49(14), 1319–1328 (1998)

    Article  Google Scholar 

  • C. Donovan, L. Butler: Testing novel quantitative indicators of research ‘quality', esteem and ‘user engagement': An economics pilot study, Res. Evaluation 16(4), 231–242 (2007)

    Article  Google Scholar 

  • M.F. Fox: Women and men faculty in academic science and engineering: Social-organizational indicators and implications, Am. Behav. Sci. 53(7), 997–1012 (2010)

    Article  Google Scholar 

  • K.E. Smith, E. Stewart: We need to talk about impact: Why social policy academics need to engage with the UK's research impact agenda, J. Soc. Policy 46(1), 109–127 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Science and Engineering, University of Wolverhampton, Wolverhampton, UK

    Mike Thelwall

Authors
  1. Mike Thelwall
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mike Thelwall .

Editor information

Editors and Affiliations

  1. ECOOM and Faculty of Economics and Business, KU Leuven, Leuven, Belgium

    Wolfgang Glänzel

  2. Amsterdam, The Netherlands

    Henk F. Moed

  3. Competence Center Policy – Industry – Innovation, Fraunhofer Institute for Systems and Innovation Research ISI, Karlsruhe, Germany

    Ulrich Schmoch

  4. Faculty of Science and Engineering, University of Wolverhampton, Wolverhampton, UK

    Mike Thelwall

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer International Publishing AG, part of Springer Nature

About this chapter

Cite this chapter

Thelwall, M. (2019). Online Indicators for Non-Standard Academic Outputs. In: Glänzel, W., Moed, H.F., Schmoch, U., Thelwall, M. (eds) Springer Handbook of Science and Technology Indicators. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-030-02511-3_33

Download citation

Publish with us

Policies and ethics

Search

Navigation