Skip to main content
SpringerLink
Log in
Book cover

Springer Handbook of Science and Technology Indicators pp 609–632Cite as

Societal Impact Measurement of Research Papers

  • Chapter

Part of the book series: Springer Handbooks ((SHB))

Abstract

What are the results of public investment in research from which society actually derives a benefit? The scope of research evaluations becomes broader when societal products (outputs), societal use (societal references), and societal benefits (changes in society) of research are considered. This chapter presents an overview of the literature in the area of societal impact measurement of scientific papers. It describes major research projects on societal impact measurements. Problems of societal impact assessments are discussed as well as proposals to measure societal impact. The chapter discusses the role of alternative metrics (altmetrics) in measuring societal impact. There is an ongoing debate in scientometrics as to whether altmetrics are able to measure this kind of impact.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   299.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   379.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • B. Godin, C. Dore: Measuring the Impacts of Science: Beyond the Economic Dimension, HIST Lecture Helsinki Institute for Science and Technology Studies, Helsinki, Finland, http://www.csiic.ca/PDF/Godin_Dore_Impacts.pdf (2005)

  • L. Bornmann: Scientific peer review, Annu. Rev. Inf. Sci. Technol. 45, 199–245 (2011)

    Article  Google Scholar 

  • H. Khazragui, J. Hudson: Measuring the benefits of university research: Impact and the REF in the UK, Res. Eval. 24(1), 51–62 (2015)

    Article  Google Scholar 

  • S.P.L. de Jong, P. van Arensbergen, F. Daemen, B. van der Meulen, P. van den Besselaar: Evaluation of research in context: An approach and two cases, Res. Eval. 20(1), 61–72 (2011)

    Article  Google Scholar 

  • S. Mostert, S. Ellenbroek, I. Meijer, G. van Ark, E. Klasen: Societal output and use of research performed by health research groups, Health Res. Policy Syst. 8(1), 30 (2010)

    Article  Google Scholar 

  • M. Power: The Audit Society: Rituals of Verification (Oxford Univ. Press, Oxford 1999)

    Book  Google Scholar 

  • G. Gruening: Origin and theoretical basis of new public management, Int. Public Manag. J. 4(1), 1–25 (2001)

    Article  Google Scholar 

  • J.C. Petit: Why do we need fundamental research?, Euro. Rev. 12(2), 191–207 (2004)

    Article  Google Scholar 

  • S. Hanney, T. Packwood, M. Buxton: Evaluating the benefits from health research and development centres: A categorization, a model and examples of application, Evaluation 6(2), 137–160 (2000)

    Article  Google Scholar 

  • M. Jubb: The scholarly ecosystem. In: Academic Professional Publishing, ed. by R. Campbell, E. Pentz, I. Borthwick (Chandos, Oxford 2012) pp. 53–77

    Chapter  Google Scholar 

  • A.J. Milat, A.E. Bauman, S. Redman: A narrative review of research impact assessment models and methods, Health Res. Policy Syst. (2015), https://doi.org/10.1186/s12961-015-0003-1

    Article  Google Scholar 

  • G. Cohen, J. Schroeder, R. Newson, L. King, L. Rychetnik, A.J. Milat, S. Chapman: Does health intervention research have real world policy and practice impacts: Testing a new impact assessment tool, Health Res. Policy Syst. 13, 12 (2015)

    Article  Google Scholar 

  • C. Donovan: The Australian research quality framework: A live experiment in capturing the social, economic, environmental, and cultural returns of publicly funded research, New Dir. Eval. 2008(118), 47–60 (2008)

    Article  Google Scholar 

  • J. Grant: Evaluating the outcomes of biomedical research on healthcare, Res. Eval. 8(1), 33–38 (1999)

    Article  Google Scholar 

  • L. Bornmann: Mimicry in science?, Scientometrics 86(1), 173–177 (2011)

    Article  Google Scholar 

  • J.B. Spaapen, H. Dijstelbloem, F. Wamelink: Evaluating Research in Context: A Method for Comprehensive Assessment (Consultative Committee of Sector Councils for Research and Development, The Hague 2007)

    Google Scholar 

  • R. Barré: S and T indicators for policy making in a changing science-society relationship. In: Handbook of Quantitative Science and Technology Research, ed. by H. Moed, W. Glänzel, U. Schmoch (Springer, Dordrecht 2005) pp. 115–131

    Chapter  Google Scholar 

  • K. Lähteenmäki-Smith, K. Hyytinen, P. Kutinlahti, J. Konttinen: Research with an Impact Evaluation Practises in Public Research Organisations (VTT Technical Research Centre of Finland, Kemistintie 2006)

    Google Scholar 

  • P. Dahler-Larsen: The Evaluation Society (Stanford Univ. Press, Stanford 2011)

    Book  Google Scholar 

  • G.M. Lamm: Innovation Works™. A case study of an integrated pan-European technology transfer model, B.I.F. Futura 21(2), 86–90 (2006)

    Google Scholar 

  • P. Nightingale, A. Scott: Peer review and the relevance gap: Ten suggestions for policy-makers, Sci. Public Policy 34(8), 543–553 (2007)

    Article  Google Scholar 

  • J. Ziman: Why must scientists become more ethically sensitive than they used to be?, Science 282(5395), 1813–1814 (1998)

    Article  Google Scholar 

  • E. Ernø-Kjølhede, F. Hansson: Measuring research performance during a changing relationship between science and society, Res. Eval. 20(2), 131–143 (2011)

    Article  Google Scholar 

  • J. Cohen: A hard look at global health measures, Science 345(6202), 1260–1265 (2014), https://doi.org/10.1126/science.345.6202.1260

    Article  Google Scholar 

  • Toronto Region Research Alliance: The Race for Global Leadership in Innovation: An Analysis of National R and D Strategies (TRRA, Toronto 2001)

    Google Scholar 

  • M. Cleary, N. Siegfried, D. Jackson, G.E. Hunt: Making a difference with research: Measuring the impact of mental health research, Int. J. Ment. Health Nurs. 22(2), 103–105 (2013)

    Article  Google Scholar 

  • J. Molas-Gallart, A. Salter, P. Patel, A. Scott, X. Duran: Measuring Third Stream Activities. Final Report to the Russell Group of Universities (SPRU, Brighton 2002)

    Google Scholar 

  • I. Peters, A. Jobmann, A. Eppelin, C.P. Hoffmann, S. Künne: Altmetrics for Large, Multidisciplinary Research Groups: A Case Study of the Leibniz Association, https://www.alexandria.unisg.ch/Publikationen/nach-neuen-Publikationen/231796 (2014)

  • ERiC: Evaluating the Societal Relevance of Academic Research: A Guide (Delft Univ. Technology, Delft 2010)

    Google Scholar 

  • B. Göransson, R. Maharajh, U. Schmoch: New activities of universities in transfer and extension: Multiple requirements and manifold solutions, Sci. Public Policy 36(2), 157–164 (2009)

    Article  Google Scholar 

  • J. Wilsdon, L. Allen, E. Belfiore, P. Campbell, S. Curry, S. Hill, B. Johnson: The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management (HEFCE, Bristol 2015)

    Book  Google Scholar 

  • A. Dance: Impact: Pack a punch, Nature 502, 397–398 (2013)

    Article  Google Scholar 

  • G.N. Samuel, G.E. Derrick: Societal impact evaluation: Exploring evaluator perceptions of the characterization of impact under the REF2014, Res. Eval. 24(3), 229–241 (2015)

    Article  Google Scholar 

  • J. Grimson: Measuring research impact: Not everything that can be counted counts, and not everything that counts can be counted. In: Bibliometrics: Use and Abuse in the Review of Research Performance, ed. by W. Blockmans, L. Engwall, D. Weaire (Portland, London 2014) pp. 29–41

    Google Scholar 

  • T. Penfield, M.J. Baker, R. Scoble, M.C. Wykes: Assessment, evaluations, and definitions of research impact: A review, Res. Eval. 23(1), 21–32 (2014), https://doi.org/10.1093/reseval/rvt021

    Article  Google Scholar 

  • Advisory Group: Research Quality Framework: Assessing the Quality and Impact of Research in Australia. Research Impact (Department of Education, Canberra 2006)

    Google Scholar 

  • Committee for Scientific and Technology Policy: Assessing the Impact of State Interventions in Research—Techniques, Issues and Solutions (Directorate for Science, Technology and Innovation, Brussels 2014)

    Google Scholar 

  • R. Ruegg, I. Feller: A Toolkit for Evaluating Public R and D Investment: Models, Methods, and Findings from ATP's First Decade (National Institute of Standards and Technology, Gaithersburg 2003)

    Google Scholar 

  • B.R. Martin: Assessing the impact of basic research on society and the economy. In: Rethink. Impact Basic Res. Soc. Economy, Vienna (2007)

    Google Scholar 

  • SISOP: Conceptualizing the Social Impact of Science, http://sisobproject.wordpress.com/2011/07/20/conceptualizing-the-social-impact-of-science-2/ (2011)

  • A.J. Salter, B.R. Martin: The economic benefits of publicly funded basic research: A critical review, Res. Policy 30(3), 509–532 (2001)

    Article  Google Scholar 

  • Higher Education Funding Council for England: Research Excellence Framework. Second Consultation on the Assessment and Funding of Research (HEFCE, Bristol 2009)

    Google Scholar 

  • L. Langfeldt, L. Scordato: Assessing the Broader Impacts of Research: A Review of Methods and Practices (NIFU, Oslo 2015)

    Google Scholar 

  • National Research Council: Furthering America's Research Enterprise (The National Academies, Washington 2014)

    Google Scholar 

  • B. van der Meulen, A. Rip: Evaluation of societal quality of public sector research in the Netherlands, Res. Eval. 9(1), 11–25 (2000)

    Article  Google Scholar 

  • Health Economics Research Group, O.o.H.E., RAND Europe: Medical research: What's it worth? estimating the economic benefits from medical research in the UK (Evaluation Forum, London 2008)

    Google Scholar 

  • B. Gregersen, L.T. Linde, J.G. Rasmussen: Linking between Danish universities and society, Sci. Public Policy 36(2), 151–156 (2009)

    Article  Google Scholar 

  • A. Oancea: Buzzwords and values: The prominence of “impact” in UK research policy and governance, Res. Trends (33), 6–8 (2013)

    Google Scholar 

  • SIAMPI: SIAMPI Workshop—Brussels, http://www.siampi.eu/Content/SIAMPI/Report%20SIAMPI%20workshop.pdf (2010)

  • L. Bornmann: Measuring impact in research evaluations: A thorough discussion of methods for, effects of and problems with impact measurements, High. Educ. 73(5), 775–787 (2017)

    Article  Google Scholar 

  • J. Ziman: Real Science. What it is, and What it Means (Cambridge Univ. Press, Cambridge 2000)

    Book  Google Scholar 

  • G. Bechmann, M. Decker, U. Fiedeler, B.J. Krings: Technology assessment in a complex world, Int. J. Foresight Innov. Policy 3(1), 6–27 (2007)

    Article  Google Scholar 

  • B.R. Martin, J. Irvine: Assessing basic research—Some partial indicators of scientific progress in radio astronomy, Res. Policy 12(2), 61–90 (1983)

    Article  Google Scholar 

  • L. Rymer: Measuring the Impact of Research—The Context for Metric Development (The Group of Eight, Turner 2011)

    Google Scholar 

  • P.V. Ovseiko, A. Oancea, A.M. Buchan: Assessing research impact in academic clinical medicine: A study using research excellence framework pilot impact indicators, BMC Health Serv. Res. (2012), https://doi.org/10.1186/1472-6963-12-478

    Article  Google Scholar 

  • J.B. Holbrook, S. Hrotic: Blue skies, impacts, and peer review, RT A J. Res. Policy Eval. (2013), https://doi.org/10.13130/2282-5398/2914

    Article  Google Scholar 

  • G.E. Derrick, I. Meijer, E. van Wijk: Unwrapping “impact” for evaluation: A co-word analysis of the UK REF2014 policy documents using VOSviewer. In: Proc. Sci. Technol. Indic. Conf. 2014, Leiden “Context Counts: Pathways to Master Big and Little Data”, ed. by P. Wouters (Univ. Leiden, Leiden 2014) pp. 145–154

    Google Scholar 

  • M. Knott: Academic Publications to Become Less Important when Funding University Research, The Sydney Morning Herald (15.11.2015), http://www.smh.com.au/federal-politics/political-news/academic-publications-to-become-less-important-when-funding-university-research-20151112-gkxkgl

  • L. Bornmann: Measuring the societal impact of research, EMBO Rep. 13(8), 673–676 (2012)

    Article  Google Scholar 

  • L. Bornmann: What is societal impact of research and how can it be assessed? A literature survey, J. Am. Soc. Inf. Sci. Technol. 64(2), 217–233 (2013)

    Article  Google Scholar 

  • M. Buxton, S. Hanney: How can payback from health services research be assessed?, J. Health Serv. Res. Policy 1(1), 35–43 (1996)

    Article  Google Scholar 

  • C. Donovan, S. Hanney: The ‘Payback Framework’ explained, Res. Eval. 20(3), 181–183 (2011)

    Article  Google Scholar 

  • C. Donovan: State of the art in assessing research impact: Introduction to a special issue, Res. Eval. 20(3), 175–179 (2011)

    Article  Google Scholar 

  • R. Miettinen, J. Tuunainen, T. Esko: Epistemological, artefactual and interactional–institutional foundations of social impact of academic research, Minerva 53(3), 257–277 (2015)

    Article  Google Scholar 

  • S. de Jong, K. Barker, D. Cox, T. Sveinsdottir, P. Van den Besselaar: Understanding societal impact through productive interactions: ICT research as a case, Res. Eval. 23(2), 89–192 (2014)

    Article  Google Scholar 

  • SIAMPI: Final Report on Social Impacts of Research, http://www.siampi.eu/Content/SIAMPI_Final%20report.pdf. Last Access: 11/1/2018 (2011)

  • C.H. Llewellyn Smith: What's the Use of Basic Science?, http://wwwnew.jinr.ru/section.asp?sd_id=94 (1997)

  • L. Bornmann, W. Marx: How should the societal impact of research be generated and measured? A proposal for a simple and practicable approach to allow interdisciplinary comparisons, Scientometrics 98(1), 211–219 (2014)

    Article  Google Scholar 

  • L. Bornmann: Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime, J. Informetr. 8(4), 935–950 (2014)

    Article  Google Scholar 

  • J.A. Wallin: Bibliometric methods: Pitfalls and possibilities, Basic Clin. Pharmacol. Toxicol. 97(5), 261–275 (2005)

    Article  Google Scholar 

  • C.M. Pålsson, B. Göransson, C. Brundenius: Vitalizing the Swedish university system: Implementation of the ‘third mission’, Sci. Public Policy 36(2), 145–150 (2009)

    Article  Google Scholar 

  • L. King's College, D. Science: The Nature, Scale and Beneficiaries of Research Impact: An Initial Analysis of Research Excellence Framework (REF) 2014 Impact Case Studies (King's College London, London 2015)

    Google Scholar 

  • R. Van Noorden: Seven thousand stories capture impact of science, Nature 518(7538), 150–151 (2015)

    Article  Google Scholar 

  • G. Derrick: Intentions and strategies for evaluating the societal impact of research: Insights from REF2014 evaluators. In: Proc. Sci. Technol. Indic. Conf. 2014, Leiden “Context Counts: Pathways to Master Big and Little Data”, ed. by P. Wouters (Univ. Leiden, Leiden 2014) pp. 136–144

    Google Scholar 

  • P.M. Atkinson: Assess the real cost of research assessment, Nature 516(7530), 145–145 (2014)

    Article  Google Scholar 

  • R. Smith: Measuring the social impact of research, BMJ 323(7312), 528 (2001)

    Article  Google Scholar 

  • J.M. Bensing, W.M.C.M. Caris-Verhallen, J. Dekker, D.M.J. Delnoij, P.P. Groenewegen: Doing the right thing and doing it right: Toward a framework for assessing the policy relevance of health services research, Int. J. Technol. Assess. Health Care 19(04), 604–612 (2003)

    Article  Google Scholar 

  • T. Niederkrotenthaler, T.E. Dorner, M. Maier: Development of a practical tool to measure the impact of publications on the society based on focus group discussions with scientists, BMC Public Health 11, 588 (2011)

    Article  Google Scholar 

  • S. de Rijcke, P.F. Wouters, A.D. Rushforth, T.P. Franssen, B. Hammarfelt: Evaluation practices and effects of indicator use—A literature review, Res. Eval. 25(2), 161–169 (2016)

    Article  Google Scholar 

  • Anon: The maze of impact metrics, Nature 502(7471), 271 (2013)

    Google Scholar 

  • M. Thelwall, K. Kousha: Web indicators for research evaluation. Part 2: Social media metrics, Prof. Inform. 24(5), 607–620 (2015)

    Google Scholar 

  • M. Thelwall, N. Maflahi: Guideline references and academic citations as evidence of the clinical value of health research, J. Assoc. Inf. Sci. Technol. 67(4), 960–966 (2016)

    Article  Google Scholar 

  • R. Costas, Z. Zahedi, P. Wouters: Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective, J. Assoc. Inf. Sci. Technol. 66(10), 2003–2019 (2015)

    Article  Google Scholar 

  • L. Bornmann, R. Haunschild, W. Marx: Policy documents as sources for measuring societal impact: How is climate change research perceived in policy documents?, Scientometrics 109(3), 1477–1495 (2016)

    Article  Google Scholar 

  • R. Haunschild, L. Bornmann: How many scientific papers are mentioned in policy-related documents? An empirical investigation using web of science and altmetric data, Scientometrics 110(3), 1209–1216 (2017)

    Article  Google Scholar 

  • N. Robinson-Garcia, D. Torres-Salinas, Z. Zahedi, R. Costas: New data, new possibilities: Exploring the insides of altmetric.com, Prof. Inform. 23(4), 359–366 (2014)

    Google Scholar 

  • L. Colledge: Snowball Metrics Recipe Book (Snowball Metrics Program Partners, Amsterdam 2014)

    Google Scholar 

  • L. Bornmann, R. Haunschild: Which people use which scientific papers? An evaluation of data from F1000 and Mendeley, J. Informetr. 9(3), 477–487 (2015)

    Article  Google Scholar 

  • L. Bornmann: Usefulness of altmetrics for measuring the broader impact of research: A case study using data from PLOS and F1000Prime, Aslib J. Inf. Manag. 67(3), 305–319 (2015)

    Article  Google Scholar 

  • L. Bornmann: Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics, Scientometrics 103(3), 1123–1144 (2015)

    Article  Google Scholar 

  • M. Erdt, A. Nagarajan, S.C. Sin, Y.L. Theng: Altmetrics: An analysis of the state-of-the-art in measuring research impact on social media, Scientometrics 109(2), 1117–1166 (2016)

    Article  Google Scholar 

  • L. Bornmann, R. Haunschild: How to normalize Twitter counts? A first attempt based on journals in the twitter index, Scientometrics 107(3), 1405–1422 (2016)

    Article  Google Scholar 

  • L. Bornmann, R. Haunschild: Normalization of mendeley reader impact on the reader- and paper-side: A comparison of the mean discipline normalized reader score (MDNRS) with the mean normalized reader score (MNRS) and bare reader counts, J. Informetr. 10(3), 776–788 (2016)

    Article  Google Scholar 

  • L. Bornmann, R. Haunschild: Measuring field-normalized impact of papers on specific societal groups: An altmetrics study based on Mendeley data, Res. Eval. 26(3), 230–241 (2017)

    Article  Google Scholar 

  • R. Fairclough, M. Thelwall: National research impact indicators from Mendeley readers, J. Informetr. 9(4), 845–859 (2015)

    Article  Google Scholar 

  • R. Haunschild, L. Bornmann: Normalization of Mendeley reader counts for impact assessment, J. Informetr. 10(1), 62–73 (2016)

    Article  Google Scholar 

  • R. Haunschild, H. Schier, L. Bornmann: Proposal of a minimum constraint for indicators based on means or averages, J. Informetr. 10(2), 485–486 (2016)

    Article  Google Scholar 

  • M. Thelwall: Three practical field normalised alternative indicator formulae for research evaluation, J. Informetr. 11(1), 128–151 (2017)

    Article  Google Scholar 

  • R. Haunschild, L. Bornmann: Normalization of zero-inflated data: An empirical analysis of a new indicator family. In: 16th Int. Soc. Scientometr. Informetr. Conf. ISSI 2016, Wuhan (2017), https://arxiv.org/abs/1704.02211

    Google Scholar 

  • L. Bornmann, R. Haunschild: Normalization of Zero-Inflated Data: An Empirical Analysis of a New Indicator Family and its Use with Altmetrics Data, J. Informetr. 12(3), 998–1011 (2018)

    Article  Google Scholar 

  • A. Patel, J. Chavda: Measuring research impact: Bibliometrics, social media, altmetrics, and the BJGP, Br. J. Gen. Pract. 66(642), E59–E61 (2016)

    Article  Google Scholar 

  • L. Bornmann, R. Haunschild: To what extent does the Leiden manifesto also apply to altmetrics? A discussion of the manifesto against the background of research into altmetrics, Online Inf. Rev. 40(4), 529–543 (2016)

    Article  Google Scholar 

  • S. Haustein, T.D. Bowman, K. Holmberg, A. Tsou, C.R. Sugimoto, V. Lariviere: Tweets as impact indicators: Examining the implications of automated “bot” accounts on twitter, J. Assoc. Inf. Sci. Technol. 67(1), 232–238 (2016)

    Article  Google Scholar 

  • J.A. Arevalo, J.A. Cordon-Garcia, B.M. Barba: Almetrics: Measuring the influence of the media on the social impact of research, Cuad. Doc. Multimed. 27(1), 75–101 (2016)

    Google Scholar 

  • S. Haustein: Grand challenges in altmetrics: Heterogeneity, data quality and dependencies, Scientometrics 108(1), 413–423 (2016)

    Article  Google Scholar 

  • J. Callaert, B. Van Looy, A. Verbeek, K. Debackere, B. Thijs: Traces of prior art: An analysis of non-patent references found in patent documents, Scientometrics 69(1), 3–20 (2006)

    Article  Google Scholar 

  • K. Kousha, M. Thelwall: Patent Citation Analysis with Google, J. Assoc. Inf. Sci. Technol. 68(1), 48–61 (2017)

    Article  Google Scholar 

  • F. Narin, K.S. Hamilton, D. Olivastro: The increasing linkage between US technology and public science, Res. Policy 26(3), 317–330 (1997)

    Article  Google Scholar 

  • C.K.N. Chang, A. Breitzman: Using patents prospectively to identify emerging, high-impact technological clusters, Res. Eval. 18(5), 357–364 (2009)

    Article  Google Scholar 

  • F. Narin, M.P. Carpenter, P. Woolf: Technological performance assessments based on patents and patent citations, IEEE Trans. Eng. Manag. 31(4), 172–183 (1984)

    Article  Google Scholar 

  • Elsevier Research Intelligence: SciVal Patent Metrics, https://www.elsevier.com/research-intelligence/resource-library/scival-patent-metrics (2016)

  • J.M. Grimshaw, I.T. Russell: Effect of clinical guidelines on medical-practice—A systematic review of rigorous evaluations, Lancet 342(8883), 1317–1322 (1993)

    Article  Google Scholar 

  • P. Shekelle, S. Woolf, J.M. Grimshaw, H.J. Schunemann, M.P. Eccles: Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development, Implement. Sci. (2012), https://doi.org/10.1186/1748-5908-7-62

    Article  Google Scholar 

  • J.P. Andersen: Association between quality of clinical practice guidelines and citations given to their references. In: 14th Int. Soc. Scientometr. Informetr. Conf., ISSI 2013, Vienna (2013)

    Google Scholar 

  • M. Thelwall, N. Maflahi: Guideline references and academic citations as evidence of the clinical value of health research, J. Assoc. Inf. Sci. Technol. 67(4), 960–966 (2015), https://doi.org/10.1002/asi.23432

    Article  Google Scholar 

  • D. Kryl, L. Allen, K. Dolby, B. Sherbon, I. Viney: Tracking the impact of research on policy and practice: Investigating the feasibility of using citations in clinical guidelines for research evaluation, BMJ Open 2(2), 7 (2012)

    Article  Google Scholar 

  • G. Lewison, R. Sullivan: The impact of cancer research: How publications influence UK cancer clinical guidelines, Br. J. Cancer 98(12), 1944–1950 (2008)

    Article  Google Scholar 

  • T.H. Jones, S. Hanney: Tracing the indirect societal impacts of biomedical research: Development and piloting of a technique based on citations, Scientometrics 107(3), 975–1003 (2016)

    Article  Google Scholar 

  • F. Bunn, D. Trivedi, P. Alderson, L. Hamilton, A. Martin, E. Pinkney, S. Iliffe: The impact of Cochrane reviews: A mixed-methods evaluation of outputs from Cochrane review groups supported by the national institute for health research, Health Technol. Assess. (2015), https://doi.org/10.3310/hta19280

    Article  Google Scholar 

  • A. Ritter, K. Lancaster: Measuring research influence on drug policy: A case example of two epidemiological monitoring systems, Int. J. Drug Policy 24(1), 30–37 (2013)

    Article  Google Scholar 

  • J. Liu: New Source Alert: Policy Documents, http://www.altmetric.com/blog/new-source-alert-policy-documents/ (2014)

  • T. Arthur: Categorizing Policy Document Citations in PlumX, http://plumanalytics.com/categorizing-policy-document-citations-in-plumx/ (2016)

  • A. Ballabeni, A. Boggio, D. Hemenway: Policies to increase the social value of science and the scientist satisfaction. An exploratory survey among Harvard bioscientists, F1000Research 3, 20 (2014)

    Article  Google Scholar 

  • L. Bornmann: Is there currently a scientific revolution in scientometrics?, J. Assoc. Inf. Sci. Technol. 65(3), 647–648 (2014)

    Article  Google Scholar 

  • L. Bornmann: Scientific revolution in scientometrics: The broadening of impact from citation to societal. In: Theories of Informetrics and Scholarly Communication, ed. by C.R. Sugimoto (De Gruyter, Berlin 2016) pp. 347–359

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division for Science and Innovation Studies, Administrative Headquarters of the Max Planck Society, Munich, Germany

    Lutz Bornmann

  2. Central Information Service, Max Planck Institute for Solid State Research, Stuttgart, Germany

    Robin Haunschild

Authors
  1. Lutz Bornmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robin Haunschild
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lutz Bornmann .

Editor information

Editors and Affiliations

  1. ECOOM and Faculty of Economics and Business, KU Leuven, Leuven, Belgium

    Wolfgang Glänzel

  2. Amsterdam, The Netherlands

    Henk F. Moed

  3. Competence Center Policy – Industry – Innovation, Fraunhofer Institute for Systems and Innovation Research ISI, Karlsruhe, Germany

    Ulrich Schmoch

  4. Faculty of Science and Engineering, University of Wolverhampton, Wolverhampton, UK

    Mike Thelwall

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer International Publishing AG, part of Springer Nature

About this chapter

Cite this chapter

Bornmann, L., Haunschild, R. (2019). Societal Impact Measurement of Research Papers. In: Glänzel, W., Moed, H.F., Schmoch, U., Thelwall, M. (eds) Springer Handbook of Science and Technology Indicators. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-030-02511-3_23

Download citation

Publish with us

Policies and ethics

Search

Navigation