Skip to main content
SpringerLink
Log in

Evidence in Evaluation Research

  • Living reference work entry
  • First Online:
Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy

  • 23 Accesses

Abstract

The introduction of a new health technology (e.g., drug, device, vaccine, complex interventions, systems) always repercuss on the clinical flow, healthcare organizational levels, and on the society. In this scenario, researchers, managers, health professionals, and other stakeholders should carefully consider the efficacy or effectiveness, safety, and efficiency or appropriateness of the technology drawn in an explicit analytical framework at different stages of its life cycle (i.e., innovation, regulation and diffusion, implementation, adoption, and obsolescence). “Evaluation research” refers to rigorous research methods and designs grounded in social sciences and structured to quantify the favorable influence of technologies or their effect over time, aiming at providing evidence for decision-making and extending knowledge, especially in public policies. Example of some approaches include experimental trials, observational studies, patient-simulations, record and playback techniques, network analysis, data mining, benchmarking. Evaluation research models – using both qualitative and quantitative approaches (e.g., process analysis, outcome analysis, cost-effectiveness analysis, value of information, multicriteria decision-analysis), should be done before, during, and after the implementation of a technology in order to answer the questions related to its impacts from different perspectives. This chapter provides an overview of the concepts, history, and methods for evaluation research, as well as discusses the available evidence on this topic in the field of pharmacy services.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Alexander WE, Farrell JP. Evaluation research: some problems and a possible solution. Can J Educ. 1981;6(2)

    Google Scholar 

  • Andronis L, Billingham LJ, Bryan S, James ND, Barton PM. A practical application of value of information and prospective payback of research to prioritize evaluative research. Med Decis Mak. 2016;36(3):321–34.

    Article  Google Scholar 

  • Banta D, Kristensen FB, Jonsson E. A history of health technology assessment at the European level. Int J Technol Assess Health Care. 2009;25(Suppl 1):68–73.

    Article  Google Scholar 

  • Better Evaluation. https://wwwbetterevaluationorg/. 2021.

    Google Scholar 

  • Bonetti AF, Tonin FS, Della Rocca AM, Lucchetta RC, Fernandez-Llimos F, Pontarolo R. Methodological quality and risk of bias of meta-analyses of pharmacy services: a systematic review. Res Social Adm Pharm. 2020;42:1252.

    Google Scholar 

  • Boruch RF, Wortman PM, Cordray DS. Reanalyzing program evaluations. San Francisco: JosseyBass; 1981.

    Google Scholar 

  • Campbel DT, Stanley JC. Experimental and quasi-experimental designs for research. Dallas/Geneva, IL/Hopewell, NJ/Palo Alto/London: Houghton Mifflin Company Boston; 1966.

    Google Scholar 

  • Chalmers I. It’s official: evaluative research must become part of routine care in the NHS. J R Soc Med. 2000;93(11):555–6.

    Article  CAS  Google Scholar 

  • Chang R, Little TD. Innovations for evaluation research: multiform protocols, visual analog scaling, and the retrospective pretest-posttest design. Eval Health Prof. 2018;41(2):246–69.

    Article  Google Scholar 

  • Chisholm-Burns MA, Graff Zivin JS, Lee JK, Spivey CA, Slack M, Herrier RN, et al. Economic effects of pharmacists on health outcomes in the United States: a systematic review. Am J Health Syst Pharm. 2010;67(19):1624–34.

    Article  Google Scholar 

  • Cohen J. Statistical power analysis for the behavioral sciences. New York: Routledge; 1977.

    Google Scholar 

  • De Rijdt T, Willems L, Simoens S. Economic effects of clinical pharmacy interventions: a literature review. Am J Health Syst Pharm. 2008;65(12):1161–72.

    Article  Google Scholar 

  • Drummond MF, Schwartz JS, Jonsson B, Luce BR, Neumann PJ, Siebert U, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care. 2008;24(3):244–58. discussion 362-8

    Article  Google Scholar 

  • Elliott RA, Putman K, Davies J, Annemans L. A review of the methodological challenges in assessing the cost effectiveness of pharmacist interventions. PharmacoEconomics. 2014;32(12):1185–99.

    Article  Google Scholar 

  • Feinstein AR. Clinical judgment. Malabar: Krieger; 1967.

    Google Scholar 

  • Frazao TDC, Camilo DGG, Cabral ELS, Souza RP. Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps. BMC Med Inform Decis Mak. 2018;18(1):90.

    Article  Google Scholar 

  • Garattini S, Jakobsen JC, Wetterslev J, Bertele V, Banzi R, Rath A, et al. Evidence-based clinical practice: overview of threats to the validity of evidence and how to minimise them. Eur J Intern Med. 2016;32:13–21.

    Article  Google Scholar 

  • Garrido MV, Kristensen FB, Nielsen CP, Busse R. Health technology assessment and health policy-making in Europe: current status, challenges and potential. World Health Organization, European Observatory on Health Systems and Policies; 2008.

    Google Scholar 

  • Glass GV, McGaw B, Smith ML. Meta-analysis in social research. London: Sage; 1981.

    Google Scholar 

  • Goodman CS. Healthcare technology assessment: methods, framework, and role in policy making. Am J Manag Care. 1998;4:SP200-14.

    Google Scholar 

  • Gutierrez-Ibarluzea I, Chiumente M, Dauben HP. The life cycle of health technologies. Challenges and ways forward. Front Pharmacol. 2017;8:14.

    Article  Google Scholar 

  • Henshall C, Mardhani-Bayne L, Fronsdal KB, Klemp M. Interactions between health technology assessment, coverage, and regulatory processes: emerging issues, goals, and opportunities. Int J Technol Assess Health Care. 2011;27(3):253–60.

    Article  Google Scholar 

  • Houle SKD, Carter CA, Tsuyuki RT, Grindrod KA. Remunerated patient care services and injections by pharmacists: an international update. Can Pharm J (Ott). 2019;152(2):92–108.

    Article  Google Scholar 

  • House VG. Evaluation research: the need for multiple criteria. J Adv Nurs. 1977;2(1):15–20.

    Article  CAS  Google Scholar 

  • Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Value Health. 2013;16(2):e1–5.

    Article  Google Scholar 

  • Huynh AK, Hamilton AB, Farmer MM, Bean-Mayberry B, Stirman SW, Moin T, et al. A pragmatic approach to guide implementation evaluation research: strategy mapping for complex interventions. Front Public Health. 2018;6:134.

    Article  Google Scholar 

  • Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514.

    Article  Google Scholar 

  • Ioannidis JP. Meta-analyses can be credible and useful: a new standard. JAMA Psychiat. 2017;74(4):311–2.

    Article  Google Scholar 

  • Jackson J, Urick B. Performance-based pharmacy payment models: the case for change. Aust Health Rev. 2019;43(5):502–7.

    Article  Google Scholar 

  • Kaplan B, Shaw NT. Future directions in evaluation research: people, organizational, and social issues. Methods Inf Med. 2004;43(3):215–31.

    Article  CAS  Google Scholar 

  • Kenny DA. Correlation and causality. New York: Wiley; 1979.

    Google Scholar 

  • Kosari S, Deeks LS, Naunton M, Dawda P, Postma MJ, Tay GH, et al. Funding pharmacists in general practice: a feasibility study to inform the design of future economic evaluations. Res Social Adm Pharm. 2020;17:1012.

    Article  Google Scholar 

  • Lakens D, Hilgard J, Staaks J. On the reproducibility of meta-analyses: six practical recommendations. BMC Psychol. 2016;4(1):24.

    Article  Google Scholar 

  • Luker KA. An overview of evaluation research in nursing. J Adv Nurs. 1981;6(2):87–93.

    Article  CAS  Google Scholar 

  • McNeece CA, DiNitto DM, Johnson PJ. The utility of evaluation research for administrative decision-making. Adm Soc Work. 1983;7(3–4):77–87.

    Article  CAS  Google Scholar 

  • Moecker R, Terstegen T, Haefeli WE, Seidling HM. The influence of intervention complexity on barriers and facilitators in the implementation of professional pharmacy services – a systematic review. Res Social Adm Pharm. 2021;17:1651.

    Article  Google Scholar 

  • Mosteller F. Innovation and evaluation. Science. 1981;211(4484)

    Google Scholar 

  • Murphy EM, Rodis JL, Mann HJ. Three ways to advocate for the economic value of the pharmacist in health care. J Am Pharm Assoc (2003). 2020;60(6):e116–e24.

    Article  Google Scholar 

  • Mussen F, Salek S, Walker S. A quantitative approach to benefit-risk assessment of medicines – part 1: the development of a new model using multi-criteria decision analysis. Pharmacoepidemiol Drug Saf. 2007;16(S1):S2–S15.

    Article  Google Scholar 

  • National Academies of Sciences, Engineering, and Medicine. Reproducibility and replicability in science. Washington, DC; 2019.

    Google Scholar 

  • Newman TV, San-Juan-Rodriguez A, Parekh N, Swart ECS, Klein-Fedyshin M, Shrank WH, et al. Impact of community pharmacist-led interventions in chronic disease management on clinical, utilization, and economic outcomes: an umbrella review. Res Social Adm Pharm. 2020;16(9):1155–65.

    Article  Google Scholar 

  • O’Donnell JC, Pham SV, Pashos CL, Miller DW, Smith MD. Health technology assessment: lessons learned from around the world–an overview. Value Health. 2009;12 Suppl 2:S1–5.

    Article  Google Scholar 

  • Onozato T, Francisca Dos Santos Cruz C, Milhome da Costa Farre AG, Silvestre CC, de Oliveira Santos Silva R, Araujo Dos Santos G Jr, et al. Factors influencing the implementation of clinical pharmacy services for hospitalized patients: a mixed-methods systematic review. Res Social Adm Pharm. 2020;16(4):437–49.

    Article  Google Scholar 

  • Page MJ, Altman DG, Shamseer L, McKenzie JE, Ahmadzai N, Wolfe D, et al. Reproducible research practices are underused in systematic reviews of biomedical interventions. J Clin Epidemiol. 2018;94:8–18.

    Article  Google Scholar 

  • Raine R, Fitzpatrick R, de Pury J. Challenges, solutions and future directions in evaluative research. J Health Serv Res Policy. 2016a;21(4):215–6.

    Article  Google Scholar 

  • Raine R, Fitzpatrick R, Barratt H, Bevan G, Black N, Boaden R, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. In: Health services and delivery research. Southampton; 2016b.

    Google Scholar 

  • Rauh S, Torgerson T, Johnson AL, Pollard J, Tritz D, Vassar M. Reproducible and transparent research practices in published neurology research. Res Integr Peer Rev. 2020;5:5.

    Article  Google Scholar 

  • Rubio-Valera M, Bosmans J, Fernandez A, Penarrubia-Maria M, March M, Trave P, et al. Cost-effectiveness of a community pharmacist intervention in patients with depression: a randomized controlled trial (PRODEFAR study). PLoS One. 2013;8(8):e70588.

    Article  CAS  Google Scholar 

  • Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71–2.

    Article  CAS  Google Scholar 

  • Saokaew S, Maphanta S, Thangsomboon P. Impact of pharmacist's interventions on cost of drug therapy in intensive care unit. Pharm Pract (Granada). 2009;7(2):81–7.

    Article  Google Scholar 

  • Satterfield JM, Spring B, Brownson RC, Mullen EJ, Newhouse RP, Walker BB, et al. Toward a transdisciplinary model of evidence-based practice. Milbank Q. 2009;87(2):368–90.

    Article  Google Scholar 

  • Suchman EA. Evaluation research. New York: Russell Saga Foundation; 1967.

    Google Scholar 

  • Tavakol M, Gruppen LD, Torabi S. Using evaluation research to improve medical education. Clin Teach. 2010;7(3):192–6.

    Article  Google Scholar 

  • Tervonen T, van Valkenhoef G, Buskens E, Hillege HL, Postmus D. A stochastic multicriteria model for evidence-based decision making in drug benefit-risk analysis. Stat Med. 2011;30(12):1419–28.

    Article  Google Scholar 

  • Thompson MS. Benefit-cost analysis for program evaluation. London: Sage; 1980.

    Google Scholar 

  • Tonin FS, Borba HH, Leonart LP, Mendes AM, Steimbach LM, Pontarolo R, et al. Methodological quality assessment of network meta-analysis of drug interventions: implications from a systematic review. Int J Epidemiol. 2019a;48(2):620–32.

    Article  Google Scholar 

  • Tonin FS, Wiecek E, Torres-Robles A, Pontarolo R, Benrimoj SCI, Fernandez-Llimos F, et al. An innovative and comprehensive technique to evaluate different measures of medication adherence: the network meta-analysis. Res Social Adm Pharm. 2019b;15(4):358–65.

    Article  Google Scholar 

  • Tonin FS, Aznar-Lou I, Pontinha VM, Pontarolo R, Fernandez-Llimos F. Principles of pharmacoeconomic analysis: the case of pharmacist-led interventions. Pharm Pract (Granada). 2021;19(1):2302.

    Article  Google Scholar 

  • Touchette DR, Doloresco F, Suda KJ, Perez A, Turner S, Jalundhwala Y, et al. Economic evaluations of clinical pharmacy services: 2006–2010. Pharmacotherapy. 2014;34(8):771–93.

    Article  Google Scholar 

  • Tukey IW. Some thoughts on clinical trials, especially problems of multiplicity. Science 1977;198(4318):679–84.

    Google Scholar 

  • Urick BY, Urmie JM. Framework for assessing pharmacy value. Res Social Adm Pharm. 2019;15(11):1326–37.

    Article  Google Scholar 

  • Wagstaff A, Culyer AJ. Four decades of health economics through a bibliometric lens. J Health Econ. 2012;31(2):406–39.

    Article  Google Scholar 

  • Wortman PL. Evaluation research: a methodological perspective. Ann Rev PsychoL. 1983;34:223–60.

    Article  Google Scholar 

  • Zeater S, Benrimoj SI, Fernandez-Llimos F, Garcia-Cardenas V. A model for the financial assessment of professional services in community pharmacy: a systematic review. J Am Pharm Assoc (2003). 2019;59(1):108–16 e1.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Pharmaceutical Sciences Postgraduate Programme, Federal University of Paraná, Curitiba, Brazil

    Fernanda S. Tonin

  2. CINTESIS – Center for Health Technology and Services Research, Laboratory of Pharmacology, Department of Drug Sciences, Faculty of Pharmacy, University of Porto, Porto, Portugal

    Fernando Fernandez-Llimos

Authors
  1. Fernanda S. Tonin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando Fernandez-Llimos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Fernandez-Llimos .

Section Editor information

  1. College of Pharmacy, Qatar University, Doha, Qatar

    Derek Charles Stewart

  2. Department of Pharmacy, University of Huddersfield, Huddersfield, UK

    Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Tonin, F.S., Fernandez-Llimos, F. (2022). Evidence in Evaluation Research. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_56-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-50247-8_56-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-50247-8

  • Online ISBN: 978-3-030-50247-8

  • eBook Packages: Springer Reference Biomedicine and Life SciencesReference Module Biomedical and Life Sciences

Publish with us

Policies and ethics

Search

Navigation