Abstract
The introduction of a new health technology (e.g., drug, device, vaccine, complex interventions, systems) always repercuss on the clinical flow, healthcare organizational levels, and on the society. In this scenario, researchers, managers, health professionals, and other stakeholders should carefully consider the efficacy or effectiveness, safety, and efficiency or appropriateness of the technology drawn in an explicit analytical framework at different stages of its life cycle (i.e., innovation, regulation and diffusion, implementation, adoption, and obsolescence). “Evaluation research” refers to rigorous research methods and designs grounded in social sciences and structured to quantify the favorable influence of technologies or their effect over time, aiming at providing evidence for decision-making and extending knowledge, especially in public policies. Example of some approaches include experimental trials, observational studies, patient-simulations, record and playback techniques, network analysis, data mining, benchmarking. Evaluation research models – using both qualitative and quantitative approaches (e.g., process analysis, outcome analysis, cost-effectiveness analysis, value of information, multicriteria decision-analysis), should be done before, during, and after the implementation of a technology in order to answer the questions related to its impacts from different perspectives. This chapter provides an overview of the concepts, history, and methods for evaluation research, as well as discusses the available evidence on this topic in the field of pharmacy services.
Similar content being viewed by others
References
Alexander WE, Farrell JP. Evaluation research: some problems and a possible solution. Can J Educ. 1981;6(2)
Andronis L, Billingham LJ, Bryan S, James ND, Barton PM. A practical application of value of information and prospective payback of research to prioritize evaluative research. Med Decis Mak. 2016;36(3):321–34.
Banta D, Kristensen FB, Jonsson E. A history of health technology assessment at the European level. Int J Technol Assess Health Care. 2009;25(Suppl 1):68–73.
Better Evaluation. https://wwwbetterevaluationorg/. 2021.
Bonetti AF, Tonin FS, Della Rocca AM, Lucchetta RC, Fernandez-Llimos F, Pontarolo R. Methodological quality and risk of bias of meta-analyses of pharmacy services: a systematic review. Res Social Adm Pharm. 2020;42:1252.
Boruch RF, Wortman PM, Cordray DS. Reanalyzing program evaluations. San Francisco: JosseyBass; 1981.
Campbel DT, Stanley JC. Experimental and quasi-experimental designs for research. Dallas/Geneva, IL/Hopewell, NJ/Palo Alto/London: Houghton Mifflin Company Boston; 1966.
Chalmers I. It’s official: evaluative research must become part of routine care in the NHS. J R Soc Med. 2000;93(11):555–6.
Chang R, Little TD. Innovations for evaluation research: multiform protocols, visual analog scaling, and the retrospective pretest-posttest design. Eval Health Prof. 2018;41(2):246–69.
Chisholm-Burns MA, Graff Zivin JS, Lee JK, Spivey CA, Slack M, Herrier RN, et al. Economic effects of pharmacists on health outcomes in the United States: a systematic review. Am J Health Syst Pharm. 2010;67(19):1624–34.
Cohen J. Statistical power analysis for the behavioral sciences. New York: Routledge; 1977.
De Rijdt T, Willems L, Simoens S. Economic effects of clinical pharmacy interventions: a literature review. Am J Health Syst Pharm. 2008;65(12):1161–72.
Drummond MF, Schwartz JS, Jonsson B, Luce BR, Neumann PJ, Siebert U, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care. 2008;24(3):244–58. discussion 362-8
Elliott RA, Putman K, Davies J, Annemans L. A review of the methodological challenges in assessing the cost effectiveness of pharmacist interventions. PharmacoEconomics. 2014;32(12):1185–99.
Feinstein AR. Clinical judgment. Malabar: Krieger; 1967.
Frazao TDC, Camilo DGG, Cabral ELS, Souza RP. Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps. BMC Med Inform Decis Mak. 2018;18(1):90.
Garattini S, Jakobsen JC, Wetterslev J, Bertele V, Banzi R, Rath A, et al. Evidence-based clinical practice: overview of threats to the validity of evidence and how to minimise them. Eur J Intern Med. 2016;32:13–21.
Garrido MV, Kristensen FB, Nielsen CP, Busse R. Health technology assessment and health policy-making in Europe: current status, challenges and potential. World Health Organization, European Observatory on Health Systems and Policies; 2008.
Glass GV, McGaw B, Smith ML. Meta-analysis in social research. London: Sage; 1981.
Goodman CS. Healthcare technology assessment: methods, framework, and role in policy making. Am J Manag Care. 1998;4:SP200-14.
Gutierrez-Ibarluzea I, Chiumente M, Dauben HP. The life cycle of health technologies. Challenges and ways forward. Front Pharmacol. 2017;8:14.
Henshall C, Mardhani-Bayne L, Fronsdal KB, Klemp M. Interactions between health technology assessment, coverage, and regulatory processes: emerging issues, goals, and opportunities. Int J Technol Assess Health Care. 2011;27(3):253–60.
Houle SKD, Carter CA, Tsuyuki RT, Grindrod KA. Remunerated patient care services and injections by pharmacists: an international update. Can Pharm J (Ott). 2019;152(2):92–108.
House VG. Evaluation research: the need for multiple criteria. J Adv Nurs. 1977;2(1):15–20.
Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Value Health. 2013;16(2):e1–5.
Huynh AK, Hamilton AB, Farmer MM, Bean-Mayberry B, Stirman SW, Moin T, et al. A pragmatic approach to guide implementation evaluation research: strategy mapping for complex interventions. Front Public Health. 2018;6:134.
Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514.
Ioannidis JP. Meta-analyses can be credible and useful: a new standard. JAMA Psychiat. 2017;74(4):311–2.
Jackson J, Urick B. Performance-based pharmacy payment models: the case for change. Aust Health Rev. 2019;43(5):502–7.
Kaplan B, Shaw NT. Future directions in evaluation research: people, organizational, and social issues. Methods Inf Med. 2004;43(3):215–31.
Kenny DA. Correlation and causality. New York: Wiley; 1979.
Kosari S, Deeks LS, Naunton M, Dawda P, Postma MJ, Tay GH, et al. Funding pharmacists in general practice: a feasibility study to inform the design of future economic evaluations. Res Social Adm Pharm. 2020;17:1012.
Lakens D, Hilgard J, Staaks J. On the reproducibility of meta-analyses: six practical recommendations. BMC Psychol. 2016;4(1):24.
Luker KA. An overview of evaluation research in nursing. J Adv Nurs. 1981;6(2):87–93.
McNeece CA, DiNitto DM, Johnson PJ. The utility of evaluation research for administrative decision-making. Adm Soc Work. 1983;7(3–4):77–87.
Moecker R, Terstegen T, Haefeli WE, Seidling HM. The influence of intervention complexity on barriers and facilitators in the implementation of professional pharmacy services – a systematic review. Res Social Adm Pharm. 2021;17:1651.
Mosteller F. Innovation and evaluation. Science. 1981;211(4484)
Murphy EM, Rodis JL, Mann HJ. Three ways to advocate for the economic value of the pharmacist in health care. J Am Pharm Assoc (2003). 2020;60(6):e116–e24.
Mussen F, Salek S, Walker S. A quantitative approach to benefit-risk assessment of medicines – part 1: the development of a new model using multi-criteria decision analysis. Pharmacoepidemiol Drug Saf. 2007;16(S1):S2–S15.
National Academies of Sciences, Engineering, and Medicine. Reproducibility and replicability in science. Washington, DC; 2019.
Newman TV, San-Juan-Rodriguez A, Parekh N, Swart ECS, Klein-Fedyshin M, Shrank WH, et al. Impact of community pharmacist-led interventions in chronic disease management on clinical, utilization, and economic outcomes: an umbrella review. Res Social Adm Pharm. 2020;16(9):1155–65.
O’Donnell JC, Pham SV, Pashos CL, Miller DW, Smith MD. Health technology assessment: lessons learned from around the world–an overview. Value Health. 2009;12 Suppl 2:S1–5.
Onozato T, Francisca Dos Santos Cruz C, Milhome da Costa Farre AG, Silvestre CC, de Oliveira Santos Silva R, Araujo Dos Santos G Jr, et al. Factors influencing the implementation of clinical pharmacy services for hospitalized patients: a mixed-methods systematic review. Res Social Adm Pharm. 2020;16(4):437–49.
Page MJ, Altman DG, Shamseer L, McKenzie JE, Ahmadzai N, Wolfe D, et al. Reproducible research practices are underused in systematic reviews of biomedical interventions. J Clin Epidemiol. 2018;94:8–18.
Raine R, Fitzpatrick R, de Pury J. Challenges, solutions and future directions in evaluative research. J Health Serv Res Policy. 2016a;21(4):215–6.
Raine R, Fitzpatrick R, Barratt H, Bevan G, Black N, Boaden R, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. In: Health services and delivery research. Southampton; 2016b.
Rauh S, Torgerson T, Johnson AL, Pollard J, Tritz D, Vassar M. Reproducible and transparent research practices in published neurology research. Res Integr Peer Rev. 2020;5:5.
Rubio-Valera M, Bosmans J, Fernandez A, Penarrubia-Maria M, March M, Trave P, et al. Cost-effectiveness of a community pharmacist intervention in patients with depression: a randomized controlled trial (PRODEFAR study). PLoS One. 2013;8(8):e70588.
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71–2.
Saokaew S, Maphanta S, Thangsomboon P. Impact of pharmacist's interventions on cost of drug therapy in intensive care unit. Pharm Pract (Granada). 2009;7(2):81–7.
Satterfield JM, Spring B, Brownson RC, Mullen EJ, Newhouse RP, Walker BB, et al. Toward a transdisciplinary model of evidence-based practice. Milbank Q. 2009;87(2):368–90.
Suchman EA. Evaluation research. New York: Russell Saga Foundation; 1967.
Tavakol M, Gruppen LD, Torabi S. Using evaluation research to improve medical education. Clin Teach. 2010;7(3):192–6.
Tervonen T, van Valkenhoef G, Buskens E, Hillege HL, Postmus D. A stochastic multicriteria model for evidence-based decision making in drug benefit-risk analysis. Stat Med. 2011;30(12):1419–28.
Thompson MS. Benefit-cost analysis for program evaluation. London: Sage; 1980.
Tonin FS, Borba HH, Leonart LP, Mendes AM, Steimbach LM, Pontarolo R, et al. Methodological quality assessment of network meta-analysis of drug interventions: implications from a systematic review. Int J Epidemiol. 2019a;48(2):620–32.
Tonin FS, Wiecek E, Torres-Robles A, Pontarolo R, Benrimoj SCI, Fernandez-Llimos F, et al. An innovative and comprehensive technique to evaluate different measures of medication adherence: the network meta-analysis. Res Social Adm Pharm. 2019b;15(4):358–65.
Tonin FS, Aznar-Lou I, Pontinha VM, Pontarolo R, Fernandez-Llimos F. Principles of pharmacoeconomic analysis: the case of pharmacist-led interventions. Pharm Pract (Granada). 2021;19(1):2302.
Touchette DR, Doloresco F, Suda KJ, Perez A, Turner S, Jalundhwala Y, et al. Economic evaluations of clinical pharmacy services: 2006–2010. Pharmacotherapy. 2014;34(8):771–93.
Tukey IW. Some thoughts on clinical trials, especially problems of multiplicity. Science 1977;198(4318):679–84.
Urick BY, Urmie JM. Framework for assessing pharmacy value. Res Social Adm Pharm. 2019;15(11):1326–37.
Wagstaff A, Culyer AJ. Four decades of health economics through a bibliometric lens. J Health Econ. 2012;31(2):406–39.
Wortman PL. Evaluation research: a methodological perspective. Ann Rev PsychoL. 1983;34:223–60.
Zeater S, Benrimoj SI, Fernandez-Llimos F, Garcia-Cardenas V. A model for the financial assessment of professional services in community pharmacy: a systematic review. J Am Pharm Assoc (2003). 2019;59(1):108–16 e1.
Author information
Authors and Affiliations
Corresponding author
Section Editor information
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this entry
Cite this entry
Tonin, F.S., Fernandez-Llimos, F. (2022). Evidence in Evaluation Research. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_56-1
Download citation
DOI: https://doi.org/10.1007/978-3-030-50247-8_56-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50247-8
Online ISBN: 978-3-030-50247-8
eBook Packages: Living Reference Biomedicine and Life SciencesReference Module Biomedical and Life Sciences