Abstract
E-learning is a method that meets the time and distance needs of postgraduate medical education. The general workload of postgraduates and disasters such as the Covid-19 pandemic create a need for e-learning. Therefore, in recent years, e-learning has drawn the attention of researchers because of its potential as a practical instructional approach in PGME. Systematic review method was employed in this study aiming to examine the e-learning applications in PGME through a summary of several recent studies that meet predetermined criteria regarding e-learning for postgraduate medical education. The PICOS (population, intervention, comparison, outcome, study design) framework was used to decide on inclusion and exclusion criteria. On reporting the results of the study, The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was used. The review examines the differences between the identified studies in terms of research methods, data collection tools, study duration, sample size, and data analysis method. In addition, it compares the learning technology used, the format of e-learning delivery, types of e-learning, e-learning delivery features, learning content, content type, knowledge type, and learning outcome components of the studies. 57 studies that matched the desired quality indicators were identified from 10 databases. The majority of the studies worked on knowledge, attitude, and satisfaction. The studies were based on various methodologies and were carried out in a wide range of subject areas. There is a need to conduct more studies to compare different methods, tools, and pedagogical approaches in future studies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Abraham, R. R., & Komattil, R. (2017). Heutagogic approach to developing capable learners. Medical Teacher, 39(3), 295–299. https://doi.org/10.1080/0142159X.2017.1270433
Al-Balas, M., Al-Balas, H. I., Jaber, H. M., Obeidat, K., Al-Balas, H., Aborajooh, E. A., Al-Taher, R., & Al-Balas, B. (2020). Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: Current situation, challenges, and perspectives. BMC Medical Education, 20(1), 1–7. https://doi.org/10.1186/s12909-020-02257-4
Albayrak, E. (2014). The effects of design factors on students' success and test anxiety in electronic tests. International Online Journal of Educational Sciences, 6(2), 460–474. https://doi.org/10.15345/iojes.2014.02.017
Almarzooq, Z. I., Lopes, M., & Kochar, A. (2020). Virtual learning during the COVID-19 pandemic: A disruptive technology in graduate medical education. Journal of the American College of Cardiology, 75(20), 2635–2638. https://doi.org/10.1016/j.jacc.2020.04.015
Al-Shorbaji, N., Atun, R., Car, J., Majeed, A., Wheeler. E. (Eds.) (2015). E-learning for undergraduate health professional education: A systematic review informing a radical transformation of health workforce development. World Health Organization. Retrieved January 7, 2022, from https://apps.who.int/iris/handle/10665/330089
Audigé, L., Bhandari, M., & Kellam, J. (2004). How reliable are reliability studies of fracture classifications? A systematic review of their methodologies. Acta Orthopaedica Scandinavica, 75(2), 184–194. https://doi.org/10.1080/00016470412331294445
Augestad, K. M., & Lindsetmo, R. O. (2009). Overcoming distance: Video-conferencing as a clinical and educational tool among surgeons. World Journal of Surgery, 33(7), 1356–1365. https://doi.org/10.1007/s00268-009-0036-0
Azlan, C. A., Wong, J. H. D., Tan, L. K., Huri, M. S. N. A., Ung, N. M., Pallath, V., Tan, C. P. L., Yeong, C. H., & Ng, K. H. (2020). Teaching and learning of postgraduate medical physics using internet-based e-learning during the COVID-19 pandemic: A case study from Malaysia. Physica Medica, 80, 10–16. https://doi.org/10.1016/j.ejmp.2020.10.002
Barr, K. P., & Massagli, T. L. (2014). New challenges for the graduate medical educator: Implementing the milestones. American Journal of Physical Medicine & Rehabilitation, 93(7), 624–631. https://doi.org/10.1097/PHM.0000000000000073
Barteit, S., Guzek, D., Jahn, A., Bärnighausen, T., Jorge, M. M., & Neuhann, F. (2020). Evaluation of e-learning for medical education in low-and middle-income countries: A systematic review. Computers & Education, 145, 103726. https://doi.org/10.1016/j.compedu.2019.103726
Baydas, O., Kucuk, S., Yilmaz, R. M., Aydemir, M., & Goktas, Y. (2015). Educational technology research trends from 2002 to 2014. Scientometrics, 105, 709–725. https://doi.org/10.1007/s11192-015-1693-4
Buckley, S., Coleman, J., Davison, I., Khan, K. S., Zamora, J., Malick, S., Morley, D., Pollard, D., Ashcroft, T., Popovic, C., & Sayers, J. (2009). The educational effects of portfolios on undergraduate student learning: A Best Evidence Medical Education (BEME) systematic review. BEME Guide No. 11. Medical Teacher, 31(4), 282–298. https://doi.org/10.1080/01421590902889897
Butterworth, K., Hayes, B., & Zimmerman, M. (2011). Remote and rural: Do mentors enhance the value of distance learning continuing medical education? Education for Health, 24(3), 539. Retrieved March 2, 2022, from https://www.educationforhealth.net/text.asp?2011/24/3/539/101423
Büyüköztürk, Ş, Kılıç Çakmak, E., Akgün, Ö. E., Karadeniz, Ş, & Demirel, F. (2016). Qualitative studies. Scientific research methods (11th ed.). Pegem A. Publishing.
Childs, S., Blenkinsopp, E., Hall, A., & Walton, G. (2005). Effective e-learning for health professionals and students—barriers and their solutions: A systematic review of the literature—findings from the HeXL project. Health Information & Libraries Journal, 22, 20–32. https://doi.org/10.1111/j.1470-3327.2005.00614.x
Clark, W., & Luckin, R. (2013). What the research says: iPads in the classroom. Institute of Education University of London. Retrieved January 11, 2022, from https://scholar.google.com/scholar_lookup?hl=en&publication_year=2013&author=W.+Clark&author=R.+Luckin&title=What+the+research+says.+iPads+in+the+classroom
Colbert, C. Y., French, J. C., Herring, M. E., & Dannefer, E. F. (2017). Fairness: The hidden challenge for competency-based postgraduate medical education programs. Perspectives on Medical Education, 6(5), 347–355. https://doi.org/10.1007/s40037-017-0359-8
Cook, D. A. (2007). Web-based learning: Pros, cons and controversies. Clinical Medicine, 7(1), 37. https://doi.org/10.7861/clinmedicine.7-1-37
Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008). Internet-based learning in the health professions: A meta-analysis. JAMA, 300(10), 1181–1196. https://doi.org/10.1001/jama.300.10.1181
Cullen, M. W., Geske, J. B., Anavekar, N. S., McAdams, J. A., Beliveau, M. E., Ommen, S. R., & Nishimura, R. A. (2019). Reinvigorating continuing medical education: Meeting the challenges of the digital age. Mayo Clinic Proceedings, 94(12), 2501–2509. Elsevier. https://doi.org/10.1016/j.mayocp.2019.07.004
Custers, E. J. (2010). Long-term retention of basic science knowledge: A review study. Advances in Health Sciences Education, 15(1), 109–128. https://doi.org/10.1007/s10459-008-9101-y
Custers, E. J., & Ten Cate, O. T. (2011). Very long-term retention of basic science knowledge in doctors after graduation. Medical Education, 45(4), 422–430. https://doi.org/10.1111/j.1365-2923.2010.03889.x
Daniel, M., Gordon, M., Patricio, M., Hider, A., Pawlik, C., Bhagdev, R., Ahmad, S., Alston, S., Park, S., Pawlikowska, T., Rees, E., Doyle, A. J., Pammi, M., Thammasitboon, S., Haas, M., Peterson, W., Lew, M., Khamees, D., Spadafore, M., … Stojan, J. (2021). An update on developments in medical education in response to the COVID-19 pandemic: A BEME scoping review: BEME Guide No. 64. Medical Teacher, 43(3), 253–271. https://doi.org/10.1080/0142159X.2020.1864310
De Leeuw, R. A., Westerman, M., Nelson, E., Ket, J. C. F., & Scheele, F. (2016). Quality specifications in postgraduate medical e-learning: An integrative literature review leading to a postgraduate medical e-learning model. BMC Medical Education, 16(1), 1–10. https://doi.org/10.1186/s12909-016-0700-7
De Leeuw, R., De Soet, A., Van Der Horst, S., Walsh, K., Westerman, M., & Scheele, F. (2019). How we evaluate postgraduate medical e-learning: Systematic review. JMIR Medical Education, 5(1), e13128. https://doi.org/10.1186/s12909-016-0700-7
Dhir, S. K., Verma, D., Batta, M., & Mishra, D. (2017). E-learning in medical education in India. Indian Paediatrics, 54(10), 871–877. Retrieved October 1, 2021, from https://link.springer.com/article/10.1007/s13312-017-1152-9
Dong, L., Gao, T., Zheng, W., Zeng, K., & Wu, X. (2021). E-learning for continuing medical education of neurology residents. Mind, Brain, and Education, 15(1), 48–53. https://doi.org/10.1111/mbe.12271
Edigin, E., Eseaton, P. O., Shaka, H., Ojemolon, P. E., Asemota, I. R., & Akuna, E. (2020). Impact of COVID-19 pandemic on medical postgraduate training in the United States. Medical Education Online, 25(1), 1774318. https://doi.org/10.1080/10872981.2020.1774318
Ellaway, R., & Masters, K. (2008). AMEE Guide 32: E-learning in medical education Part 1: Learning, teaching and assessment. Medical Teacher, 30(5), 455–473. https://doi.org/10.1080/01421590802108331
Eskander, J., Rajaguru, P. P., & Greenberg, P. B. (2021). Evaluating wellness interventions for resident physicians: A systematic review. Journal of Graduate Medical Education, 13(1), 58–69. https://doi.org/10.4300/JGME-D-20-00359.1
European Health Forum Gastein (2010, October 11). Possible shortage of up to two million health care workers by 2020: EU taking action to prevent impending crisis in providing health care. Retrieved January 3, 2022, from http://pr.euractiv.com/pr/possible-shortage-two-million-health-care-workers-2020-eu-taking-action-prevent-impending-crisis
Frawley, T., Goh, E., & Law, R. (2019). Quality assurance at hotel management tertiary institutions in Australia: An insight into factors behind domestic and international student satisfaction. Journal of Hospitality and Tourism Education, 31(1), 1–9. https://doi.org/10.1080/10963758.2018.1480961
Gandhi, V., Al-Hadithy, N., Göpfert, A., Knight, K., van Hove, M., & Hockey, P. (2020). Integrating sustainability into postgraduate medical education. Future Healthcare Journal, 7(2), 102. https://doi.org/10.7861/fhj.2020-0042
Goktas, Y., Kucuk, S., Aydemir, M., Telli, E., Arpacik, O., Yildirim, G., & Reisoglu, I. (2012). Educational technology research trends in Turkey: A content analysis of the 2000–2009 decade. Educational Sciences: Theory & Practice, 12(1), 177–199.
Gordon, M., Patricio, M., Horne, L., Muston, A., Alston, S. R., Pammi, M., Thammasitboon, S., Park, S., Pawlikowska, T., Rees, E. L., Doyle, A. J., & Daniel, M. (2020). Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME Guide No. 63. Medical Teacher, 42(11), 1202–1215. https://doi.org/10.1080/0142159X.2020.1807484
Graham, C. R., Woodfield, W., & Harrison, J. B. (2013). A framework for institutional adoption and implementation of blended learning in higher education. The Internet and Higher Education, 18, 4–14. https://doi.org/10.1016/j.iheduc.2012.09.003
Hew, K. F., Kale, U., & Kim, N. (2007). Past research in instructional technology: Results of a content analysis of empirical studies published in three prominent instructional technology journals from the year 2000 through 2004. Journal of Educational Computing Research, 36(3), 269–300. https://doi.org/10.2190/K3P8-8164-L56J-33W4
Higgins, J. P. T., & Green, S. (2011). Cochrane handbook for systematic reviews of interventions (Vol. 4). John Wiley & Sons.
Ituma, A. (2011). An evaluation of students’ perceptions and engagement with e-learning components in a campus-based university. Active Learning in Higher Education, 12(1), 57–68. https://doi.org/10.1177/1469787410387722
Kambhampati, S. B. S., Vaishya, R., & Vaish, A. (2020). Unprecedented surge in publications related to Covid-19 in the first three months of pandemic: A bibliometric analytic report. Journal of Clinical Orthopaedics and Trauma, 11(3), 304–306. https://doi.org/10.1016/j.jcot.2020.04.030
Kanneganti, A., Lim, K. M., Chan, G. M., Choo, S. N., Choolani, M., Ismail-Pratt, I., & Logan, S. J. (2020). Pedagogy in a pandemic: COVID-19 and virtual continuing medical education (vCME) in obstetrics and gynaecology. Acta Obstetricia Et Gynecologica Scandinavica, 99(6), 692. https://doi.org/10.1111/aogs.13885
Kerfoot, B. P., & Baker, H. (2012). An online spaced-education game for global continuing medical education: A randomized trial. Annals of Surgery, 256(1), 33–38. https://doi.org/10.1097/SLA.0b013e31825b3912
Kerfoot, B. P., & Brotschi, E. (2009). Online spaced education to teach urology to medical students: A multi-institutional randomized trial. The American Journal of Surgery, 197(1), 89–95. https://doi.org/10.1016/j.amjsurg.2007.10.026
Khamees, D., Peterson, W., Patricio, M., Pawlikowska, T., Commissaris, C., Austin, A., Davis, M., Spadafore, M., Griffith, M., Hider, A., Pawlik, C., Stojan, J., Grafton-Clarke, C., Uraiby, H., Thammasitboon, S., Gordon, M., & Daniel, M. (2022). Remote learning developments in postgraduate medical education in response to the COVID-19 pandemic: A BEME systematic review: BEME Guide No 71. Medical Teacher, 44(5), 466–485. https://doi.org/10.1080/0142159X.2022.2040732
Khorsan, R., & Crawford, C. (2014). External validity and model validity: A conceptual approach for systematic review methodology. Evidence-Based Complementary and Alternative Medicine. https://doi.org/10.1155/2014/694804
Kucuk, S., Aydemir, M., Yildirim, G., Arpacik, O., & Goktas, Y. (2013). Educational technology research trends in Turkey from 1990 to 2011. Computers & Education, 68, 42–60. https://doi.org/10.1016/j.compedu.2013.04.016
Lapolla, P., & Mingoli, A. (2020). COVID-19 changes medical education in Italy: Will other countries follow? Postgraduate Medical Journal, 96(1137), 375–376. https://doi.org/10.1136/postgradmedj-2020-137876
Lee, T. C., Murray, J., & McDonald, E. G. (2019). An online educational module on transfusion safety and appropriateness for resident physicians: A controlled before–after quality-improvement study. CMAJ Open, 7(3), E492–E496. https://doi.org/10.9778/cmajo.20180211
Lewis, K. O., Cidon, M. J., Seto, T. L., Chen, H., & Mahan, J. D. (2014). Leveraging e-learning in medical education. Current Problems in Paediatric and Adolescent Health Care, 44(6), 150–163. https://doi.org/10.1016/j.cppeds.2014.01.004
Lin, H. C., & Hwang, G. J. (2019). Research trends of flipped classroom studies for medical courses: A review of journal publications from 2008 to 2017 based on the technology-enhanced learning model. Interactive Learning Environments, 27(8), 1011–1027. https://doi.org/10.1080/10494820.2018.1467462
Links, M. J. (2018). Beyond competency-based continuing professional development. Medical Teacher, 40(3), 253–258. https://doi.org/10.1080/0142159X.2017.1401219
Malek, M., Mohammadi, S., & Attarchi, M. (2011). Occupational stress and influencing factors, in medical residents of one of the educational hospitals of Tehran University of Medical Sciences. Razi Journal of Medical Sciences, 18(87), 24–35.
Masood, M. (2004). Trends and issues as reflected in traditional educational technology literature: A content analysis (Unpublished doctoral dissertation). Indiana University.
Mathes, E. F., Frieden, I. J., Cho, C. S., & Boscardin, C. K. (2014). Randomized controlled trial of spaced education for paediatric residency education. Journal of Graduate Medical Education, 6(2), 270–274. https://doi.org/10.4300/JGME-D-13-00056.1
Mazmanian, P. E., & Davis, D. A. (2002). Continuing medical education and the physician as a learner: Guide to the evidence. JAMA, 288(9), 1057–1060. https://doi.org/10.1001/jama.288.9.1057
McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence based inquiry (7th ed.). Longman.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
Montrieux, H., Van Der Linde, R., Schellens, T., & De Marez, L. (2015). Teaching and learning with mobile technology: A qualitative explorative study about the introduction of tablet devices in secondary education. PLoS ONE, 10(12), e0144008. https://doi.org/10.1371/journal.pone.0144008
Naciri, A., Radid, M., Kharbach, A., & Chemsi, G. (2021). E-learning in health professions education during the COVID-19 pandemic: A systematic review. Journal of Educational Evaluation for Health Professions, 18(27). https://doi.org/10.3352/jeehp.2021.18.27
O’Doherty, D., Dromey, M., Lougheed, J., Hannigan, A., Last, J., & McGrath, D. (2018). Barriers and solutions to online learning in medical education–an integrative review. BMC Medical Education, 18(1), 1–11. https://doi.org/10.1186/s12909-018-1240-0
Orozalieva, G., Loutan, L., Azimova, A., Baroffio, A., Heller, O., Lab, B., ..., & Beran, D. (2021). Reforms in medical education: Lessons learnt from Kyrgyzstan. Global Health Action, 14(1), 1944480. https://doi.org/10.1080/16549716.2021.1944480
Pahinis, K., Stokes, C. W., Walsh, T. F., Tsitrou, E., & Cannavina, G. (2008). A blended learning course taught to different groups of learners in a dental school: Follow-up evaluation. Journal of Dental Education, 72(9), 1048–1057. https://doi.org/10.1002/j.0022-0337.2008.72.9.tb04579.x
Reeves, T. C. (1995). Questioning the questions of instructional technology research. In M. Simonson, & M. Anderson (eds.). Proceedings of the 1995 annual national convention of the association for educational communications and technology (AECT 1995), Anaheim, CA (pp 459–470). Association for Educational Communications and Technology.
Regmi, K., & Jones, L. (2020). A systematic review of the factors–enablers and barriers–affecting e-learning in health sciences education. BMC Medical Education, 20(1), 1–18. https://doi.org/10.1186/s12909-020-02007-6
Rosser, J. C., Young, S. M., & Klonsky, J. (2007). Telementoring: An application whose time has come. Surgical Endoscopy, 21(8), 1458–1463. https://doi.org/10.1007/s00464-007-9263-3
Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of e-learning in medical education. Academic Medicine, 81(3), 207–212. Retrieved January 3, 2022, from https://journals.lww.com/academicmedicine/fulltext/2006/03000/theimpactofelearninginmedicaleducation.2.aspx?casa_token=iRlPTGH2Dn0AAAAA:jcUdQLUctNvv3vds4GOXU8byHPIjrcUgYyVvNrc2O0p3B4_dH_kfqr6wuJO-Je-GlJHB2btdoUyoeQ4NO8hs1E_bhnw
Santos, G. N. M., Leite, A. F., Figueiredo, P. T. D. S., Pimentel, N. M., Flores-Mir, C., de Melo, N. S., Guerra, E. N. S., & De Luca Canto, G. (2016). Effectiveness of e-learning in oral radiology education: A systematic review. Journal of Dental Education, 80(9), 1126–1139. https://doi.org/10.1002/j.0022-0337.2016.80.9.tb06195.x
Schittek, M., Mattheos, N., Lyon, H. C., & Attström, R. (2001). Computer assisted learning. A review. European Journal of Dental Education: Review Article, 5(3), 93–100. https://doi.org/10.1034/j.1600-0579.2001.050301.x
Sozbilir, M., & Kutu, H. (2008). Development and current status of science education research in Turkey. Essays in Education, 24(1), 3.
Star, J. R. (2005). Reconceptualizing procedural knowledge. Journal for Research in Mathematics Education, 36(5), 404–411. https://doi.org/10.2307/30034943
Te Pas, E., Wieringa-de Waard, M., Blok, B. S., Pouw, H., & van Dijk, N. (2016). Didactic and technical considerations when developing e-learning and CME. Education and Information Technologies, 21(5), 991–1005. https://doi.org/10.1007/s10639-014-9364-2
Vuchkova, J., Maybury, T., & Farah, C. S. (2012). Digital interactive learning of oral radiographic anatomy. European Journal of Dental Education, 16(1), e79–e87. https://doi.org/10.1111/j.1600-0579.2011.00679.x
Wang, S., & Dai, M. (2020). Status and situation of postgraduate medical students in China under the influence of COVID-19. Postgraduate Medical Journal, 96(1142), 728–730. https://doi.org/10.1136/postgradmedj-2020-137763
Wang, Z. Y., Zhang, L. J., Liu, Y. H., Jiang, W. X., Tang, S. L., & Liu, X. Y. (2021). Process evaluation of e-learning in continuing medical education: evidence from the China-gates foundation tuberculosis control program. Infectious Diseases of Poverty, 10(1), 1–11. https://doi.org/10.1186/s40249-021-00810-x
WHO (World Health Organization) (2023). OpenWHO. Retrieved June 30, 2023, from https://openwho.org/
Wilkerson, J. R., & Lang, W. S. (2003). Portfolios, the Pied Piper of teacher certification assessments: Legal and psychometric issues. Education Policy Analysis Archives, 11, 45–45. https://doi.org/10.14507/epaa.v11n45.2003
Wu, A. S., & Zeshan, M. (2020). Medical education in the time of COVID-19: A literature review on e-learning. Journal of the American Academy of Child and Adolescent Psychiatry, 59(10), S253. https://doi.org/10.1016/j.jaac.2020.08.418
Yeh, D. D., & Park, Y. S. (2015). Improving learning efficiency of factual knowledge in medical education. Journal of Surgical Education, 72(5), 882–889. https://doi.org/10.1016/j.jsurg.2015.03.012
Zhang, H., & Shaw, R. (2020). Identifying research trends and gaps in the context of COVID-19. International Journal of Environmental Research and Public Health, 17(10), 3370. https://doi.org/10.3390/ijerph17103370
Zufic, J., & Kalpic, D. (2009). More efficient e-learning through design: color of text and background. In T. Bastiaens, J. Dron & C. Xin (Eds.), Proceedings of E-learn 2009--world conference on E-learning in corporate, government, healthcare, and higher education (pp. 3314–3319). Vancouver: Association for the Advancement of Computing in Education (AACE). Retrieved July 15, 2023 from https://www.learntechlib.org/primary/p/32959/
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest/Competing interests
The authors declare that they have no conflict of interest.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hopcan, S., Polat, E. & Albayrak, E. Research trends in e-learning practices for postgraduate medical education: A systematic review. Educ Inf Technol 29, 5921–5945 (2024). https://doi.org/10.1007/s10639-023-12035-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-023-12035-6