Abstract
Problem based learning is becoming widely popular as an effective teaching method in medical education. Paying individual attention to a small group of students in medical problem-based learning (PBL) can place burden on the workload of medical faculty whose time is very costly. Intelligent tutoring systems offer a cost effective alternative in helping to train the students, but they are typically prone to brittleness and the knowledge acquisition bottleneck. Existing tutoring systems accept a small set of approved solutions for each problem scenario stored into the system. Plausible student solutions that lie outside the scope of the explicitly encoded ones receive little acknowledgment from the system. Tutoring hints are also confined to the knowledge space of the approved solutions, leading to brittleness in the tutoring approach. We report the clinical reasoning gains off a tutoring system for medical PBL that employs and represents the widely available medical knowledge source UMLS as the domain ontology. We exploit the structure of the concept hierarchy to expand the plausible solution space and generate hints based on the problem solving context. Evaluation of student learning outcomes led to highly significant learning gains (Mann-Whitney, p < 0.001).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Al-Mubaid, H., & Nguyen, H.A. (2006). A cluster based approach for semantic similarity in the biomedical domain. In Proceedings of the 28th IEEE EMBS annual international conference. New York, USA.
Anderson, J.R., Corbett, A., Koedinger, K., Pelletier, R. (1996). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167–207.
Barrows, H. (1986). A taxonomy of problem-based learning methods. Medical Education, 20, 481–486.
Barrows, H.S. (1996). Problem-based learning in medicine and beyond: A brief overview. In Wilkerson, L. & Gislaers, W.H. (eds.) Bringing problem-based learning to higher education: Theory and practice (pp. 3–12). San Francisco, CA: Jossey-Bass.
Crowley, R., & Medvedeva, O. (2006). An intelligent tutoring system for visual classification problem solving. Artificial Intelligence in Medicine, 36(1), 85–117.
Day, M.Y., Lu, C., Yang, J.D., Chiou, G., Ong, C.S., Hsu, W. (2005). Designing an ontology-based intelligent tutoring agent with instant messaging. In Fifth IEEE international conference on advanced learning technologies (pp. 318–320).
Feltovich, P.J., & Barrows, H.S. (1984). Issues of generality in medical problem solving. In Schmidt, H.G. & De Volder, M.L. (eds.) Tutorials in problem-based learning: A new direction in teaching the health professions. The Netherlands: Van Gorcum.
Groves, M., Schott, I., Alexander, H. (2002). Assessing clinical reasoning: A method to monitor its development in PBL curriculum. Medical Teacher, 5, 507–515.
Kazi, H., Haddawy, P., Suebnukarn, S. (2008). Expanding the space of plausible solutions for robustness in an intelligent tutoring system. In Proceedings of 9th international conference on intelligent tutoring systems (pp. 583–592). Montreal, Canada.
Kazi, H., Haddawy, P., Suebnukarn, S. (2009). Expanding the space of plausible solutions in a medical tutoring system for problem based learning. International Journal of Artificial Intelligence in Education, 19(3), 309–334.
Kazi, H., Haddawy, P., Suebnukarn, S. (2011). METEOR: Medical Tutor Employing Ontology for Robustness. In Proceedings of the 16th international conference on intelligent user interfaces (pp. 247–256). Palo Alto, USA.
Kazi, H., Haddawy, P., Suebnukarn, S. (2012). Employing UMLS for generating hints in a tutoring system for medical problem-based learning. Journal of Biomedical Informatics, 45(3), 557–565.
Kodaganallur, V., Weitz, R.R., Rosenthal, D. (2005). A comparison of model-tracing and constraint-based intelligent tutoring paradigms. International Journal of Artificial Intelligence in Education, 15(2), 117–144.
Koschmann, T., Kelson, A., Feltovich, P., Barrows, H.S. (1996). Computer-supported problem-based learning: A principled approach to the use of computers in collaborative learning. In Koschmann, T. (ed.) CSCL: Theory and practice of an emerging paradigm (pp. 83–124). Mahwah, NJ: Erlbaum.
Lee, C.H., Seu, J.H., Evens, M.W. (2002). Building an ontology for CIRCSIM tutor. In Proceedings of the 13th midwest AI and Cognitive science society conference (pp. 161–168). MAICSS.
Mitrovic, A. (1998). Experiences in implementing constraint-based modelling in SQL-Tutor. In Proceedings of the 4th international conference on intelligent tutoring systems (pp. 414–423).
Mitrovic, A., Koedinger, K.R., Martin, B. (2003). A comparative analysis of cognitive tutoring and constraint-based modeling. In User modeling 2003: 9th international conference (UM 2003), 22–26 June 2003. Lecture notes in computer science, 2702 (pp. 313–322).
Murray, T. (1999). Authoring intelligent tutoring systems: an analysis of the state of the art. International Journal of Artificial Intelligence in Education, 10, 98–129.
Pople, H.E. Jr. (1982). Heuristic methods for imposing structure on ill-structured problems: The structuring of medical diagnostics. In Szolovits, P. (ed.) Artificial intelligence in medicine, Chapter 5. Boulder, Colorado, USA: Westview Press.
Sowa, J.F. (2000). Knowledge representation: Logical, philosophical and computational foundations. Brooks/Cole: Belmont.
Suebnukarn, S., & Haddawy, P. (2006). Modeling individual and collaborative problem-solving in medical problem-based learning. User Modeling and User-Adapted Interaction 16(3–4), 211–248.
Suebnukarn, S., & Haddawy, P. (2007). COMET: A collaborative tutoring system for medical problem-based learning. IEEE Intelligent Systems, 22(4), 70–77.
Suraweera, P., Mitrovic, A., & Martin, B. (2005). A knowledge acquisition system for constraint based intelligent tutoring systems. In Conference on artificial intelligence in education.
Unified Medical Language System. (2012). http://www.nlm.nih.gov/research/umls
Acknowledgements
We would like to thank the students and general practitioners at Thammasat University for their time and effort during the system evaluations.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kazi, H., Haddawy, P. & Suebnukarn, S. Clinical reasoning gains in medical PBL: an UMLS based tutoring system. J Intell Inf Syst 41, 269–284 (2013). https://doi.org/10.1007/s10844-013-0244-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10844-013-0244-9