Abstract
Semantic-based computer-assisted automated question generator has been increasingly popular as a tool for creating personalized assessment questions.Various question generator tools have been proposed, such as those which generate structured question from a text file, generate Multiple Choice Question (MCQ) from a text file or from ontology-based knowledge representation. A comparison framework and evaluation methodology is required to evaluate different question generator tools. This paper discusses the requirement and criteria to carry out performance comparison of different MCQ generator. A feature comparison of Question Generation (QG) tool, namely Mimos-QG with the existing QG tools is presented. We have evaluated our QG tool based on several standard criteria such as the correctness of (a) distractor generation, (b) answer choice grouping strategy and (c) syntactical and pedagogical quality on three different domain ontologies. The experimental result indicated that Mimos-QG is capable of producing good quality direct type and grouping type multiple choice questions.
The authors developed this system under the Semantic Technology Research & Development initiative funded by the Ministry of Science, Technology & Innovation (MOSTI), Malaysia.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Andrenucci, A., Sneiders, E.: Automated Question Answering: Review of the Main Approaches. In: 3rd International Conf. on Information Technology and Applications, pp. 514–519 (2005)
McGough, J., Mortensen, J., Johnson, J., Fadali, S.: A Web-Based Testing System with Dynamic Question Generation. In: 31st ASEE/IEEE Frontiers in Education Conference, vol. 3, pp. 3–23 (2001)
Wang, W.M., Hao, T.Y., Liu, W.Y.: Automatic Question Generation for Learning Evaluation in Medicine. In: Leung, H., Li, F., Lau, R., Li, Q. (eds.) ICWL 2007. LNCS, vol. 4823, pp. 242–251. Springer, Heidelberg (2008)
Jonathan, C., Gwen, A., Eskenazi, E.M.: Automatic question generation for vocabulary assessment. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, Canada, pp. 819–826 (2005)
Burdescu, D.D., Mihaescu, M.C.: Building a decision support system for students by using concept maps. Proceedings of International Conference on Enterprise Information Systems (ICEIS 2008), Spain (2008)
Papasalouros, A., Kotis, K., Kanaris, K.: Automatic generation of multiple-choice questions from domain ontologies. In: IADIS e-Learning 2008 Conference, Amsterdam (2008)
Sicilia, M.A., Barriocanal, E.G.: On the convergence of formal ontologies and standardized e-learning. International Journal of Distance Education 3(2), 13–29 (2005)
Knight, C., Gasevic, D., Richards, G.: An ontology-based framework for bridging learning design and learning content. Educational Technology & Society 9(1), 23–27 (2006)
OWL Web Ontology Language Overview, http://www.w3.org/TR/owl-features/
Tosic, M., Cubric, M.: SeMCQ – Protégé Plugin for Automatic Ontology-Driven Multiple Choice Question Tests Generation. In: 11th Intl. Protégé Conference, Poster and Demo Session (2009)
Cubric, M.: Tosic. M.: Towards automatic generation of eAssessment using semantic web technologies. In: International Computer Assisted Assessment (CAA) Conference, Southampton (2010)
Stanescu, L., Spahiu, C.S., Ion, A., Spahiu, A.: Question Generation for Learning Evaluation. In: International Multi-Conference on Computer Science and Information Technology, pp. 509–513 (2008)
Heilman, M., Noah, S.A.: Question Generation via overgenerating transformations and ranking. In: The 14th Annual Conference on Artificial Intelligence in Education Workshop on Question Generation (2009)
Ishaq, A., Nanjappa, S.B., Acharya, S.: Method and system for automatically generating questions for a programming. Infosys Technologies Limited, assignee. Patent 12/319566 (2009)
Tsumori, S., Kaijiri, K.: System Design for Automatic Generation of Multiple Choice Questions Adapted to Students’ Understanding. In: Proceedings of the 8th International Conference on Information Technology Based Higher Education and Training, Japan, pp. 541–546 (2007)
Anderson, L.W., Sosniak, L.A.: Bloom’s Taxonomy: A forty-year retrospective The National Society For The Study Of Education (1994)
Tan, S.Y., Lukose, D.: System and method for ontology-based question generation. Mimos Berhad (Patent - Pending)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tan, SY., Kiu, CC., Lukose, D. (2012). Evaluating Multiple Choice Question Generator. In: Lukose, D., Ahmad, A.R., Suliman, A. (eds) Knowledge Technology. KTW 2011. Communications in Computer and Information Science, vol 295. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32826-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-32826-8_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32825-1
Online ISBN: 978-3-642-32826-8
eBook Packages: Computer ScienceComputer Science (R0)