Abstract
As lifelong learning becomes increasingly important in our society, mechanisms allowing students to evaluate their progress must be provided. A commonly used and widely accepted feedback mechanism is the multiple-choice test. Manual creation of multiple choice questions is often a time consuming process involving many iterations of trail and error. Using text processing and natural language processing techniques, automated multiple choice question generation, in recent years, is getting much closer to reality than ever. However, one of the most difficult tasks in both manual creation and automated generation of this kind of tests is the creation of distractors, because unsuitable distractors allow students to easily guess the correct answer, which counteracts the goal of these questions. In this paper, we investigated the desired properties of distractors and identified relevant text processing algorithms, specifically, latent semantic analysis and stylometry, for distractor selection. The refined distrators are compared with baseline distrators generated by our existing Automated Question Creator (AQC). Our preliminary evaluation shows that this novel combined approach produces distractors with a higher quality than those of the baseline AQC system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abbasi, A., Chen, H.: Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace. ACM Trans. Inf. Syst. 26, 7:1–7:29 (2008)
Aldabe, I., Maritxalar, M.: Automatic Distractor Generation for Domain Specific Texts. In: Loftsson, H., Rögnvaldsson, E., Helgadóttir, S. (eds.) IceTAL 2010. LNCS, vol. 6233, pp. 27–38. Springer, Heidelberg (2010), http://dx.doi.org/10.1007/978-3-642-14770-8_5
Alias-i: Lingpipe 4.0.0 (2010), http://alias-i.com/lingpipe (access date: December 16, 2010)
Bransford, J.D., Brown, A.L., Cocking, R.R. (eds.): How People Learn: Brain, Mind, Experience, and School: Expanded Edition. The National Academies Press (2000)
Brown, J.C., Frishkoff, G.A., Eskenazi, M.: Automatic question generation for vocabulary assessment. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT 2005, pp. 819–826. Association for Computational Linguistics, Morristown (2005), http://dx.doi.org/10.3115/1220575.1220678
Chang, V., Gütl, C.: Generation y learning in the 21st century: Integration of virtual worlds and cloud computing services. In: Abas, Z.W., Jung, I., Luca, J. (eds.) Proceedings of Global Learn Asia Pacific 2010, pp. 1888–1897. AACE, Penang (2010)
Chen, C.Y., Liou, H.C., Chang, J.S.: Fast: an automatic generation system for grammar tests. In: Proceedings of the COLING/ACL on Interactive Presentation Sessions, pp. 1–4. Association for Computational Linguistics, Morristown (2006), http://dx.doi.org/10.3115/1225403.1225404
Cunningham, H., Maynard, D., Bontcheva, K., Tablan, V.: GATE: A framework and graphical development environment for robust NLP tools and applications. In: Proceedings of the 40th Anniversary Meeting of the Association for Computational Linguistics (2002)
Gütl, C.: Automatic limited-choice and completion test creation, assessment and feedback in modern learning processes. In: LRN Conference 2008, Guatemala (February 2008)
Gütl, C., Chang, V.: Ecosystem-based theoretical models for learning in environments of the 21st century. International Journal of Emerging Technologies in Learning (iJET) 3(1) (2008)
Gütl, C., Lankmayr, K., Weinhofer, J.: Enhanced approach of automatic creation of test items to foster modern learning setting. In: Proc. of the 9th European Conference on e-Learning, pp. 225–234 (November 2010)
Gütl, C., Lankmayr, K., Weinhofer, J., Höfler, M.: Enhanced approach of automatic test item creation to foster modern e-education. Electronic Journal of e-Learning (2011) (to appear)
Haladyna, T.M.: Developing and Validating Multiple-Choice Test Items, 3rd edn. Lawrence Erlbaum Associates (2004)
Hendrikson, C.: Project management for construction (2000), http://pmbook.ce.cmu.edu/index.html (access date: November 17, 2010)
Holmes, D.: Authorship attribution. Computers and the Humanities 28, 87–106 (1994), http://dx.doi.org/10.1007/BF01830689
Holmes, D.: The Evolution of Stylometry in Humanities Scholarship. Literary and Linguistic Computing 13(3), 111–117 (1998), http://llc.oxfordjournals.org/content/13/3/111.abstract
Klein, S., Simmons, R.F.: A computational approach to grammatical coding of english words. J. ACM 10, 334–347 (1963), http://doi.acm.org/10.1145/321172.321180
Landauer, T.K., Foltz, P.W., Laham, L.: An introduction to latent semantic analysis. Discourse Processes 25, 259–284 (1998)
Mitkov, R., Ha, L.A.: Computer-aided generation of multiple-choice tests. In: Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing, vol. 2, pp. 17–22. Association for Computational Linguistics, Morristown (2003), http://dx.doi.org/10.3115/1118894.1118897
Mitkov, R., Ha, L.A., Varga, A., Rello, L.: Semantic similarity of distractors in multiple-choice tests: extrinsic evaluation. In: GEMS 2009: Proceedings of the Workshop on Geometrical Models of Natural Language Semantics, pp. 49–56. Association for Computational Linguistics, Morristown (2009)
Naber, D.: A Rule-Based Style and Grammar Checker. Diploma thesis, University of Bielefeld (2003), http://www.languagetool.org/ (access date: January 2011)
Schutz, A.: Xtrak4me - extraction of keyphrases for metadata creation. SmILE: Semantic Information Systems and Language Engineering Group (2008), http://smile.deri.ie/projects/keyphrase-extraction (access date: November 17, 2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Moser, J.R., Gütl, C., Liu, W. (2012). Refined Distractor Generation with LSA and Stylometry for Automated Multiple Choice Question Generation. In: Thielscher, M., Zhang, D. (eds) AI 2012: Advances in Artificial Intelligence. AI 2012. Lecture Notes in Computer Science(), vol 7691. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35101-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-35101-3_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35100-6
Online ISBN: 978-3-642-35101-3
eBook Packages: Computer ScienceComputer Science (R0)