Skip to main content
SpringerLink
Log in

Automatic Generation of Multiple-Choice Items for Prepositions Based on Word2vec

  • Conference paper
  • First Online:
Book cover Data Science (ICPCSEE 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 902))

Abstract

The automatic generation of multiple-choice item (MI) has attracted amounts of attention. However, only a limited number of existing research address automatic MI generation for prepositions, and even fewer consider learners’ need in the generation process. In this paper, we propose an approach to generate preposition MIs suitable for non-native English learners of different language proficiency. First we select sentences with similar difficulty level to that of a given textbook as stems by using the sentence difficulty model we constructed. Then, we use the Word2vec model to retrieve a preposition list of distractor candidates where three of them are chosen as distractors. To validate the effectiveness of our approach, we produce four tests of preposition MIs at different difficulty levels and conduct a series of experiments regarding evaluations of stem difficulty, distractor plausibility and reliability. The experimental results show that our approach can generate preposition MIs targeting learners at different levels. The results of distractor plausibility and reliability also point to the validity of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.natcorp.ox.ac.uk.

  2. 2.

    https://nlp.stanford.edu/software/lex-parser.shtml.

  3. 3.

    https://wordnet.princeton.edu.

  4. 4.

    http://www.fit.vutbr.cz/~imikolov/rnnlm/.

  5. 5.

    http://nlp.stanford.edu/software/tagger.shtml.

  6. 6.

    http://www.wjx.cn.

References

  1. Chodorow, M., Tetreault, J.R., Han, N.R.: Detection of grammatical errors involving prepositions. In: Proceedings of the Fourth ACL-SIGSEM Workshop on Prepositions, pp. 25–30. Association for Computational Linguistics (2007)

    Google Scholar 

  2. Izumi, E., Uchimoto, K., Saiga, T., Supnithi, T., Isahara, H.: Automatic error detection in the Japanese learners’ English spoken data. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 2, pp. 145–148. Association for Computational Linguistics (2003)

    Google Scholar 

  3. Dahlmeier, D., Ng, H.T., Schultz, T.: Joint learning of preposition senses and semantic roles of prepositional phrases. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, pp. 450–458. Association for Computational Linguistics (2009)

    Google Scholar 

  4. Lee, J., Yeung, C.Y., Zeldes, A., Reznicek, M., Lüdeling, A., Webster, J.: Cityu corpus of essay drafts of English language learners: a corpus of textual revision in second language writing. Lang. Resour. Eval. 49(3), 659–683 (2015)

    Article  Google Scholar 

  5. Alderson, J.C.: Do corpora have a role in language assessment? In: Using Corpora for Language Research, pp. 248–259 (1996)

    Google Scholar 

  6. Heaton, J.B.: Writing English Language Tests: A Practical Guide for Teachers of English as a Second or Foreign Language. Longman Publishing Group, Harlow (1975)

    Google Scholar 

  7. Hoshino, A., Nakagawa, H.: A real-time multiple-choice question generation for language testing: a preliminary study. In: Proceedings of the Second Workshop on Building Educational Applications Using NLP, pp. 17–20. Association for Computational Linguistics (2005)

    Google Scholar 

  8. Smith, S., Avinesh, P.V.S., Kilgarriff, A.: Gap-fill tests for language learners: corpus-driven item generation. In: Proceedings of ICON-2010: 8th International Conference on Natural Language Processing, pp. 1–6. Macmillan Publishers (2010)

    Google Scholar 

  9. Sumita, E., Sugaya, F., Yamamoto, S.: Measuring non-native speakers’ proficiency of English by using a test with automatically-generated fill-in-the-blank questions. In: Proceedings of the Second Workshop on Building Educational Applications Using NLP, pp. 61–68. Association for Computational Linguistics (2005)

    Google Scholar 

  10. Lee, J., Sturgeon, D., Luo, M.: A CALL system for learning preposition usage. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 984–993 (2016)

    Google Scholar 

  11. Pino, J., Eskenazi, M.: Semi-automatic generation of cloze question distractors effect of students’ L1. In: International Workshop on Speech and Language Technology in Education (2009)

    Google Scholar 

  12. Brown, J.C., Frishkoff, G.A., Eskenazi, M.: Automatic question generation for vocabulary assessment. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 819–826. Association for Computational Linguistics (2005)

    Google Scholar 

  13. Lin, Y.C., Sung, L.C., Chen, M.C.: An automatic multiple-choice question generation scheme for English adjective understanding. In: Workshop on Modeling, Management and Generation of Problems/Questions in eLearning, the 15th International Conference on Computers in Education (ICCE 2007), pp. 137–142 (2007)

    Google Scholar 

  14. Mitkov, R., Ha, L.A.: Computer-aided generation of multiple-choice tests. In: Proceedings of the HLT-NAACL 03 Workshop on Building Educational Applications Using Natural Language Processing, vol. 2, pp. 17–22. Association for Computational Linguistics (2003)

    Google Scholar 

  15. Coniam, D.: A preliminary inquiry into using corpus word frequency data in the automatic generation of English language cloze tests. Calico J. 14(2–4), 15–33 (1997)

    Google Scholar 

  16. Agarwal, M., Mannem, P.: Automatic gap-fill question generation from text books. In: The Workshop on Innovative Use of NLP for Building Educational Applications, pp. 56–64. Association for Computational Linguistics (2012)

    Google Scholar 

  17. Lee, J., Seneff, S.: Automatic generation of cloze items for prepositions. In: Conference of the International Speech Communication Association, INTERSPEECH 2007, Antwerp, Belgium, pp. 2173–2176 (2007)

    Google Scholar 

  18. Sakaguchi, K., Arase, Y., Komachi, M.: Discriminative approach to fill-in-the-blank quiz generation for language learners. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pp. 238–242 (2013)

    Google Scholar 

  19. Chall, J.S., Dale, E.: Readability Revisited: The New Dale-Chall Readability Formula. Brookline Books, Northampton (1995)

    Google Scholar 

  20. Lennon, C., Burdick, H.: The lexile framework as an approach for reading measurement and success (2004). Electronic Publication on http://www.lexile.com

  21. Pitler, E., Nenkova, A.: Revisiting readability: a unified framework for predicting text quality. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 186–195. Association for Computational Linguistics (2008)

    Google Scholar 

  22. Tanaka-Ishii, K., Tezuka, S., Terada, H.: Sorting by readability. Comput. Linguist. 36(2), 203–227 (2010)

    Article  Google Scholar 

  23. Heilman, M., Collins-Thompson, K., Eskenazi, M.: An analysis of statistical models and features for reading difficulty prediction. In: Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications, pp. 71–79. Association for Computational Linguistics (2008)

    Google Scholar 

  24. François, T., Miltsakaki, E.: Do NLP and machine learning improve traditional readability formulas? In: The Workshop on Predicting and Improving Text Readability for Target Reader Populations, pp. 49–57. Association for Computational Linguistics (2012)

    Google Scholar 

  25. Perera, K.: The assessment of linguistic difficulty in reading material. Educ. Rev. 32(2), 151–161 (1980)

    Article  Google Scholar 

  26. Troia, G.A. (ed.): Instruction and Assessment for Struggling Writers: Evidence-Based Practices. Guilford Press, New York (2011)

    Google Scholar 

  27. Lin, D.: On the structural complexity of natural language sentences. In: Proceedings of the 16th Conference on Computational Linguistics, vol. 2, pp. 729–733. Association for Computational Linguistics (1996)

    Google Scholar 

  28. Liu, H.: Dependency distance as a metric of language comprehension difficulty. J. Cogn. Sci. 9(2), 159–191 (2008)

    Article  Google Scholar 

  29. Lu, Q., Xu, C., Liu, H.: Can chunking reduce syntactic complexity of natural lan-guages? Complexity 21(S2), 33–41 (2016)

    Article  MathSciNet  Google Scholar 

  30. Mishra, A., Bhattacharyya, P., Carl, M.: Automatically predicting sentence translation difficulty. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 346–351 (2013)

    Google Scholar 

  31. Specia, L., Shah, K., Souza, J.G., Cohn, T.: QuEst-A translation quality estimation framework. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 79–84 (2013)

    Google Scholar 

  32. Koehn, P., et al.: Open source toolkit for statistical machine translation: factored translation models and confusion network decoding. In: Final Report of the 2006 JHU Summer Workshop (2006)

    Google Scholar 

  33. Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)

    Article  Google Scholar 

Download references

Acknowledgments

This paper is supported by the National Science Foundation of China (No.61462045).

Author information

Authors and Affiliations

  1. School of Computer Information Engineering, Jiangxi Normal University, Nanchang, 330022, China

    Wenyan Xiao, Mingwen Wang, Chenlin Zhang, Yiming Tan & Zhiming Chen

  2. Jiangxi University of Science and Technology, Nanchang, 330013, China

    Wenyan Xiao

Authors
  1. Wenyan Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingwen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chenlin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yiming Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhiming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingwen Wang .

Editor information

Editors and Affiliations

  1. Zhengzhou University, Zhengzhou, Henan, China

    Qinglei Zhou

  2. Xidian University, Xi’an, Shaanxi, China

    Qiguang Miao

  3. Harbin Institute of Technology, Harbin, China

    Hongzhi Wang

  4. Harbin University of Science and Technology, Harbin, China

    Wei Xie

  5. Zhengzhou Institute of Technology, Zhengzhou, China

    Yan Wang

  6. National Academy of Guo Ding Institute of Data Science, Beijing, China

    Zeguang Lu

Appendix: Sample Items of Preposition Test Paper for Each Textbook

Appendix: Sample Items of Preposition Test Paper for Each Textbook

figure a

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xiao, W., Wang, M., Zhang, C., Tan, Y., Chen, Z. (2018). Automatic Generation of Multiple-Choice Items for Prepositions Based on Word2vec. In: Zhou, Q., Miao, Q., Wang, H., Xie, W., Wang, Y., Lu, Z. (eds) Data Science. ICPCSEE 2018. Communications in Computer and Information Science, vol 902. Springer, Singapore. https://doi.org/10.1007/978-981-13-2206-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-2206-8_8

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-2205-1

  • Online ISBN: 978-981-13-2206-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation