Abstract
We consider the task of automatically generating math word problems (MWPs) of various difficulties that meet the needs of teachers in teaching and testing students in corresponding educational stages. Existing methods fail to produce high-quality problems while allowing the teacher control over the problem difficulty level. In this work, we introduce a controllable MWP generation pipeline that samples from an energy language model with various expert model components for realizing the target attributes. We control the difficulty of the resulting MWPs from mathematical and linguistic aspects by imposing constraints on equations, vocabulary, and topics. We also use other control attributes including fluency and distance to the conditioning sequence to manage language quality and creativity. Experiments and evaluation results demonstrate our approach improves upon the baselines in generating solvable, well-formed, and diverse MWPs of controlled difficulty levels. Lastly, we solicit feedback from various math educators who approve the effectiveness of our system for their MWP design processes. They suggest our outputs align with the expectations of problem designers showing a possibility of using such problem generators in real-life educational scenarios. Our code and data are available on request.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amini, A., Gabriel, S., Lin, P., Koncel-Kedziorski, R., Choi, Y., Hajishirzi, H.: Mathqa: towards interpretable math word problem solving with operation-based formalisms. arXiv preprint arXiv:1905.13319 (2019)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Cobbe, K., et al.: Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)
Daroczy, G., Wolska, M., Meurers, W.D., Nuerk, H.C.: Word problems: a review of linguistic and numerical factors contributing to their difficulty. Front. Psychol. 6, 348 (2015)
Dathathri, S., et al.: Plug and play language models: a simple approach to controlled text generation. arXiv preprint arXiv:1912.02164 (2019)
Goyal, K., Dyer, C., Berg-Kirkpatrick, T.: Exposing the implicit energy networks behind masked language models via metropolis-hastings. arXiv preprint arXiv:2106.02736 (2021)
Ke, P., et al.: CTRLEval: an unsupervised reference-free metric for evaluating controlled text generation. arXiv preprint arXiv:2204.00862 (2022)
Khodeir, N.A., Elazhary, H., Wanas, N.: Generating story problems via controlled parameters in a web-based intelligent tutoring system. Int. J. Inf. Learn. Technol. 35(3), 199–216 (2018)
Koncel-Kedziorski, R., Konstas, I., Zettlemoyer, L., Hajishirzi, H.: A theme-rewriting approach for generating algebra word problems. arXiv preprint arXiv:1610.06210 (2016)
Kuperman, V., Stadthagen-Gonzalez, H., Brysbaert, M.: Age-of-acquisition ratings for 30,000 English words. Behav. Res. Methods 44(4), 978–990 (2012)
Kurdi, G., Leo, J., Parsia, B., Sattler, U., Al-Emari, S.: A systematic review of automatic question generation for educational purposes. Int. J. Artif. Intell. Educ. 30(1), 121–204 (2020)
Liu, A., et al.: Dexperts: decoding-time controlled text generation with experts and anti-experts. arXiv preprint arXiv:2105.03023 (2021)
Liu, T., Fang, Q., Ding, W., Liu, Z.: Mathematical word problem generation from commonsense knowledge graph and equations. arXiv preprint arXiv:2010.06196 (2020)
Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam (2018)
Miao, S.Y., Liang, C.C., Su, K.Y.: A diverse corpus for evaluating and developing English math word problem solvers. arXiv preprint arXiv:2106.15772 (2021)
Mireshghallah, F., Goyal, K., Berg-Kirkpatrick, T.: Mix and match: learning-free controllable text generation using energy language models. arXiv preprint arXiv:2203.13299 (2022)
Polozov, O., O’Rourke, E., Smith, A.M., Zettlemoyer, L., Gulwani, S., Popović, Z.: Personalized mathematical word problem generation. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)
Pongsakdi, N., Kajamies, A., Veermans, K., Lertola, K., Vauras, M., Lehtinen, E.: What makes mathematical word problem solving challenging? Exploring the roles of word problem characteristics, text comprehension, and arithmetic skills. ZDM 52(1), 33–44 (2020)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)
Wang, K., Su, Z.: Dimensionally guided synthesis of mathematical word problems. In: IJCAI, pp. 2661–2668 (2016)
Wang, Z., Lan, A.S., Baraniuk, R.G.: Math word problem generation with mathematical consistency and problem context constraints. arXiv preprint arXiv:2109.04546 (2021)
Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38–45 (2020)
Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: evaluating text generation with BERT. arXiv preprint arXiv:1904.09675 (2019)
Zhou, Q., Huang, D.: Towards generating math word problems from equations and topics. In: Proceedings of the 12th International Conference on Natural Language Generation, pp. 494–503 (2019)
Zhou, W., Jiang, Y.E., Wilcox, E., Cotterell, R., Sachan, M.: Controlled text generation with natural language instructions (2023)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiao, Y., Shridhar, K., Cui, P., Zhou, W., Sachan, M. (2023). Automatic Educational Question Generation with Difficulty Level Controls. In: Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O.C., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2023. Lecture Notes in Computer Science(), vol 13916. Springer, Cham. https://doi.org/10.1007/978-3-031-36272-9_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-36272-9_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36271-2
Online ISBN: 978-3-031-36272-9
eBook Packages: Computer ScienceComputer Science (R0)