Skip to main content
Springer Nature Link
Log in

Using Large Language Models to Support Teaching and Learning of Word Problem Solving in Tutoring Systems

  • Conference paper
  • First Online:
Generative Intelligence and Intelligent Tutoring Systems (ITS 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14798))

Included in the following conference series:

  • 1138 Accesses

Abstract

The latest developments in Large Language Models (LLMs) open the door to significantly improving scaffolding and support when supervising word problem-solving. In this paper, we examine the potential of a large variety of open models for solving different types of arithmetical problems and discuss the potential implications for the development of Intelligent Tutoring Systems (ITSs). The results reported show that relatively small LLMs are able to correctly solve around two-thirds of single-stage word problems, obtaining a similar performance as children. Nevertheless, their behavior varies in terms of their ability to provide the correct solution for specific conceptual schemes. Beyond their potential as a problem-solving tool, the research presented opens the door to using LLMs for the implementation of virtual agent-based students.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://ollama.ai.

  2. 2.

    https://huggingface.co.

References

  1. Albornoz-De Luise, R.S., Arevalillo-Herráez, M., Arnau, D.: On using conversational frameworks to support natural language interaction in intelligent tutoring systems. IEEE Trans. Learn. Technol. 16(5), 722–735 (2023)

    Article  Google Scholar 

  2. Arevalillo-Herráez, M., Arnau, D., Marco-Giménez, L.: Domain-specific knowledge representation and inference engine for an intelligent tutoring system. Knowl.-Based Syst. 49, 97–105 (2013)

    Article  Google Scholar 

  3. Arnau-González, P., Arevalillo-Herráez, M., Albornoz De Luise, R., Arnau, D.: A methodological approach to enable natural language interaction in an intelligent tutoring system. Comput. Speech Lang. 81, 101516 (2023). https://doi.org/10.1016/j.csl.2023.101516

  4. Arnau-González, P., Mamolar, A.S., Katsigiannis, S., Althobaiti, T., Arevalillo-Herráez, M.: Toward automatic tutoring of math word problems in intelligent tutoring systems. IEEE Access 11, 67030–67039 (2023). https://doi.org/10.1109/ACCESS.2023.3290478

  5. Beal, C.R.: AnimalWatch: an intelligent tutoring system for algebra readiness. In: Azevedo, R., Aleven, V. (eds.) International Handbook of Metacognition and Learning Technologies. SIHE, vol. 28, pp. 337–348. Springer, New York (2013). https://doi.org/10.1007/978-1-4419-5546-3_22

    Chapter  Google Scholar 

  6. Carpenter, T.P., Moser, J.M., Hiebert, J.: Problem structure and first-grade children’s initial solution processes for simple addition and subtraction problems. J. Res. Math. Educ. 12(1), 27–39 (1981)

    Article  Google Scholar 

  7. Chang, K.E., Sung, Y.T., Lin, S.F.: Computer-assisted learning for mathematical problem solving. Comput. Educ. 46(2), 140–151 (2006)

    Article  Google Scholar 

  8. Cheng, L., Croteau, E., Baral, S., Heffernan, C., Heffernan, N.: Facilitating student learning with a chatbot in an online math learning platform. J. Educ. Comput. Res., 07356331241226592 (2024)

    Google Scholar 

  9. Chiang, W.L., et al.: Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality, March 2023. https://lmsys.org/blog/2023-03-30-vicuna/

  10. Cunha-Pérez, C., Arevalillo-Herráez, M., Arnau, D.: Design and evaluation of a set of methodological strategies for learning a second language in students with down syndrome using computer-based instruction. IEEE Trans. Learn. Technol. 17, 172–180 (2024). https://doi.org/10.1109/TLT.2023.3242170

  11. Duckworth, A.L., Yeager, D.S.: Measurement matters: assessing personal qualities other than cognitive ability for educational purposes. Educ. Res. 44(4), 237–251 (2015)

    Article  Google Scholar 

  12. Fischer, J.P.: L’enfant et le comptage. IREM, Strasbourg, a paraitre (1979)

    Google Scholar 

  13. González-Calero, J.A., Arnau, D., Puig, L., Arevalillo-Herráez, M.: Intensive scaffolding in an intelligent tutoring system for the learning of algebraic word problem solving. Br. J. Educ. Technol. 46(6), 1189–1200 (2015). https://doi.org/10.1111/bjet.12183

  14. Grossman, J., Lin, Z., Sheng, H., Wei, J.T.Z., Williams, J.J., Goel, S.: MathBot: transforming online resources for learning math into conversational interactions. In: AAAI 2019 Story-Enabled Intelligence (2019)

    Google Scholar 

  15. Heffernan, N.T., Koedinger, K.R.: An intelligent tutoring system incorporating a model of an experienced human tutor. In: Cerri, S.A., Gouardères, G., Paraguaçu, F. (eds.) ITS 2002. LNCS, vol. 2363, pp. 596–608. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47987-2_61

    Chapter  Google Scholar 

  16. Koedinger, K.R., Anderson, J.R.: Illustrating principled design: the early evolution of a cognitive tutor for algebra symbolization. Interact. Learn. Environ. 5(1), 161–179 (1998)

    Article  Google Scholar 

  17. Ma, L.: Knowing and Teaching Elementary Mathematics: Teachers’ Understanding of Fundamental Mathematics in China and the United States. Routledge, New Jersey (2020)

    Google Scholar 

  18. Mahan, D., Carlow, R., Castricato, L., Cooper, N., Laforte, C.: Stable beluga models. https://huggingface.co/stabilityai/StableBeluga2

  19. Nesher, P.: Levels of description in the analysis of addition and subtraction word problems. In: Carpenter, T.P., Moser, J.M., Romberg, T. (eds.) Addition and Subtraction: Developmental Perspective. Lawrence Erlbaum Associates, Hiilsdale (1981)

    Google Scholar 

  20. Nesher, P., Katriel, T.: Two cognitive modes in arithmetic word problem solving. In: Second Annual Meeting of the International Group for the Psychology of Mathematics Education, Osnabruck, West Germany (1978)

    Google Scholar 

  21. Nesher, P., Greeno, J.G., Riley, M.S.: The development of semantic categories for addition and subtraction. Educ. Stud. Math. 13(4), 373–394 (1982)

    Article  Google Scholar 

  22. Nesher, P., Teubal, E.: Verbal cues as an interfering factor in verbal problem solving. Educ. Stud. Math., 41–51 (1975)

    Google Scholar 

  23. Patel, A., Bhattamishra, S., Goyal, N.: Are NLP models really able to solve simple math word problems? In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2080–2094. Association for Computational Linguistics, Online, June 2021

    Google Scholar 

  24. Riley, M.S., Greeno, J.G., Heller, J.L.: Development of children’s problem-solving ability in arithmetic. In: Ginsburg, H.P. (ed.) The Development of Mathematical Thinking, pp. 153–196. Academic Press, New York (1984)

    Google Scholar 

  25. Ruan, S., et al.: Supporting children’s math learning with feedback-augmented narrative technology. In: Proceedings of the Interaction Design and Children Conference, pp. 567–580 (2020)

    Google Scholar 

  26. Smith, S., et al.: Using DeepSpeed and Megatron to train Megatron-turing NLG 530b, a large-scale generative language model (2022)

    Google Scholar 

  27. Steffe, L.P., Johnson, D.C.: Problem-solving performances of first-grade children. J. Res. Math. Educ. 2(1), 50–64 (1971)

    Article  Google Scholar 

  28. Tamburino, J.L.: An analysis of the modelling processes used by kindergarten children in solving simple addition and subtraction story problems. Ph.D. thesis, University of Pittsburgh (1981)

    Google Scholar 

  29. Touvron, H., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2302.13971 (2023)

  30. Vergnaud, G.: A classification of cognitive tasks and operations of thought involved in addition and subtraction problems. In: Carpenter, T.P., Moser, J.M., Romberg, T.A. (eds.) Addition and Substraction: A Cognitive Perspective, pp. 39–59. Erlbaum, Hillsdale (1982)

    Google Scholar 

  31. Vergnaud, G., Durand, C.: Structures additives et complexité psychogénétique. Revue française de pédagogie, pp. 28–43 (1976)

    Google Scholar 

  32. Vergnaud, G.: Multiplicative structures. In: Lesh, R., Landau, M. (eds.) Acquisition of Mathematics Concepts and Processes, pp. 127–174. Academic Press, New York (1983)

    Google Scholar 

  33. Wei, J., et al.: Emergent abilities of large language models. Trans. Mach. Learn. Res. 2022 (2022). https://openreview.net/forum?id=yzkSU5zdwD

  34. Wu, T., et al.: A brief overview of ChatGPT: the history, status quo and potential future development. IEEE/CAA J. Automatica Sinica 10(5), 1122–1136 (2023)

    Article  Google Scholar 

  35. Yu, L., et al.: MetaMath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12284 (2023)

  36. Zhao, W.X., et al.: A survey of large language models. arXiv preprint arXiv:2303.18223 (2023)

Download references

Acknowledgements

This research has been supported by project TED2021-129485B-C42, funded by MCIN/AEI/10.13039/501100011033 and the European Union “NextGenerationEU”/PRTR; and pre-doctoral grant ACIF/2021/439, funded by the Valencian Regional Government.

Author information

Authors and Affiliations

  1. Departament d’Informàtica, Universitat de València, 46100, Burjassot, Valencia, Spain

    Jaime Arnau-Blasco, Miguel Arevalillo-Herráez, Sergi Solera-Monforte & Yuyan Wu

Authors
  1. Jaime Arnau-Blasco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miguel Arevalillo-Herráez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sergi Solera-Monforte
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuyan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jaime Arnau-Blasco .

Editor information

Editors and Affiliations

  1. University of Macedonia, Thessaloniki, Greece

    Angelo Sifaleras

  2. Athabasca University, Athabasca, AB, Canada

    Fuhua Lin

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Arnau-Blasco, J., Arevalillo-Herráez, M., Solera-Monforte, S., Wu, Y. (2024). Using Large Language Models to Support Teaching and Learning of Word Problem Solving in Tutoring Systems. In: Sifaleras, A., Lin, F. (eds) Generative Intelligence and Intelligent Tutoring Systems. ITS 2024. Lecture Notes in Computer Science, vol 14798. Springer, Cham. https://doi.org/10.1007/978-3-031-63028-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-63028-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-63027-9

  • Online ISBN: 978-3-031-63028-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics