Abstract
Deep neural models have achieved promising progress in solving Math Word Problems (MWPs) recently. This paper presents a deep neural solver by adopting numeracy-enhanced decoding to promote the performance of expressions generation. It leverages numerical properties to enhance the capabilities of the decoder, primarily focusing on two aspects: token embedding and target prediction. For token embedding, this paper proposes a numeracy-enhanced token embedding method, which fuses the explicit numerical feature with the contextual feature for number tokens, enabling the decoder to perceive numerical properties during the inference. For target prediction, this paper proposes a dynamic target prediction method, which utilizes a numerical attention network to identify the mathematical category of the problem and adaptively invokes category-aware parameter matrices to generate diverse expressions for different problems. Experimental results demonstrate that the proposed method not only achieves competitive performance on the Chinese MWP dataset but also achieves state-of-the-art results on the NLPCC Shared Task 3 dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Change history
29 October 2023
A correction has been published.
References
Amini, A., Gabriel, S., Lin, S., Koncel-Kedziorski, R., Choi, Y., Hajishirzi, H.: MathQA: towards interpretable math word problem solving with operation-based formalisms. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2357–2367 (2019)
Bobrow, D.G.: A question-answering system for high school algebra word problems. In: Proceedings of the 1964 Fall Joint Computer Conference, pp. 591–614 (1964)
Cui, Y., Che, W., Liu, T., Qin, B., Yang, Z.: Pre-training with whole word masking for Chinese BERT. IEEE/ACM Trans. Audio Speech Lang. Process. 29, 3504–3514 (2021)
Hu, E.J., et al.: LoRA: low-rank adaptation of large language models. arXiv:2106.09685 (2021)
Jie, Z., Li, J., Lu, W.: Learning to reason deductively: math word problem solving as complex relation extraction. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pp. 5944–5955 (2022)
Kintsch, W., Greeno, J.G.: Understanding and solving word arithmetic problems. Psychol. Rev. 92, 109 (1985)
Kushman, N., Artzi, Y., Zettlemoyer, L., Barzilay, R.: Learning to automatically solve algebra word problems. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 271–281 (2014)
Li, Z., et al.: Seeking patterns, not just memorizing procedures: contrastive learning for solving math word problems. In: Findings of the Association for Computational Linguistics, NAACL 2022, pp. 2486–2496 (2022)
Liang, Z., et al.: MWP-BERT: numeracy-augmented pre-training for math word problem solving. In: Findings of the Association for Computational Linguistics, NAACL 2022, pp. 997–1009 (2022)
Liang, Z., Zhang, J., Zhang, X.: Analogical math word problems solving with enhanced problem-solution association. arXiv:2212.00837 (2022)
Lin, X., et al.: Learning relation-enhanced hierarchical solver for math word problems. IEEE Trans. Neural Netw. Learn. Syst. (2023, early access)
Lin, X., et al.: HMS: a hierarchical solver with dependency-enhanced understanding for math word problem. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 4232–4240 (2021)
Liu, Q., Guan, W., Li, S., Kawahara, D.: Tree-structured decoding for solving math word problems. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pp. 2370–2379 (2019)
Mor, G., Ankit, G., Jonathan, B.: Injecting numerical reasoning skills into language models. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 946–958 (2020)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 5485–5551 (2020)
Roy, S., Roth, D.: Unit dependency graph and its application to arithmetic word problem solving. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp. 3082–3088 (2017)
Shen, J., et al.: Generate & rank: a multi-task framework for math word problems. In: Findings of the Association for Computational Linguistics, EMNLP 2021, pp. 2269–2279 (2021)
Shi, S., Wang, Y., Lin, C.Y., Liu, X., Rui, Y.: Automatically solving number word problems by semantic parsing and reasoning. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1132–1142 (2015)
Touvron, H., et al.: LLaMA: open and efficient foundation language models. arXiv arXiv:2302.13971 (2023)
Wang, L., Wang, Y., Cai, D., Zhang, D., Liu, X.: Translating a math word problem to an expression tree. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1064–1069 (2018)
Wang, Y., Liu, X., Shi, S.: Deep neural solver for math word problems. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 845–854 (2017)
Wu, Q., Zhang, Q., Wei, Z.: An edge-enhanced hierarchical graph-to-tree network for math word problem solving. In: Findings of the Association for Computational Linguistics, EMNLP 2021, pp. 1473–1482 (2021)
Xie, Z., Sun, S.: A goal-driven tree-structured neural model for math word problems. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 5299–5305 (2019)
Yu, X., Wang, M., Gan, W., He, B., Ye, N.: A framework for solving explicit arithmetic word problems and proving plane geometry theorems. Int. J. Pattern Recognit. Artif. Intell. 33, 1940005:1–1940005:21 (2019)
Zhang, J., et al.: Teacher-student networks with multiple decoders for solving math word problem. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence, pp. 4011–4017 (2020)
Zhang, J., et al.: Graph-to-tree learning for solving math word problems. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3928–3937 (2020)
Acknowledgements
This work is partially supported by the General Program of the National Natural Science Foundation of China (Grant No: 61977029) and the China Postdoctoral Science Foundation (Grant No: 2023M731245).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Peng, R., Yang, C., Huang, L., Lyu, X., Meng, H., Yu, X. (2023). A Numeracy-Enhanced Decoding for Solving Math Word Problem. In: Liu, F., Duan, N., Xu, Q., Hong, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2023. Lecture Notes in Computer Science(), vol 14304. Springer, Cham. https://doi.org/10.1007/978-3-031-44699-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-44699-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44698-6
Online ISBN: 978-3-031-44699-3
eBook Packages: Computer ScienceComputer Science (R0)
