Abstract
The semantic matching of questions is a fundamental aspect of retrieval-based question answering (QA) systems. Text representations containing rich semantic information are required to achieve a deeper understanding of question intent. While existing large pre-trained models can obtain character-based text representations with contextual information, the specificity of Chinese sentences makes word-based text representation superior to character-based text representation. In this paper, we propose a question semantic matching method based on word-level and sentence-level interaction features. We utilize a Bidirectional Long Short-Term Memory (BiLSTM) approach to enhance the contextual information of the word representations. Additionally, we incorporate a co-attention mechanism to capture the interaction information between sentence pairs. By comparing our model with several baseline models on a self-built dataset of university financial question pairs, we have achieved remarkable performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Li, P., et al.: Dataset and neural recurrent sequence labeling model for open-domain factoid question answering. arXiv preprint arXiv:1607.06275 (2016)
Wang, D., Wang, W., Wang, S.: Research on domain-specific question answering system oriented natural language understanding: a survey. Comput. Sci. 44(8), 1–41 (2017)
Yu, J., et al.: Modelling domain relationships for transfer learning on retrieval-based question answering systems in e-commerce. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM), pp. 682–690. Association for Computing Machinery, New York, NY, USA (2018)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Wang, J., et al.: Fengshenbang 1.0: being the foundation of Chinese cognitive intelligence. arXiv preprint arXiv:2209.02970 (2022)
Gao, T., Yao, X., Chen, D.: SimCSE: simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 (2021)
Weizenbaum, J.: ELIZA—a computer program for the study of natural language communication between man and machine. Commun. ACM 9(1), 36–45 (1966)
Ferrucci, D., Brown, E., Chu-Carroll, J., Fan, J., Gondek, D., Kalyanpur, A., et al.: Building watson: an overview of the deepqa project. AI Mag. 31(3), 59–79 (2010)
Zhou, L., Gao, J., Li, D., Shum, H.Y.: The design and implementation of XiaoIce, an empathetic social chatbot. Comput. Linguist. 46(1), 53–93 (2020)
Gao, T.: Design and implementation of university financial counseling question answering prototype system based on NLP (基于NLP的高校财务咨询问答原型系统设计与实现). Master’s thesis, Beijing Jiaotong University (2020)
Chi, Y.: Research on question answering technology of enterprise financial audit based on deep learning (基于深度学习的企业财务审计问答技术研究). Master’s thesis, Harbin Engineering University (2018)
Wang, Y.: Design and implementation of financial intelligent question answering system based on knowledge graph (基于知识图谱的财务智能问答系统的设计与实现). Master’s thesis, Huazhong University of Science and Technology (2020)
Zhu, F., et al.: TAT-QA: a question answering benchmark on a hybrid of tabular and textual content in finance. arXiv preprint arXiv:2105.07624 (2021)
Zaib, M., Zhang, W.E., Sheng, Q.Z., Mahmood, A., Zhang, Y.: Conversational question answering: a survey. Knowl. Inf. Syst. 64(12), 3151–3195 (2022)
Lu, X., Deng, Y., Sun, T., Gao, Y., Feng, J., Sun, X., et al.: MKPM: multi keyword-pair matching for natural language sentences. Appl. Intell. 52(2), 1878–1892 (2022)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (NeurIPS), vol. 30, pp. 1-11 (2017)
Deng, Y., Li, X., Zhang, M., Lu, X., Sun, X.: Enhanced distance-aware self-attention and multi-level match for sentence semantic matching. Neurocomputing 501, 174–187 (2022)
Parikh, A.P., Täckström, O., Das, D., Uszkoreit, J.: A decomposable attention model for natural language inference. arXiv preprint arXiv:1606.01933 (2016)
Su, J.: Text emotion classification (IV): better loss function (2017). https://spaces.ac.cn/archives/4293
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Zhang, X., Li, Y., Lu, W., Jian, P., Zhang, G.: Intra-correlation encoding for Chinese sentence intention matching. In: Proceedings of the 28th International Conference on Computational Linguistics (COLING), pp. 5193–5204. International Committee on Computational Linguistics, Barcelona, Spain (2020)
Chen, J., Chen, Q., Liu, X., Yang, H., Lu, D., Tang, B.: The BQ corpus: a large-scale domain-specific Chinese corpus for sentence semantic equivalence identification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4946–4951. Association for Computational Linguistics, Brussels, Belgium (2018)
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Lu, W., et al.: Chinese sentence semantic matching based on multi-level relevance extraction and aggregation for intelligent human–robot interaction. Appl. Soft Comput. 131, 109795 (2022)
Acknowledgments
This work was partly supported by the National Natural Science Foundation of China under Grant (61972336, 62073284), and Zhejiang Provincial Natural Science Foundation of China under Grant (LY23F020001, LY22F020027, LY19F030008).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ying, Y., Zhang, Z., Wu, H., Dong, Y. (2024). Er-EIR: A Chinese Question Matching Model Based on Word-Level and Sentence-Level Interaction Features. In: Sun, Y., Lu, T., Wang, T., Fan, H., Liu, D., Du, B. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2023. Communications in Computer and Information Science, vol 2012. Springer, Singapore. https://doi.org/10.1007/978-981-99-9637-7_8
Download citation
DOI: https://doi.org/10.1007/978-981-99-9637-7_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9636-0
Online ISBN: 978-981-99-9637-7
eBook Packages: Computer ScienceComputer Science (R0)
