Abstract
Deep neural networks are the most popular choices for semantic matching nowadays. In this paper, we propose a novel Chinese semantic matching approach, namely Neural Fusion Model, which consists of char-word fusion encoder and encoding-interaction fusion classifier. The char-word fusion encoder models the char and word sequences separately, and then uses a char-word fusion units to fuse them together. The encoding-interaction fusion classifier jointly learns from three simple classifiers. There are a encoding based classifier, a interaction-based classifier and a fusion classifier. Among them, the fusion classifier combines encoding-based and interaction-based representation from multiple-perspective. These three classifiers share the two-layer feed-forward network for prediction. Empirical studies demonstrate the effectiveness of the proposed model for Chinese semantic matching, and it achieves the best results among non-BERT models. In addition, our BERT-equipped model obtains new state-of-the-art results on the Chinese semantic matching benchmark corpus: LCQMC and BQ.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Bingel, J., Søgaard, A.: Identifying beneficial task relations for multi-task learning in deep neural networks. In: Proceedings of the EACL, vol. 2, pp. 164–169 (2017)
Chen, J., Chen, Q., Liu, X., Yang, H., Lu, D., Tang, B.: The BQ corpus: a large-scale domain-specific Chinese corpus for sentence semantic equivalence identification. In: Proceedings of EMNLP, pp. 4946–4951 (2018)
Chen, Q., Zhu, X., Ling, Z.H., Wei, S., Jiang, H., Inkpen, D.: Enhanced LSTM for natural language inference. In: Proceedings of ACL, vol. 1, pp. 1657–1668 (2017)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL, vol. 1, pp. 4171–4186 (2019)
Heilman, M., Smith, N.A.: Tree edit models for recognizing textual entailments, paraphrases, and answers to questions. In: Proceedings of NAACL, pp. 1011–1019. Association for Computational Linguistics (2010)
Huang, P.S., He, X., Gao, J., Deng, L., Acero, A., Heck, L.: Learning deep structured semantic models for web search using clickthrough data. In: Proceedings of CIKM, pp. 2333–2338 (2013)
Huang, Q., Bu, J., Xie, W., Yang, S., Wu, W., Liu, L.: Multi-task sentence encoding model for semantic retrieval in question answering systems. In: Proceedings of IJCNN, pp. 1–8. IEEE (2019)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of EMNLP, pp. 1746–1751 (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Li, X., Meng, Y., Sun, X., Han, Q., Yuan, A., Li, J.: Is word segmentation necessary for deep learning of Chinese representations? In: Proceedings of ACL, pp. 3242–3252 (2019)
Liu, X., et al.: LCQMC: a large-scale Chinese question matching corpus. In: Proceedings of COLING, pp. 1952–1962 (2018)
Meng, Y., et al.: Glyce: glyph-vectors for Chinese character representations. In: Proceedings of NIPS, pp. 2742–2753 (2019)
Mou, L., et al.: Natural language inference by tree-based convolution and heuristic matching. In: Proceedings of ACL, vol. 2, pp. 130–136 (2016)
Mueller, J., Thyagarajan, A.: Siamese recurrent architectures for learning sentence similarity. In: Proceedings of AAAI (2016)
Nie, Y., Bansal, M.: Shortcut-stacked sentence encoders for multi-domain inference. arXiv preprint arXiv:1708.02312 (2017)
Vaswani, A., et al.: Attention is all you need. In: Proceedings of NIPS, pp. 5998–6008 (2017)
Wang, Z., Hamza, W., Florian, R.: Bilateral multi-perspective matching for natural language sentences. In: Proceedings of IJCAI, pp. 4144–4150 (2017)
Yang, R., Zhang, J., Gao, X., Ji, F., Chen, H.: Simple and effective text matching with richer alignment features. In: Proceedings of ACL, pp. 4699–4709 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, R., Miao, Q., Shi, M., Chu, M., Yu, K. (2021). Neural Fusion Model for Chinese Semantic Matching. In: Chen, H., Liu, K., Sun, Y., Wang, S., Hou, L. (eds) Knowledge Graph and Semantic Computing: Knowledge Graph and Cognitive Intelligence. CCKS 2020. Communications in Computer and Information Science, vol 1356. Springer, Singapore. https://doi.org/10.1007/978-981-16-1964-9_7
Download citation
DOI: https://doi.org/10.1007/978-981-16-1964-9_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-1963-2
Online ISBN: 978-981-16-1964-9
eBook Packages: Computer ScienceComputer Science (R0)