Skip to main content
SpringerLink
Log in

Multi-granularity interaction model based on pinyins and radicals for Chinese semantic matching

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Semantic matching plays a critical role in many downstream tasks of natural language processing. Existing semantic matching methods, which focus on learning sentence semantic features based on character and word granularities, neglect to consider the special characteristics of Chinese, e.g., pinyins and radicals. However, both pinyins and radicals contain rich semantics which are able to enhance the Chinese sentence representation. In this paper, we propose a multi-granularity interaction model based on pinyins and radicals (MIPR) for Chinese semantic matching. MIPR first employs an input encoding layer to incorporate multi-granularity information including character, word, pinyin and radical granularities together, next utilizes soft-alignment attention mechanism to devise a multi-granularity interaction layer for capturing the interaction features among different granularities and sentences, then devises a feature aggregation layer to merge the various interaction features for obtaining the final matching representation, followed by a prediction layer to judge the matching degree of the pair of input sentences. Extensive experiments on two public Chinese datasets demonstrate that MIPR achieves significant improvement against the compared models and comparable performance with BERT-based model for Chinese semantic matching task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://www.webank.com/#/product/wld

  2. https://github.com/ZQpengyu/MIPR

  3. https://github.com/fxsjy/jieba

  4. https://github.com/mozillazg/python-pinyin

  5. https://github.com/berniey/hanziconv

  6. https://github.com/wangchuan2008888/cn-radical

References

  1. Chen, J., Chen, Q., Liu, X., Yang, H., Daohe, L u, Tang, B.: The BQ corpus: A large-scale domain-specific Chinese corpus for sentence semantic equivalence identification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 4946–4951 (2018)

  2. Chen, Q., Zhu, X., Ling, Zhen-Hua, Wei, S., Jiang, H., Inkpen, D.: Enhanced LSTM for natural language inference. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 1657–1668 (2017)

  3. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019)

  4. Fei, H., Ren, Y., Ji, D.: Improving text understanding via deep syntax-semantics communication. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Findings, pp. 84–93 (2020)

  5. He, Q., Wang, H., Zhang, Y.: Enhancing generalization in natural language inference by syntax. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Findings, pp. 4973–4978 (2020)

  6. Hu, H., Richardson, K., Xu, L., Li, L., Kübler, S., Moss, L.S: Ocnli: Original Chinese natural language inference. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Findings, pp. 3512–3526 (2020)

  7. Huang, Q., Bu, J., Xie, W., Yang, S., Wu, W., Liu, L.: Multi-task sentence encoding model for semantic retrieval in question answering systems. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1–8 (2019)

  8. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751 (2014)

  9. Kim, S., Kang, I., Kwak, N.: Semantic sentence matching with densely-connected recurrent and co-attentive information. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 6586–6593 (2019)

  10. Kingma, D.P., Adam, J.B.: A method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations (2015)

  11. Lai, Y., Feng, Y., Yu, X., Wang, Z., Xu, K., Zhao, D.: Lattice CNNs for matching based Chinese question answering. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 6634–6641 (2019)

  12. Li, X., Meng, Y., Sun, X., Han, Q., Yuan, A., Li, J.: Is word segmentation necessary for deep learning of Chinese representations?. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 3242–3252 (2019)

  13. Liu, X., Chen, Q., Deng, C., Zeng, H., Chen, J., Li, D., Tang, B.: LCQMC: A large-scale Chinese question matching corpus. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 1952–1962 (2018)

  14. Liu, L., Yang, W., Rao, J., Tang, R., Lin, J.: Incorporating contextual and syntactic structures improves semantic similarity modeling. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1204–1209 (2019)

  15. Liu, S., Yang, T., Yue, T., Zhang, F., Wang, D.: PLOME: Pre-training with misspelled knowledge for Chinese spelling correction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 2991–3000 (2021)

  16. Liu, L., Zhang, Z., Zhao, H., Zhou, X., Zhou, X.: Filling the gap of utterance-aware and speaker-aware representation for multi-turn dialogue. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 13406–13414 (2021)

  17. Liu, W., Zhou, P., Zhao, Z., Wang, Z., Deng, H., Ju, Q.: FastBERT: A self-distilling BERT with adaptive inference time. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6035–6044 (2020)

  18. Lu, H., Li, Y., Chen, M., Kim, H., Serikawa, S.: Brain intelligence: go beyond artificial intelligence. Mobile Networks and Applications 23(2), 368–375 (2018)

    Article  Google Scholar 

  19. Lu, H., Yang, R., Deng, Z., Zhang, Y., Gao, G., Lan, R.: Chinese image captioning via fuzzy attention-based densenet-BiLSTM. ACM Transactions on Multimedia Computing Communications, and Applications 17(1s), 1–18 (2021)

    Article  Google Scholar 

  20. Lu, W., Yu, R., Wang, S., Wang, C., Jian, P., Huang, H.: Sentence semantic matching based on 3D CNN for human–robot language interaction. ACM Trans. Internet Technol. 21(4), 1–24 (2021)

    Google Scholar 

  21. Lu, W., Zhang, Y., Wang, S., Huang, H., Liu, Q., Luo, S.: Concept representation by learning explicit and implicit concept couplings. IEEE Intell. Syst. 36(1), 6–15 (2021)

    Article  Google Scholar 

  22. Lu, H., Zhang, M., Xu, X., Li, Y., Shen, H.T.: Deep fuzzy hashing network for efficient image retrieval. IEEE Trans. Fuzzy Syst. 29(1), 166–176 (2021)

    Article  Google Scholar 

  23. Lu, C., Zhao, Y., Lyu, B., Jin, L., Chen, Z., Zhu, S., Yu, K.: Neural graph matching networks for Chinese short text matching. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6152–6158 (2020)

  24. Lyu Lu, B., Su, C., Yu, Z.K.: Let: Linguistic knowledge enhanced graph transformer for Chinese short text matching. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 13498–13506 (2021)

  25. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv:1301 (2013)

  26. Nguyen, H., Zhang, C., Xia, C., Philip, S.Y.: Semantic matching and aggregation network for few-shot intent detection. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Findings, pp. 1209–1218 (2020)

  27. Peng, H., Ma, Y., Li, Y., Cambria, E.: Learning multi-grained aspect target sequence for Chinese sentiment analysis. Knowl.-Based Syst. 148, 167–176 (2018)

    Article  Google Scholar 

  28. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv: 1910.01108 (2019)

  29. Su, J.: Text emotion classification (iv): Better loss function (2017). https://spaces.ac.cn/archives/4293. Accessed 30 March 2017

  30. Tan, C., Wei, F., Wang, W., Lv, W., Zhou, M.: Multiway attention networks for modeling sentence pairs. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 4411–4417 (2018)

  31. Tao, H., Tong, S., Zhang, K., Xu, T., Liu, Q., Chen, E., Hou, M.: Ideography leads us to the field of cognition: A radical-guided associative model for Chinese text classification. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 13898–13906 (2021)

  32. Tao, H., Tong, S., Zhao, H., Xu, T., Jin, B., Liu, Q.: A radical-aware attention-based model for Chinese text classification. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 5125–5132 (2019)

  33. Wang, Z., Hamza, W., Florian, R.: Bilateral multi-perspective matching for natural language sentences. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 4144–4150 (2017)

  34. Wang, H., Wang, B., Duan, J., Zhang, J.: Chinese spelling error detection using a fusion lattice LSTM. ACM Trans. Asian Low Resour Lang. Inf. Process. 20, 1–11 (2021)

    Google Scholar 

  35. Xiang, L., Yang, S., Liu, Y., Li, Q., Zhu, C.: Novel linguistic steganography based on character-level text generation. Mathematics 8(9), 1558 (2020)

    Article  Google Scholar 

  36. Xing, X., Wang, T., Yang, Y, Hanjalic, A., Shen, H.T.: Radial graph convolutional network for visual question generation. IEEE Transactions on Neural Networks and Learning Systems (2020)

  37. Xu, Z., Lu, W., Li, F., Peng, X., Zhang, R.: Deep feature fusion model for sentence semantic matching. Computers, Materials and Continua 61, 601–616 (2019)

    Article  Google Scholar 

  38. Xu, X., Wang, T., Yang, Y., Zuo, L., Shen, F., Shen, H.T.: Cross-modal attention with semantic consistence for image-text matching. IEEE Transactions on Neural Networks and Learning Systems (2020)

  39. Yu, R., Lu, W., Lu, H., Wang, S., Li, F., Zhang, X., Yu, J.: Sentence pair modeling based on semantic feature map for human interaction with IoT devices. Int. J. Mach. Learn. Cybern. 12, 3081–3099 (2021)

    Article  Google Scholar 

  40. Yu, S., Wang, S., Li, Y., Feng, S., Tian, H., Wu, H., Wang, H.: Ernie 2.0: A continual pre-training framework for language understanding. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence, pp. 8968–8975 (2020)

  41. Zhang, K., Lv, G., Wang, L., Wu, L., Chen, E., Wu, F., Xie, X.: Drr-net: Dynamic re-read network for sentence semantic matching. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 7442–7449 (2019)

  42. Zhang, K., Wu, L., Lv, G., Wang, M., Chen, E., Ruan, S.: Making the relation matters: Relation of relation learning network for sentence semantic matching. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 14411–14419 (2021)

  43. Zhao, S., Huang, Y., Su, C., Li, Y., Wang, F.: Interactive attention networks for semantic text matching. In: Proceedings of the IEEE International Conference on Data Mining, pp. 861–870 (2020)

  44. Zhao, P., Lu, W., Li, Y., Yu, J., Jian, P., Zhang, X.: Chinese semantic matching with multi-granularity alignment and feature fusion. In: Proceedings of the International Joint Conference on Neural Networks (2021)

Download references

Acknowledgements

The research work is partly supported by National Natural Science Foundation of China under Grant No.61502259, National Key R&D Program of China under Grant No.2018YFC0831700 and No.2018YFC0830705, and Key Program of Science and Technology of Shandong Province under Grant No.2020CXGC010901 and No.2019JZZY020124.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China

    Pengyu Zhao, Wenpeng Lu & Weiyu Zhang

  2. School of Computing Technologies, RMIT University, Melbourne, Australia

    Shoujin Wang

  3. Australian Artificial Intelligence Institute, University of Technology Sydney, Sydney, Australia

    Xueping Peng

  4. School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China

    Ping Jian & Hao Wu

Authors
  1. Pengyu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenpeng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shoujin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xueping Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ping Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Weiyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenpeng Lu.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Synthetic Media on the Web

Guest Editors: Huimin Lu, Xing Xu, Jože Guna, and Gautam Srivastava

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, P., Lu, W., Wang, S. et al. Multi-granularity interaction model based on pinyins and radicals for Chinese semantic matching. World Wide Web 25, 1703–1723 (2022). https://doi.org/10.1007/s11280-022-01037-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-022-01037-y

Keywords

Search

Navigation