Baohua Qiang
ABSTRACT
In recent years, pre-training models represented by BERT have shown amazing text semantic representation capabilities supported by large-scale corpus, and now have become one of the mainstream solutions for semantic matching tasks in legal Q&A systems. However, pre-training models generally do not set specific pre-training tasks in terms of syntactic information, but implicitly and incidentally learn in large-scale unsupervised, which does not make good use of syntactic information as an important factor for representing semantics. To address this problem, this paper proposes a multi-granularity feature semantic matching model based on BERT and graph convolution. The model first obtains syntactic features of sentences in the form of dependent syntactic trees using existing tools, and then fuses syntactic features with word features and sentence features of BERT using an attention mechanism and graph convolutional neural network to enhance the characterization of text semantics and make the model better for semantic matching tasks. Experimental results show that the algorithm achieves significant improvements in the semantic matching task and outperforms the five comparison methods in the DIAC2019 dataset. Compared to the baseline model Sentenct-BERT, the accuracy, precision, recall and F1 values are improved by 0.33%, 1.18%, 1.03% and 1.05% improvement.
- Zhu Songhua. The current situation and countermeasures of permanent legal counsel business[J]. Law Expo, 2020(07):170-171.Google Scholar
- Pang L, Lan YY, Xu J A review of deep text matching[J]. Journal of Computer Science. 2017, 40(04):985-1003.Google Scholar
- Huang P S, He X, Gao J, Learning deep structured semantic models for web search using clickthrough data[C]. Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 2013: 2333-2338.Google Scholar
- Feng M, Xiang B, Glass M R, Applying deep learning to answer selection: A study and an open task[C]. Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding. IEEE, 2015: 813-820.Google Scholar
- Mueller J, Thyagarajan A. Siamese recurrent architectures for learning sentence similarity[C]. Proceedings of the AAAI conference on artificial intelligence. AAAI, 2016: 2786-2792.Google Scholar
- Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks[C]. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ACL, 2019: 3982-3992.Google Scholar
- Wei Z, Xu X, Wang C, An Index Construction and Similarity Retrieval Method Based on Sentence-Bert[C]. 2022 7th International Conference on Image, Vision and Computing (ICIVC). IEEE, 2022: 934-938.Google Scholar
- Jiang W, Lu J, Liang T, BERTBooster: A Knowledge Enhancement Method Jointing Incremental Training and Gradient Optimization[J]. International Journal of Intelligent Systems (IJIS). 2022, 37(11): 9390-9403.Google ScholarDigital Library
- Pang L, Lan Y, Guo J, Text matching as image recognition[C]. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. AAAI, 2016: 2793-2799.Google Scholar
- Chen Q, Zhu X, Ling Z H, Enhanced LSTM for Natural Language Inference[C]. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. ACL, 2017: 1657-1668.Google Scholar
- Zhang K, Wu L, Lv G, Making the Relation Matters: Relation of Relation Learning Network for Sentence Semantic Matching[C]. Proceedings of the Association for the Advancement of Artificial Intelligence. AAAI, 2021: 14411-14419.Google Scholar
- Xu C, Xu J, Dong Z, Semantic Sentence Matching via Interacting Syntax Graphs[C]. Proceedings of the 29th International Conference on Computational Linguistics (ICCL). ACL, 2022: 938-949.Google Scholar
- Ye W, Liu Y, Zou L, Fast Semantic Matching Via Flexible Contextualized Interaction[C]. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. ACM, 2022: 1275-1283.Google Scholar
- Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks[C]. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ACL, 2019: 3982-3992.Google Scholar
- Che W, Feng Y, Qin L, N-LTP: An Open-source Neural Language Technology Platform for Chinese[C]. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. EMNLP, 2021: 42-49.Google Scholar
- Chen Q, Zhu X, Ling Z H, Enhanced LSTM for Natural Language Inference[C]. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. ACL, 2017: 1657-1668.Google Scholar
- Wang Z, Hamza W, Florian R. Bilateral multi-perspective matching for natural language sentences[C]. Proceedings of the 26th International Joint Conference on Artificial Intelligence. IJCAI, 2017: 4144-4150.Google Scholar
- Yin W, Schütze H, Xiang B, Abcnn: Attention-based convolutional neural network for modeling sentence pairs[J]. Transactions of the Association for Computational Linguistics, 2016, 4(1): 259-272.Google ScholarCross Ref
- Huang P S, He X, Gao J, Learning deep structured semantic models for web search using clickthrough data[C]. Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 2013: 2333-2338.Google Scholar
- Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks[C]. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ACL, 2019: 3982-3992.Google Scholar
Index Terms
- Semantic Matching Algorithm for Legal Issues Based on BERT and Graph Convolution with Multi-granularity Features
Recommendations
Dual-View Distilled BERT for Sentence Embedding
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information RetrievalRecently, BERT realized significant progress for sentence matching via word-level cross sentence attention. However, the performance significantly drops when using siamese BERT-networks to derive two sentence embeddings, which fall short in capturing the ...
Multi-perspective Enhancement Of Text Semantic Matching
NLPIR '23: Proceedings of the 2023 7th International Conference on Natural Language Processing and Information RetrievalWith the widespread adoption of social media, text matching tasks have gained a prominent role in the field of natural language processing. However, traditional text matching methods often overly rely on shallow features and do not adequately consider ...
Multi-perspective Text Matching Algorithm Based on Multi-granularity Feature Convolution
ICCPR '23: Proceedings of the 2023 12th International Conference on Computing and Pattern RecognitionThe core of Chinese text matching task lies in mining the deep semantic information inside the text, exploring the semantic similarities and differences between different texts, and then analyzing the semantic similarity between two texts to be matched. ...
Comments