skip to main content
10.1145/3633637.3633686acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

Semantic Matching Algorithm for Legal Issues Based on BERT and Graph Convolution with Multi-granularity Features

Published: 28 February 2024 Publication History

Abstract

In recent years, pre-training models represented by BERT have shown amazing text semantic representation capabilities supported by large-scale corpus, and now have become one of the mainstream solutions for semantic matching tasks in legal Q&A systems. However, pre-training models generally do not set specific pre-training tasks in terms of syntactic information, but implicitly and incidentally learn in large-scale unsupervised, which does not make good use of syntactic information as an important factor for representing semantics. To address this problem, this paper proposes a multi-granularity feature semantic matching model based on BERT and graph convolution. The model first obtains syntactic features of sentences in the form of dependent syntactic trees using existing tools, and then fuses syntactic features with word features and sentence features of BERT using an attention mechanism and graph convolutional neural network to enhance the characterization of text semantics and make the model better for semantic matching tasks. Experimental results show that the algorithm achieves significant improvements in the semantic matching task and outperforms the five comparison methods in the DIAC2019 dataset. Compared to the baseline model Sentenct-BERT, the accuracy, precision, recall and F1 values are improved by 0.33%, 1.18%, 1.03% and 1.05% improvement.

References

[1]
Zhu Songhua. The current situation and countermeasures of permanent legal counsel business[J]. Law Expo, 2020(07):170-171.
[2]
Pang L, Lan YY, Xu J A review of deep text matching[J]. Journal of Computer Science. 2017, 40(04):985-1003.
[3]
Huang P S, He X, Gao J, Learning deep structured semantic models for web search using clickthrough data[C]. Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 2013: 2333-2338.
[4]
Feng M, Xiang B, Glass M R, Applying deep learning to answer selection: A study and an open task[C]. Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding. IEEE, 2015: 813-820.
[5]
Mueller J, Thyagarajan A. Siamese recurrent architectures for learning sentence similarity[C]. Proceedings of the AAAI conference on artificial intelligence. AAAI, 2016: 2786-2792.
[6]
Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks[C]. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ACL, 2019: 3982-3992.
[7]
Wei Z, Xu X, Wang C, An Index Construction and Similarity Retrieval Method Based on Sentence-Bert[C]. 2022 7th International Conference on Image, Vision and Computing (ICIVC). IEEE, 2022: 934-938.
[8]
Jiang W, Lu J, Liang T, BERTBooster: A Knowledge Enhancement Method Jointing Incremental Training and Gradient Optimization[J]. International Journal of Intelligent Systems (IJIS). 2022, 37(11): 9390-9403.
[9]
Pang L, Lan Y, Guo J, Text matching as image recognition[C]. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. AAAI, 2016: 2793-2799.
[10]
Chen Q, Zhu X, Ling Z H, Enhanced LSTM for Natural Language Inference[C]. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. ACL, 2017: 1657-1668.
[11]
Zhang K, Wu L, Lv G, Making the Relation Matters: Relation of Relation Learning Network for Sentence Semantic Matching[C]. Proceedings of the Association for the Advancement of Artificial Intelligence. AAAI, 2021: 14411-14419.
[12]
Xu C, Xu J, Dong Z, Semantic Sentence Matching via Interacting Syntax Graphs[C]. Proceedings of the 29th International Conference on Computational Linguistics (ICCL). ACL, 2022: 938-949.
[13]
Ye W, Liu Y, Zou L, Fast Semantic Matching Via Flexible Contextualized Interaction[C]. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. ACM, 2022: 1275-1283.
[14]
Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks[C]. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ACL, 2019: 3982-3992.
[15]
Che W, Feng Y, Qin L, N-LTP: An Open-source Neural Language Technology Platform for Chinese[C]. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. EMNLP, 2021: 42-49.
[16]
Chen Q, Zhu X, Ling Z H, Enhanced LSTM for Natural Language Inference[C]. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. ACL, 2017: 1657-1668.
[17]
Wang Z, Hamza W, Florian R. Bilateral multi-perspective matching for natural language sentences[C]. Proceedings of the 26th International Joint Conference on Artificial Intelligence. IJCAI, 2017: 4144-4150.
[18]
Yin W, Schütze H, Xiang B, Abcnn: Attention-based convolutional neural network for modeling sentence pairs[J]. Transactions of the Association for Computational Linguistics, 2016, 4(1): 259-272.
[19]
Huang P S, He X, Gao J, Learning deep structured semantic models for web search using clickthrough data[C]. Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 2013: 2333-2338.
[20]
Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks[C]. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). ACL, 2019: 3982-3992.

Index Terms

  1. Semantic Matching Algorithm for Legal Issues Based on BERT and Graph Convolution with Multi-granularity Features

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICCPR '23: Proceedings of the 2023 12th International Conference on Computing and Pattern Recognition
    October 2023
    589 pages
    ISBN:9798400707988
    DOI:10.1145/3633637
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 February 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. BERT
    2. Graph Convolutional Neural Network
    3. Legal Q&A
    4. Semantic matching

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • the National Natural Science Foundation of China under Grant
    • the by the Development Fund Project of Hebei Key Laboratory of Intelligent Information Perception and Processing under Grant
    • the Guilin Science and Technology Development Program under Grant
    • the Guangxi Key Research and Development Program under Grant
    • the Innovation Project of GUET Graduate Education under Grants
    • the Development Foundation of the 54th Research Institute of China Electronics Technology Group Corporation under Grant
    • the Natural Science Foundation of Guangxi under Grant

    Conference

    ICCPR 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 23
      Total Downloads
    • Downloads (Last 12 months)23
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media