Skip to main content
SpringerLink
Log in

Syntax–Aware graph convolutional network for the recognition of chinese implicit inter-sentence relations

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

In the literature, most previous studies on English implicit inter-sentence relation recognition only focused on semantic interactions, which could not exploit the syntactic interactive information in Chinese due to its complicated syntactic structure characteristics. In this paper, we propose a novel and effective model DSGCN-RoBERTa to learn the interaction features implied in sentences from both syntactic and semantic perspectives. To generate a rich contextual sentence embedding, we exploit RoBERTa, a large-scale pre-trained language model based on the transformer unit. DSGCN-RoBERTa consists of two key modules, the syntactic interaction and the semantic interaction modules. Specifically, the syntactic interaction module helps capture the depth-level structure information, including non-consecutive words and their relations, while the semantic interaction module enables the model to understand the context from the whole sentence to the local words. Furthermore, on top of such multi-perspective feature representations, we design a strength-dependent matching strategy that is able to adaptively capture the strong relevant interactive information in a fine-grained level. Extensive experiments demonstrate that the proposed method achieved state-of-the-art results on benchmarks Chinese compound sentence corpus CCCS and Chinese discourse corpus CDTB datasets. We also achieve comparable performance on the English corpus PDTB that demonstrates the superiority of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Lin Y, Ji H, Huang F, Wu L (2020) A joint neural model for information extraction with global features. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 7999–8009

  2. Yung F (2014) Towards a discourse relation-aware approach for Chinese-English machine translation. In: Proceedings of the 52th Annual Meeting of the Association for Computational Linguistics, pp 18–25, https://doi.org/10.3115/v1/P14-3003

  3. Deng Y, Xie Y, Li Y, Yang M, Du N, Fan W, Lei K, Shen Y (2019) Multi-task learning with multi-view attention for answer selection and knowledge base question answering. In: Proceedings of the 33th AAAI Conference on Artificial Intelligence, pp 6318–6325, https://doi.org/10.1609/aaai.v33i01.33016318

  4. Aliakbarpour H, Manzuri MT, Rahmani AM (2021) Improving the readability and saliency of abstractive text summarization using combination of deep neural networks equipped with auxiliary attention mechanism. J Supercomput. https://doi.org/10.1007/s11227-021-03950-x

  5. Verga P, Strubell E, McCallum A (2018) Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, pp 872–884. https://doi.org/10.18653/v1/N18-1080

  6. Ding B, Kong F, Li S, Zhou G (2014) Explicit discourse relation parsing of chinese text. J Chin Inform Process 28(6):101–106

    Google Scholar 

  7. Shu J, Yang J (2010) The computing method of semantic relevancy of clauses in chinese compound sentence. In: Proceedings of the International Symposium on Intelligence Information Processing and Trusted Computing, pp 282–285. https://doi.org/10.1109/IPTC.2010.184

  8. Hu Q, Xie F, Li Y, Liu S (2016) Statistical analysis of the collocation networks of relative words in chinese complex sentence based on complex network theory. Chin Inform Process 30(4):56–64

    Google Scholar 

  9. Qin L, Zhang Z, Zhao H, Hu Z, Xing EP (2017) Adversarial connective-exploiting networks for implicit discourse relation classification. In: Proceedings of the 55th Annual Meeting of the 480 Association for Computational Linguistics, pp 1006–1017. https://doi.org/10.18653/v1/P17-1093

  10. Zhao L, Xu W, Gao S, Guo J (2020) Cross-sentence N-ary relation classification using LSTMs on graph and sequence structures. Knowl Based Syst 207. https://doi.org/10.1016/j.knosys.2020.106266

  11. Dai Z, Huang R (2018) Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in a paragraph. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 141–151

  12. Shi W, Demberg V (2019) Next sentence prediction helps implicit discourse relation classification within and across domains. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 5790–5796. https://doi.org/10.18653/v1/D19-1586

  13. Zhang Y, Jian P, Meng F, Geng R, Cheng W, Zhou J (2019) Semantic graph convolutional network for implicit discourse relation classification. Comput Res Reposit. http://arxiv.org/abs/1910.09183

  14. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) RoBERTa: a robustly optimized BERT Pretraining Approach

  15. Zhou W, Yuan C (2008) Automatic recognizing of complex sentences with coordinating relation. Appl Res Comput 25(3):764–766. https://doi.org/10.3969/j.issn.1001-3695.2008.03.034

    Article  Google Scholar 

  16. Wu Y, Shi J, Wan F, Lv X (2013) Automatic identification of chinese coordination discourse relation. Acta Scientiarum Naturalium Universitatis Pekinensis 49(1):1–6. https://doi.org/10.13209/j.0479-8023.2013.001

  17. Yang J, Chen Z, Shen X, Hu J (2017) Automatic recognition of relation category of non-saturated compound sentences with two clauses. Appl Res Comput 34(10):2950–2953. https://doi.org/10.3969/j.issn.1001-3695.2017.10.016

    Article  Google Scholar 

  18. Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, pp 1746–1751. https://doi.org/10.3115/v1/D14-1181

  19. Wang B (2018) Disconnected recurrent neural networks for text categorization. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 2311–2320. https://doi.org/10.18653/v1/P18-1215

  20. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Proceedings of the Annual Conference on Neural Information Processing Systems, pp 5998–6008

  21. Bai H, Zhao H (2018) Deep enhanced representation for implicit discourse relation recognition. In: Proceedings of the 27th International Conference on Computational Linguistics, pp 571–583

  22. Dai Z, Huang R (2019) A Regularization approach for incorporating event knowledge and coreference relations into neural discourse parsing. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp 2974–2985. https://doi.org/10.18653/v1/D19-1295

  23. Liu X, Ou J, Song Y, Jiang X (2020) On the importance of word and sentence representation learning in implicit discourse relation classification. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence, pp 3830–3836

  24. Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Proceedings of the North American Chapter of the Association for Computational Linguistics, pp 2227–2237. https://doi.org/10.18653/v1/n18-1202

  25. Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 4171–4186. https://doi.org/10.18653/v1/N19-1423

  26. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations

  27. Fu TJ, Li PH, Ma WY (2019) GraphRel: Modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 1409–1418. https://doi.org/10.18653/v1/P19-1136

  28. Zaporojets K, Deleu J, Develder C, Demeester T (2021) DWIE: an entity-centric dataset for multi-task document-level information extraction. Inform Process Manag 58(4). https://doi.org/10.1016/j.ipm.2021.102563

  29. Qiao C, Hu X (2020) A neural knowledge graph evaluator: combining structural and semantic evidence of knowledge graphs for predicting supportive knowledge in scientific QA. Inform Process Manag 57(6):102–309. https://doi.org/10.1016/j.ipm.2020.102309

    Article  Google Scholar 

  30. Hu L, Li C, Shi C, Yang C, Shao C (2020) Graph neural news recommendation with long-term and short-term interest modeling. Inform Process Manag 57(2). https://doi.org/10.1016/j.ipm.2019.102142

  31. Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: Proceedings of the 33th AAAI Conference on Artificial Intelligence, pp 7370–7377. https://doi.org/10.1609/aaai.v33i01.33017370

  32. Bastings J, Titov I, Aziz W, Marcheggiani D, Sima’an K (2017) Graph convolutional encoders for syntax-aware neural machine translation. In: Palmer M, Hwa R, Riedel S (eds) Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP, Association for Computational Linguistics, pp 1957–1967. https://doi.org/10.18653/v1/d17-1209

  33. Zhang Y, Qi P, Manning CD (2018) Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 2205–2215

  34. Xu S, Li P, Kong F, Zhu Q, Zhou G (2019) Topic tensor network for implicit discourse relation recognition in Chinese. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 608–618. https://doi.org/10.18653/v1/P19-1058

  35. Schlichtkrull MS, Kipf TN, Bloem P, van den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: Proceedings of the 15th European Semantic Web Conference, Springer, vol 10843, pp 593–607. https://doi.org/10.1007/978-3-319-93417-4_38

  36. Xing F, Yao S (2005) Construction and utilization of chinese compound sentence corpus. In: Proceedings of the 3rd HNC and Linguistics Research Seminar, pp 432–439

  37. Zhou Y, Xue N (2012) PDTB-style discourse annotation of Chinese text. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pp 69–77

  38. Prasad R, Dinesh N, Lee A, Miltsakaki E, Robaldo L, Joshi A, Webber B (2008) The penn discourse TreeBank 2.0. In: Proceedings of the International Conference on Language Resources and Evaluation

  39. Ji Y, Eisenstein J (2015) one vector is not enough: entity-augmented distributed semantics for discourse relations. Trans Assoc Comput Ling 3:329–344. https://doi.org/10.1162/tacl_a_00142

    Article  Google Scholar 

  40. Rönnqvist S, Schenk N, Chiarcos C (2017) A recurrent neural model with attention for the recognition of Chinese implicit discourse relations. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp 256–262. https://doi.org/10.18653/v1/P17-2040

  41. Kingma DP, Ba J (2017) Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations

  42. Johnson R, Zhang T (2017) Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, pp 562–570. https://doi.org/10.18653/v1/P17-1052

  43. Guo F, He R, Dang J, Wang J (2020) Working memory-driven neural networks with a novel knowledge enhancement paradigm for implicit discourse relation recognition. Proc AAAI Conf Artif Intell 34:7822–7829. https://doi.org/10.1609/aaai.v34i05.6287

    Article  Google Scholar 

  44. He R, Wang J, Guo F, Han Y (2020) TransS-driven joint learning architecture for implicit discourse relation recognition. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 139–148. https://doi.org/10.18653/v1/2020.acl-main.14

Download references

Acknowledgements

This work was supported in part by The National Key Research and Development Program of China under Grant no.2018YFC0809804 and the National Social Science Fund of China under Grant no.18BYY174.

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, 430072, China

    Kaili Sun & Huyin Zhang

  2. School of Computer Science, Central China Normal University, Wuhan, 430079, China

    Yuan Li & Linfei Yuan

  3. Artificial Intelligence Institute, Wuhan University, Wuhan, 430072, China

    Chi Guo

  4. Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan, 430079, China

    Quan Hu

Authors
  1. Kaili Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huyin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chi Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Linfei Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Quan Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huyin Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, K., Li, Y., Zhang, H. et al. Syntax–Aware graph convolutional network for the recognition of chinese implicit inter-sentence relations. J Supercomput 78, 16529–16552 (2022). https://doi.org/10.1007/s11227-022-04476-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04476-6

Keywords

Search

Navigation