Skip to main content

Advertisement

SpringerLink
Log in

Toward automatic support for leading court debates: a novel task proposal & effective approach of judicial question generation

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Court debate, with multiple parties (e.g., judge, plaintiff, defendant), is an essential component in a civil trial where the judge leads the conversation and the litigants respond in turn following the judge’s question. Unlike other types of dialogues, the judge’s leading role can be critical with respect to the goal of case investigation, and it is non-trivial to examine the case logic considering also the need for specialized domain knowledge. Judge question generation in court debate is a novel but significant task to assist/train the junior judges to raise effective questions in a legal context as well as help the litigants prepare a court debate in advance. We propose an innovative end-to-end model called ’Judicial Questioning Aid’ which is capable of proactively leading the court debate by asking useful questions to a certain litigant given previous context. Unlike prior efforts in Natural Language Generation (NLG), the proposed model encodes the contextual utterance information with respect to global legal knowledge and local case judicial factors, as well as simulates the intention switch across different conversation turns. Extensive experiments based on a large civil trial dataset show that the proposed model can generate more accurate and readable questions against several alternatives in the multi-party court debate scene.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

The authors will release the data after publication.

Code availability

The authors will release the code after publication.

Notes

  1. The modern litigation of most countries adopts the basic structure of ’two opposites and judge in center’ where a civil action involves the resolution of a dispute between two or more parties through the court system [3,4,5].

  2. Sensitive information (e.g., person’s name) has been removed for privacy assurance.

  3. The knowledge graph we utilize is defined by experienced judges to ensure its quality and accuracy.

  4. https://github.com/jichangzhen/JQA

  5. It can be used to filter redundant information of the \(\mathcal {Z}\), and compared with Bi-LSTM, the model convergence speed can be accelerated.

  6. A neural network model that can directly copy the words from context information, to solve Out-Of-Vocabulary problem.

  7. Private lending dispute cases are the most frequent cause of civil cases in X (anonymized). This data set is provided by the High Court of X. All court transcripts are manually recorded by a court clerk.

  8. The last utterance of each sample comes from a judge. We use it as the output and all its context is treated as input. All of the output cases covered more than \(99\%\) of the questions a judge asked during the trial.

  9. Note also that to implement the encoders in the baselines, we concatenated the role embeddings with word embeddings representing the context, same as we did in our proposed method.

  10. All the annotators took basic annotation training before the experimental study.

  11. As the intent relies on the path discovered in the TLT, in the ablation test of this feature, we only remove its usage as described in Sect. 4.2.1 while keeping it in the intent enhancement part.

  12. Examples of procedural questions: Does the plaintiff have any supplements?, Does the plaintiff apply for withdrawal?.

  13. Examples of factual questions: When did the plaintiff get divorced?, When was the loan issued?, Where was the loan delivered?.

References

  1. Center JI Some federal judges handle inordinate caseloads. available from https://trac.syr.edu/tracreports/judge/501/. Accessed: 2019-05-06

  2. Center JI As workloads rise in federal courts, judge counts remain flat. available from https://trac.syr.edu/tracreports/judge/364/. Accessed: 2019-05-06

  3. CMPA-Legal, regulatory proceedings: When multiple defendants are involved in a legal action, things can get complicated. Accessed: 2020-01-22

  4. Andrews N (2001) Multi-party proceedings in england: representative and group actions. Duke J Comp Int’l L 11:249

    Google Scholar 

  5. Hanotiau B, Lee JB (2007) Complex arbitrations-multiparty, multicontract, multi-issue and class actions. Rev Bras Arb 4:182

    Google Scholar 

  6. Deng Y, Li Y, Sun F, Ding B, Lam W (2021) Unified conversational recommendation policy learning via graph-based reinforcement learning. In: 44th International ACM SIGIR conference on research and development in information retrieval . ACM

  7. Wang J, Liu J, Bi W, Liu X, He K, Xu R, Yang M (2020) Improving knowledge-aware dialogue generation via knowledge base question answering. In: Proceedings of the AAAI conference on artificial intelligence, pp. 9169–9176

  8. Kaiser M, Saha Roy R, Weikum G (2021) Reinforcement learning from reformulations in conversational question answering over knowledge graphs. In: 44th International ACM SIGIR conference on research and development in information retrieval . ACM

  9. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A.N, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Conference on neural information processing systems, pp. 5998–6008

  10. Nagel S (1960) Using simple calculations to predict judicial decisions. American Behav Sci 4(4):24–28

    Article  Google Scholar 

  11. McElvain G, Sanchez G, Matthews S, Teo D, Pompili F, Custis T (2019) Westsearch plus: A non-factoid question-answering system for the legal domain. In: Proceedings of the 42rd international ACM SIGIR conference on research and development in information retrieval, pp. 1361–1364

  12. Biega A.J, Potash P, Daumé H, Diaz F, Finck M (2020) Operationalizing the legal principle of data minimization for personalization. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp. 399–408

  13. Bhattacharya P, Ghosh K, Pal A, Ghosh S (2020) Hier-spcnet: A legal statute hierarchy-based heterogeneous network for computing legal case document similarity. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp. 1657–1660

  14. Shao Y (2020) Towards legal case retrieval. In: Proceedings of the 43rd International ACM SIGIR conference on research and development in information retrieval, pp. 2485–2485

  15. Dong Q, Niu S (2021) Legal judgment prediction via relational learning. In: Proceedings of the 44rd international ACM SIGIR conference on research and development in information retrieval, pp. 983–992

  16. Ma L, Zhang Y, Wang T, Liu X, Ye W, Sun C, Zhang S (2021) Legal judgment prediction with multi-stage case representation learning in the real court setting. In: Proceedings of the 44rd international ACM SIGIR conference on research and development in information retrieval, pp. 993–1002

  17. Shao Y, Wu Y, Liu Y, Mao J, Zhang M, Ma S (2021) Investigating user behavior in legal case retrieval. In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pp. 962–972

  18. Ma Y, Shao Y, Wu Y, Liu Y, Zhang R, Zhang M, Ma S (2021) Lecard: a legal case retrieval dataset for chinese law system. In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pp. 2342–2348

  19. Roegiest A, Lee E (2019) On tradeoffs between document signature methods for a legal due diligence corpus. In: Proceedings of the 42rd international ACM SIGIR conference on research and development in information retrieval, pp. 1001–1004

  20. Yue L, Liu Q, Jin B, Wu H, Zhang K, An Y, Cheng M, Yin B, Wu D (2021) Neurjudge: A circumstance-aware neural framework for legal judgment prediction. In: Proceedings of the 44rd international ACM SIGIR conference on research and development in information retrieval, pp. 973–982

  21. Liu B, Wu Y, Liu Y, Zhang F, Shao Y, Li C, Zhang M, Ma S (2021) Conversational vs traditional: Comparing search behavior and outcome in legal case retrieval. In: Proceedings of the 44rd international ACM SIGIR conference on research and development in information retrieval, pp. 1622–1626

  22. Ashley KD (2017) Artificial intelligence and legal analytics: new tools for law practice in the digital age. Cambridge University Press, England

    Book  Google Scholar 

  23. Li J, Yang M, Li C (2021) CLC-RS: a Chinese legal case retrieval system with masked language ranking. Association for Computing Machinery, New York, pp 4734–4738

    Google Scholar 

  24. Zhou X, Zhang Y, Liu X, Sun C, Si L (2019) Legal intelligence for e-commerce: multi-task learning by leveraging multiview dispute representation. In: Proceedings of the 42rd international ACM SIGIR conference on research and development in information retrieval, pp. 315–324

  25. Wang P, Fan Y, Niu S, Yang Z, Zhang Y, Guo J (2019) Hierarchical matching network for crime classification. In: Proceedings of the 42rd international ACM SIGIR conference on research and development in information retrieval, pp. 325–334

  26. Kanapala A, Pal S, Pamula R (2019) Text summarization from legal documents: a survey. Artif Intell Rev 51(3):371–402

    Article  Google Scholar 

  27. Merchant K, Pande Y (2018) Nlp based latent semantic analysis for legal text summarization. In: 2018 International conference on advances in computing, communications and informatics (ICACCI), pp. 1803–1807 . IEEE

  28. Duan X, Zhang Y, Yuan L, Zhou X, Liu X, Wang T, Wang R, Zhang Q, Sun C, Wu F (2019) Legal summarization for multi-role debate dialogue via controversy focus mining and multi-task learning. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp. 1361–1370

  29. Fawei B, Pan J.Z, Kollingbaum M, Wyner A.Z (2018) A methodology for a criminal law and procedure ontology for legal question answering. In: Joint international semantic technology conference, pp. 198–214

  30. Wang Z, Wang B, Duan X, Wu D, Wang S, Hu G, Liu T (2019) IFlyLegal: a Chinese legal system for consultation, law searching, and document analysis. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 97–102

  31. Bin W, Kun K, Changlong S, Jun F, Yating Z, Xinli Z, Jianghong Z, Yinsheng Z, Fei W (2022) A full-process intelligent trial system for smart court. Front Inf Technol Electron Eng 23(2):186

    Article  Google Scholar 

  32. Weizenbaum J (1966) Eliza-a computer program for the study of natural language communication between man and machine. Commun ACM 9(1):36–45

    Article  Google Scholar 

  33. Schatzmann J, Weilhammer K, Stuttle M, Young S (2006) A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. The knowledge engineering review 21(2):97–126

    Article  Google Scholar 

  34. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Conference neural information processing system, pp. 27:3104–3112

  35. Luong T, Pham H, Manning C.D (2015) Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp. 1412–1421

  36. Bahdanau D, Cho K, Bengio, Y (2015) Neural machine translation by jointly learning to align and translate. In: International conference on learning representations

  37. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  38. Gehring J, Auli M, Grangier D, Yarats D, Dauphin Y.N (2017) Convolutional sequence to sequence learning. In: International conference on machine learning, pp. 1243–1252 . PMLR

  39. Kalchbrenner N, Espeholt L, Simonyan K, Oord Avd, Graves A, Kavukcuoglu K (2016) Neural machine translation in linear time. arXiv preprint arXiv:1610.10099

  40. See A, Liu P.J, Manning C.D (2017) Get to the point: Summarization with pointer-generator networks. In: Proceedings of the 55th annual meeting of the association for computational linguistics, pp. 1073–1083

  41. Gu J, Lu Z, Li H, Li VOK (2016) Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th annual meeting of the association for computational linguistics, pp. 1631–1640

  42. Zhang H, Lan Y, Pang L, Guo J, Cheng X (2019) Recosa: Detecting the relevant contexts with self-attention for multi-turn dialogue generation. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 3721–3730

  43. Cai D, Wang Y, Bi W, Tu Z, Liu X, Shi S (2019) Retrieval-guided dialogue response generation via a matching-to-generation framework. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp. 1866–1875

  44. Zhang Z, Li J, Zhu P, Zhao H, Liu G (2018) Modeling multi-turn conversation with deep utterance aggregation. In: Proceedings of the 27th international conference on computational linguistics: system demonstrations, pp. 3740–3752

  45. Tan M, Wang D, Gao Y, Wang H, Potdar S, Guo X, Chang S, Yu M (2019) Context-aware conversation thread detection in multi-party chat. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp. 6456–6461

  46. Zhang R, Lee H, Polymenakos L, Radev D (2018) Addressee and response selection in multi-party conversations with speaker interaction rnns. In: Proceedings of the AAAI conference on artificial intelligence, pp. 5690–5697

  47. Le R, Hu W, Shang M, You Z, Bing L, Zhao D, Yan R (2019) Who is speaking to whom? learning to identify utterance addressee in multi-party conversations. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp. 1909–1919

  48. Liu S, Chen H, Ren Z, Feng Y, Liu Q, Yin D (2018) Knowledge diffusion for neural dialogue generation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 1489–1498

  49. Zhou H, Young T, Huang M, Zhao H, Xu J, Zhu X (2018) Commonsense knowledge aware conversation generation with graph attention. In: IJCAI International joint conference on artificial intelligence, pp. 4623–4629

  50. Meng C, Ren P, Chen Z, Monz C, Ma J, de Rijke M (2020) Refnet: A reference-aware network for background based conversation. In: Proceedings of the AAAI conference on artificial intelligence, pp. 8496–8503

  51. Liu Z, Niu Z.-Y, Wu H, Wang H (2019) Knowledge aware conversation generation with explainable reasoning over augmented graphs. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp. 1782–1792

  52. Zhong P, Wang D, Miao C (2019) Knowledge-enriched transformer for emotion detection in textual conversations. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp. 165–176

  53. Tang D, Duan N, Qin T, Yan Z, Zhou M (2017) Question answering and question generation as dual tasks

  54. Du X, Cardie C (2017) Identifying where to focus in reading comprehension for neural question generation. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp. 2067–2073

  55. Wang T, Yuan X, Trischler A (2017) A joint model for question answering and question generation

  56. Guo H, Pasunuru R, Bansal M (2018) Soft layer-specific multi-task summarization with entailment and question generation. In: Proceedings of the 56th annual meeting of the association for computational linguistics, pp. 687–697

  57. Elsahar H, Gravier C, Laforest F (2018) Zero-shot question generation from knowledge graphs for unseen predicates and entity types. In: Proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies

  58. Zhou X, Li L, Dong D, Liu Y, Chen Y, Zhao W.X, Yu D, Wu H (2018) Multi-turn response selection for chatbots with deep attention matching network. In: Proceedings of the 56th annual meeting of the association for computational linguistics, pp. 1118–1127

  59. Ba JL, Kiros JR, Hinton GE (2016) Layer normalization. stat 1050:21

  60. Goodfellow I, Bengio Y, Courville A (2016) Deep Learning

  61. Nallapati R, Zhou B, Gulcehre C, Xiang B, et al: (2016) Abstractive text summarization using sequence-to-sequence rnns and beyond. In: Proceedings of the 20th SIGNLL conference on computational natural language learning, pp. 280–290

  62. Serban I, Sordoni A, Bengio Y, Courville A, Pineau J (2016) Building end-to-end dialogue systems using generative hierarchical neural network models. In: Proceedings of the AAAI conference on artificial intelligence, pp. 3776–3783

  63. Lin C.-Y, Hovy E (2003) Automatic evaluation of summaries using n-gram co-occurrence statistics. In: Proceedings of the 2003 conference of the north american chapter of the association for computational linguistics: human language technologies, pp. 150–157

  64. Papineni K, Roukos S, Ward T, Zhu W.-J (2002) Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the association for computational linguistics, pp. 311–318

  65. Ke P, Guan J, Huang M, Zhu X (2018) Generating informative responses with controlled sentence function. In: Proceedings of the 56th annual meeting of the association for computational linguistics, pp. 1499–1508

  66. Zhu Q, Cui L, Zhang W.-N, Wei F, Liu T (2019) Retrieval-enhanced adversarial training for neural response generation. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 3763–3773

  67. Pitoura E, Tsaparas P, Flouris G, Fundulaki I, Papadakos P, Abiteboul S, Weikum G (2018) On measuring bias in online information. ACM SIGMOD Record, 16–21

  68. Mahoney K (2015) Judicial bias: the ongoing challenge. J Disput Res 2015(1):4

    Google Scholar 

Download references

Acknowledgements

This work has been supported by the National Key R &D Program of China (2018YFC0830200; 2018YFC0830206; 2020YFB1406902 and 2020YFC0832505).

Funding

The research leading to these results received funding from [National Key R &D Program of China] under Grant Agreement No [2018YFC0830200; 2018YFC0830206; 2020YFB1406902 and 2020YFC0832505].

Author information

Authors and Affiliations

  1. Harbin Institute of Technology, Harbin, Heilongjiang, China

    Changzhen Ji, Conghui Zhu & Tiejun Zhao

  2. Alibaba Group, Hangzhou, Zhejiang, China

    Yating Zhang & Changlong Sun

  3. Worcester Polytechnic Institute, Worcester, Massachusetts, USA

    Xiaozhong Liu

  4. University of Innsbruck, Innrain, Innsbruck, Austria

    Adam Jatowt

  5. Nanyang Technological University, Nanyang Avenue, Singapore

    Sourav S. Bhowmick

Authors
  1. Changzhen Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yating Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaozhong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Adam Jatowt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sourav S. Bhowmick
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Changlong Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Conghui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tiejun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Changzhen Ji and Yating Zhang contributed to conceptualization; Changzhen Ji, Yating Zhang and Xiaozhong Liu contributed to methodology; Changzhen Ji contributed to Software, Changzhen Ji contributed to validation, Changzhen Ji contributed to investigation, Changzhen Ji and Yating Zhang contributed to Formal analysis; Changzhen Ji, Yating Zhang and Xiaozhong Liu contributed to writing-original draft preparation; Yating Zhang, Xiaozhong Liu, Adam Jatowt, and Sourav S Bhowmick contributed to writing-review and editing; Yating Zhang, Changlong Sun, Conghui Zhu and Tiejun Zhao contributed to Funding acquisition; Changzhen Ji contributed to resources; Yating Zhang contributed to supervision.

Corresponding author

Correspondence to Conghui Zhu.

Ethics declarations

Conflict of interest

All the authors declare they have no Conflict of interest/Competing interests.

Consent for publication

All the authors have given their consent to publish the paper.

Consent to participate

All the authors have given their consent to participate.

Ethics approval

The content of the paper is written in accordance to common ethical standards in the scientific community.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, C., Zhang, Y., Liu, X. et al. Toward automatic support for leading court debates: a novel task proposal & effective approach of judicial question generation. Neural Comput & Applic 34, 16367–16385 (2022). https://doi.org/10.1007/s00521-022-07588-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07588-5

Keywords

Search

Navigation