Commonalities-, specificities-, and dependencies-enhanced multi-task learning network for judicial decision prediction

Abstract

Judicial Decision Prediction (JDP) aims to determine judicial decisions per the fact description of a criminal case. It comprises multiple subtasks, i.e., law article prediction, charge prediction, and term of the penalty prediction. Besides, there exist three properties among the subtasks, i.e., $\mathit{Commonalities},\mathit{Specificities}$, and Dependencies. Nonetheless, existing approaches are usually well-designed for only a specific subtask, or take one of the properties into consideration for multiple subtasks. In this paper, we propose a novel Commonalities-, Specificities- and Dependencies-Enhanced Multi-Task Learning Network, to unify multiple subtasks accompanied by the properties in a framework. Further, while handling the Dependencies, we elaborate a learning module to ensure each subtask to learn contributions from other subtasks to varying degrees, a denoising module to minimize noise interferences among subtasks, and a reinforcing module to guarantee further enhancement for each subtask. Experimental results on two widely used datasets demonstrate that our model significantly and consistently outperforms previous state-of-the-art methods on most evaluation metrics across all subtasks.

Introduction

Judicial Decision Prediction (JDP) refers to determining appropriate judicial decisions (e.g., relevant law articles, charges, and term of penalty) for a criminal case based on its corresponding fact description. JDP is a promising application in legal assistant systems. It helps non-legal professionals to obtain approximate judgments and professionals (e.g., lawyers and judges) to view as a handy reference. Also, JDP is a challenging task compared to fundamental applications of NLP. On the one hand, it contains multiple subtasks, i.e., law article prediction, charge prediction, and term of the penalty prediction, denoted as Task 1, 2, and 3, respectively. On the other hand, it involves a higher level of reasoning and understanding.

In recent years, many methods [1], [2], [3], [4] have been proposed for single subtask (e.g., charge prediction), ignoring the fact that JDP consists of multiple subtasks and possesses the following three properties.

As aforementioned, JDP contains three subtasks, namely, law article prediction, charge prediction and term of penalty prediction. The three subtasks tend to represent some similarities while making predictions according to the given fact description. For example, as shown in Fig. 1, all subtasks are apt to focus on the keyword ”steal” stated in fact description. In other words, the keyword ”steal” could facilitate the prediction of the three subtasks. Through the keyword ”steal”, Task 1 can easily predict the law articles related to it, Taks 2 can directly determine the charge is theft, and Task 3 can predict the approximate term of penalty. We define the similarities among subtasks as commonalities.

In addition to the commonalities among subtasks, each subtask possess some unique characteristics. For example, different subtasks have different emphasizes on different contents of the fact description while making predictions. In detail, as shown in Fig. 1, Task 3 is particularly sensitive to some digits/numbers in the fact description, that is, ”6” and ”10,550”. The numbers represent the number of thefts committed and the amount of money stolen, respectively. The larger the number, the longer the term of penalty predicted. In contrast, Tasks 1 and 2 are less sensitive to numbers, that is, the number of thefts committed and the amount of money stolen have little or nearly no effects on the law and charge prediction. We call the properties specific to a subtask as specificities.

In contrast to the conspicuousness of commonalities and specificities, the relationships among subtasks may be more obscure. During judgments, a judge conforms to a judicial logic. The judge first deduces relevant laws violated by a defendant based on fact description, then determines the charges according to the laws, and finally determines the term of penalty by considering the laws and the charges comprehensively [5]. Besides, subtasks contribute differently to each other. As shown in Fig. 1, the content of the law article stipulates the term of imprisonment (i.e., “fixed-term imprisonment of no more than three years”), which can directly facilitate the prediction of term of penalty (“fixed-term imprisonment of 14 $\mathit{months}$”). Such implicit relationships among subtasks are defined as dependencies.

In this work, we propose a novel Commonalities-, Specificities-, and Dependencies-Enhanced Multi-Task Learning Network, abbreviated as CSDNet, to model multiple subtasks jointly leveraging the three properties aforementioned. Moreover, while handling the dependencies, CSDNet is consistent with the intuition that each subtask contributes to other subtasks to varying degrees and can auto-learn the degrees by a well-designed learning module. Identifying the “degree of contribution” enables one subtask to learn auxiliary information from other subtasks accurately, thereby hugely reducing noise interferences. Even so, the noise interferences cannot be wholly eradicated. In order to minimize noise interferences among subtasks to the hilt, we adopt a similar approach with the learning module to devise a denoising module, that is, learning the “degree of contamination” caused by one subtask to other subtasks. Through “degree of contamination”, one subtask can properly eliminate the noise interferences caused by the interaction with other subtasks. After learning and denoising modules, the original information of one subtask may be suffering from the occupation of other subtasks and lose its dominance. Therefore, to avoid this situation, we invent a reinforcement module to learn the “degree of enhancement” of a subtask to ensure that each subtask still occupies a dominant position after incorporating the information of other subtasks.

Notably, to the best of our knowledge, we are the first to incorporate the commonalities, specificities, and dependencies in a unified network and the first to figure out the issue of noise interferences among subtasks in JDP. Besides, we propose a combination of LSTMs [6], named Q-LSTM, and make it as the fundamental component of CSDNet.

In summary, our key contributions lie on the following aspects:

• We propose a novel multi-task learning network to address multiple subtasks of JDP incorporating the three properties, i.e., Commonalities, Specificities, and Dependencies.

• We elaborate a Learning Module to let each subtask to learn contributions from other subtasks to varying degrees, a Denoising Module to eliminate noise interferences, and a Reinforcing Module to achieve further enhancement.

• We invent a combination of LSTMs, named Q-LSTM, which excels conventional deep neural networks, i.e., CNN, LSTM, HLSTM and stacked BiLSTM over single subtask of JDP. In addition, the proposed Q-LSTM shows competitive generalization performance in text classification, named entity recognition and part of speech tagging tasks.

• Extensive experiments on two datasets demonstrate that CSDNet significantly and consistently exceeds previous state-of-the-art methods across all representative subtasks.