Semantic types of legal norms in German laws: classification and analysis using local linear explanations

Abstract

This paper describes the automated classification of legal norms in German statutes with regard to their semantic type. We propose a semantic type taxonomy for norms in the German civil law domain consisting of nine different types focusing on functional aspects, such as Duties, Prohibitions, Permissions, etc. We performed four iterations in classifying legal norms with a rule-based approach using a manually labeled dataset, i.e., tenancy law, of the German Civil Code ($\hbox {n} = 601$). During this experiment the $F_1$ score continuously improved from 0.52 to 0.78. In contrast, a machine learning based approach for the classification was implemented. A performance of $F_1 = 0.83$ was reached. Traditionally, machine learning classifiers lack of transparency with regard to their decisions. We extended our approach using so-called local linear approximations, which is a novel technique to analyze and inspect a trained classifier’s behavior. We can show that there are significant similarities of manually crafted knowledge, i.e., rules and pattern definitions, and the trained decision structures of machine learning approaches.