1 Introduction

Classifier selection and evaluation are difficult tasks requiring time and knowledge about the underlying data. One of the most important ingredients when assessing classifiers is the used classification performance measure. An analogous decision has to be made in association rule mining, where the overwhelming number of generated rules is usually trimmed by a selected interestingness measure. However, many researchers often carry out their experiments with respect to few selected measures, without discussing their properties and justifying their choice simply by the measure’s popularity.

To aid the analysis of properties of measures based on two-by-two contingency tables, we put forward Tetrahedron, a web-based visualization tool for analyzing entire ranges of measure values. The proposed application visualizes 4D data in 3D using the barycentric coordinate system [1, 2]. Tetrahedron produces 3D WebGL plots with zooming, rotating, animation, and detailed configuration capabilities. The presented tool can be used to compare properties of existing measures, as well as devise new metrics.

2 The Visualization Technique

A confusion matrix for binary classification (Table 1) consists of four entries: \( TP \), \( FP \), \( FN \), \( TN \). However, for a dataset of n examples these four entries are sum-constrained, as \(n = TP + FP + FN + TN \) . Therefore, for a given constant n, any three values in the confusion matrix uniquely define the fourth value. This property allows to visualize any classification performance measure based on the two-class confusion matrix using a 4D barycentric coordinate system, tailored to sum-constrained data. The same holds for any \(2\times 2\) matrix, for example, those used to define rule interestingness measures [2].

Table 1. Confusion matrix for two-class classification
Full size table
Fig. 1.
figure 1

Tetrahedron

Full size image

The barycentric coordinate system is a coordinate system in which point locations are specified relatively to hyper-sides of a simplex. A 4D barycentric coordinate system is a tetrahedron, where each dimension is represented as one of the four vertices. Choosing vectors that represent \( TP \), \( FP \), \( FN \), \( TN \) as vertices of a regular tetrahedron in a 3D space, one arrives at a barycentric coordinate system depicted in Fig. 1.

In this system, every confusion matrix \(\left[ {\begin{matrix} TP &{} FN \\ FP &{} TN \end{matrix}} \right] \) is represented as a point of the tetrahedron. Let us illustrate this fact with a few examples. Figure 1 shows a skeleton of a tetrahedron with four exemplary points:

  • one located in vertex \(\mathsf {TP}\), which represents \(\left[ {\begin{matrix} n &{} 0 \\ 0 &{} 0 \end{matrix}} \right] \),

  • one located in the middle of edge \(\mathsf {TP}\)–\(\mathsf {FP}\), which represents \(\left[ {\begin{matrix} n/2 &{} 0 \\ n/2 &{} 0 \end{matrix}} \right] \),

  • one located in the middle of face \(\triangle \mathsf {TP}\)–\(\mathsf {FP}\)–\(\mathsf {FN}\), which represents \(\left[ {\begin{matrix} n/3 &{} n/3 \\ n/3 &{} 0 \end{matrix}} \right] \),

  • one located in the middle of the tetrahedron, which represents \(\left[ {\begin{matrix} n/4 &{} n/4 \\ n/4 &{} n/4 \end{matrix}} \right] \).

One way of understanding this representation is to imagine a point in the tetrahedron as the center of mass of the examples in a confusion matrix. If all n examples are true positives, then the entire mass of the predictions is at \( TP \) and the point coincides with vertex \(\mathsf {TP}\). If all examples are false negatives, the point lies on vertex \(\mathsf {FN}\), etc. Generally, whenever \(a > b\) (\(a, b \in \{ TP , FN , FP , TN \}\)) then the point is closer to the vertex corresponding to a rather than b.

Using the barycentric coordinate system makes it possible to depict the originally 4D data (two-class confusion matrices) as points in 3D. Moreover, an additional variable based on the depicted four values may be rendered as color. In the presented tool, we adapt this procedure to color-code the values of classification performance and rule interestingness measures. A more in-depth description of the visualization and its possible applications can be found in [1, 2].

3 Tool Overview

The described visualization technique has been implemented as an interactive web-based application. An online version, compatible with all modern web browsers across different client platforms, is publicly availableFootnote 1. The application can visualize 86 predefined 4D measures, including 21 classification measures, 16 rule interestingness measures, and 49 general-purpose formulas based on a two-by-two matrix. The user can also visualize custom measures by providing their formula. The main functionalities of the application are:

Fig. 2.
figure 2

Application overview

Full size image

  • Interactive 3D tetrahedron visualization. The visualization (Fig. 2a) supports: 86 predefined measures, rotating, zooming, four rendering precisions, saving as an html with WebGL, and exporting images. The user may choose to visualize external views, inner layers, and control point-padding.

  • Cross-sections. A useful way of visualizing measure values can also be achieved be cutting the tetrahedron with a plane and analyzing the obtained slice. In this application the user can visualize cross-sections (Fig. 2b) which correspond to different class distributions. Interestingly, this particular kind of cross-sections produces a 2D space analogous to that used in ROC charts.

  • Parameter animations. Several of the application options can be animated. These options can change the visualization parameters automatically in constant intervals creating an animation (Fig. 2c). Such animations can be useful when attempting to analyze: consecutive layers of the tetrahedron, the impact of measure parameters (e.g. the impact of \(\beta \) in F\(_\beta \)-score), or the effect of changing class distributions on cross-sections.

  • Custom measure definition. It is possible to define a custom measure to be visualized by providing its formula (Fig. 2d).

Since classification accuracy is one of the most intuitive performance measures, let us use it to exemplify visualizations produced by our tool with the default (blue: 0, red: 1) color map. One can notice that confusion matrices with a high number of \( FP \) and \( FN \) result in low accuracy (blue), whereas high \( TP \) and \( TN \) yield high accuracy (red). Cross-sections for two different class ratios show that on imbalanced data high accuracy can be achieved by trivial majority classifiers. More examples of visual-based analyses can be found in [1, 2] (Fig. 3).

Fig. 3.
figure 3

Visualizations of classification accuracy (Color figure online)

Full size image

4 Conclusions

We propose Tetrahedron, a web-based visualization tool for analyzing and prototyping measures based on a two-by-two matrix. Its main features include: interactive 3D WebGL barycentric plots, zooming, parameter animation, performing cross-sections, providing custom measure formulas, and saving plots with a single click. Such functionality facilitates visual inspection of various measure properties, such as determining measure monotonicity, symmetries, maximas, or undefined values. Thus, the presented tool can be used to gain further understanding of existing machine learning measures, as well as devise new ones.