Convex and nonconvex nonparametric frontier-based classification methods for anomaly detection

Abstract

Effective methods for determining the boundary of the normal class are very useful for detecting anomalies in commercial or security applications—a problem known as anomaly detection. This contribution proposes a nonparametric frontier-based classification (NPFC) method for anomaly detection. By relaxing the commonly used convexity assumption in the literature, a nonconvex-NPFC method is constructed and the nonconvex nonparametric frontier turns out to provide a more conservative boundary enveloping the normal class. By reflecting on the monotonic relation between the characteristic variables and the membership, the proposed NPFC method is in a more general form since both input-like and output-like characteristic variables are incorporated. In addition, by allowing some of the training observations to be misclassified, the convex- and nonconvex-NPFC methods are extended from a hard nonparametric frontier to a soft one, which also provides a more conservative boundary enclosing the normal class. Both simulation studies and a real-life data set are used to evaluate and compare the proposed NPFC methods to some well-established methods in the literature. The results show that the proposed NPFC methods have competitive classification performance and have consistent advantages in detecting abnormal samples, especially the nonconvex-NPFC methods.

Appendix: Discussions on the choices of cut-off super-efficiency

In this contribution, a negative cut-off super-efficiency denoted by $c_{\Lambda }$ is introduced to decide the noisy and less important training observations. A larger value of $c_{\Lambda }$ means that more training observations will be identified as noisy and of low importance and thus, will be excluded from the construction of a soft nonparametric frontier.

For both the C and NC cases in Sect. 3.2, the simulation is executed for 100 times. Here, an example for the C case is extracted to show different C and NC soft nonparametric frontiers constructed from different choices of $c_{\Lambda }$.

The resulted frontiers under different choices of $c_{\Lambda }$ are displayed in Figs. 5 and 6. In every sub-figure, the normal training observations are represented by blue crosses. The training observations that are identified as noisy and of low importance are further marked by red circles. These training observations are excluded while constructing the corresponding soft nonparametric frontier. The soft nonparametric frontier is represented by the blue solid lines.

Fig. 5

C soft nonparametric frontiers with different choices of $c_{\Lambda }$

Fig. 6

NC soft nonparametric frontiers with different choices of $c_{\Lambda }$

Similar observations can be derived from Figs. 5 and 6. With the increase in $c_{\Lambda }$, more training observations are identified as noisy and of low importance. Accordingly, the soft nonparametric frontier becomes more conservative. In comparison with the C soft nonparametric frontier, the NC soft nonparametric frontier is more conservative, since more training observations are excluded.

A proper choice of $c_{\Lambda }$ is important for identifying as accurately as possible noisy and less important training observations. In the following, the suggested way of deciding the value of $c_{\Lambda }$ is explained with the above simulation example.

Fig. 7

Illustrative diagram of deciding a cut-off super-efficiency

By solving model (7) under the C case for 700 normal training observations, 5 of them are identified as frontier observations. Their values of $\delta _{\text {super},\Lambda ^\text {C}}^*$ ordered from largest to smallest are represented by blue diamonds in Fig. 7a. It is observed that 4 out of 5 frontier observations have a $\delta _{\text {super},\Lambda ^\text {C}}^*$ larger than $-$0.1. Only one frontier observations has $\delta _{\text {super},\Lambda ^\text {C}}^*=-0.1537$. Thus, $c_{\Lambda ^\text {C}}=-0.1$ is chosen.

Similarly, by solving model (7) under the NC case for the same normal training observations, 11 of them are identified as frontier observations. Their values of $\delta _{\text {super},\Lambda ^\text {NC}}^*$ ordered from largest to smallest are as shown in Fig. 7b. It is observed that 10 out of 11 frontier observations have the value of $\delta _{\text {super},\Lambda ^\text {NC}}^*$ larger than $-$0.2. Only one frontier observations has $\delta _{\text {super},\Lambda ^\text {NC}}^*=-0.2494$. Thus, $c_{\Lambda ^\text {NC}}=-0.2$ is chosen.

One suggested way of deciding the value of $c_{\Lambda }$ is illustrated. Note that we do not intend to suggest that this is the optimal way. It is worthwhile for future researches to explore the other methods of deciding a proper $c_{\Lambda }$.