Minimizing False Positives of a Decision Tree Classifier for Intrusion Detection on the Internet

Abstract

Machine learning or data mining technologies are often used in network intrusion detection systems. An intrusion detection system based on machine learning utilizes a classifier to infer the current state from the observed traffic attributes. The problem with learning-based intrusion detection is that it leads to false positives and so incurs unnecessary additional operation costs. This paper investigates a method to decrease the false positives generated by an intrusion detection system that employs a decision tree as its classifier. The paper first points out that the information-gain criterion used in previous studies to select the attributes in the tree-constructing algorithm is not effective in achieving low false positive rates. Instead of the information-gain criterion, this paper proposes a new function that evaluates the goodness of an attribute by considering the significance of error types. The proposed function can successfully choose an attribute that suppresses false positives from the given attribute set and the effectiveness of using it is confirmed experimentally. This paper also examines the more trivial leaf rewriting approach to benchmark the proposed method. The comparison shows that the proposed attribute evaluation function yields better solutions than the leaf rewriting approach.

Appendix: Leaf Rewriting Heuristic

Section 4 uses the following heuristic to evaluate the leaf rewriting approach.

When a decision tree is built, each leaf is assigned a subset of the training data set. An element of the subset is the instance that reaches the leaf as a result of the decision process. For a leaf labeled with an attack state (referred to as an “attack leaf” hereafter), we consider that u _m instances have state s _m (s _m  ∈ S). Let s ₁ be the normal state and let the leaf be labeled s ₂. Then, the employed heuristic computes value e as follows.

$$ e = \frac{{u_{1} + \varepsilon }}{{\sum\nolimits_{m = 2}^{M} {u_{m} } + \varepsilon }} $$

(17)

where ε is a positive constant. In Sect. 4, ε was set at 0.5.

For an attack leaf associated with one or more instances, e < 1 because u ₁ ≤ u ₂. If value e is large, the data subset includes many normal state instances compared to the attack state instances. Thus, it is rational to change the leaf label to “normal.” Because of this, the heuristic sorts the list of attack leaves in decreasing order of e. Let K be the number of attack leaves. The heuristic chooses the first K − L attack leaves in the sorted list, and changes their labels to “normal”. The labels of the remaining L attack leaves are unchanged. The decrease in false positives is controlled by parameter L. If L > K, the heuristic does not modify any leaves.

If no data instances are associated with an attack leaf, u ₁ is replaced by the number of normal state instances at its parent node. This means that e > 1 and e is larger for such a leaf than for a leaf associated with data instances. If a leaf is not associated with any instances, the decision process will reach the leaf only infrequently for the test data as well. Thus, rewriting the label is not likely to increase false negatives for such a leaf. Thus, it is better to change the leaf label than those of other leaves by setting a larger value for e.