Fast and Accurate Steepest-Descent Consistency-Constrained Algorithms for Feature Selection

Abstract

Realizing a good balance to the fundamental trade-off between accuracy and efficiency has been an important problem of feature selection. The algorithm of Interact was an important breakthrough, and the algorithms of Sdcc and Lcc were stemmed from Interact. Lcc has fixed a certain theoretical drawback of Interact in accuracy, while Sdcc has improved accuracy of Interact by expanding the search space. However, when comparing Sdcc and Lcc, we find that Sdcc can output smaller feature sets with smaller Bayesian risks than Lcc (advantages of Sdcc) but can show only worse classification accuracy when used with classifiers (disadvantages). Furthermore, because Sdcc searches answers in much wider spaces than Lcc, it is a few ten times slower in practice. In this paper, we show two methods to improve Sdcc in both accuracy and efficiency and actually propose two algorithms, namely, Fast Sdcc and Accurate Sdcc. We show through experiments that these algorithms can output further smaller feature sets with better classification accuracy than Sdcc. Their classification accuracy appears better than Lcc. In terms of time complexity, Fast Sdcc and Accurate Sdcc improve Sdcc significantly and are only a few times slower than Lcc.