A k–Skyband Approach for Feature Selection

Abstract

Distance concentration is a phantom menace for the labeling of high dimensional data by distance-based classifiers. Filter methods reduce data dimensionality, but they also add their ranking bias indirectly into the classification procedure. In this study, we examine the filtering problem from another perspective, in which multiple filters are aggregated according to classifiers’ constraints. Our approach, named S-Filter, is designed as a top-k skyline (k-skyband) search over multiple rankings by relying on the concept of $\mathcal {F}$–dominance for weighted and monotone linear functions. Unlike existing approaches, S-Filter provides a deterministic strategy for joining multiple filters and avoids the semantic problem of breaking top-k ties. S-Filter’s first stage uses labeling-driven measures, e.g., F1-Score, for assessing the quality of each filter with regards to a particular classifier, whereas range-tolerance intervals around the initial quality measures define the partial search weights. Next, S-Filter applies the FSA instance-optimal algorithm for selecting all the dimensions that can be among the top-k for a weight within the range-tolerance intervals. Experiments on high dimensional datasets show that S-Filter outperforms state-of-the-art filters in two scenarios: (i) exploratory analysis on varying k and range-tolerance intervals, and (ii) data reduction to its intrinsic dimensionality.

The authors thank the National Council for Scientific and Technological Development and Faperj (G. E-26/203.215/2016 and I. Sed. 2018) for their financial support.