Non-convex approximation based l₀-norm multiple indefinite kernel feature selection

Abstract

Multiple kernel learning (MKL) for feature selection utilizes kernels to explore complex properties of features, which has been shown to be among the most effective for feature selection. To perform feature selection, a natural way is to use the l₀-norm to get sparse solutions. However, the optimization problem involving l₀-norm is NP-hard. Therefore, previous MKL methods typically utilize a l₁-norm to get sparse kernel combinations. However, the l₁-norm, as a convex approximation of l₀-norm, sometimes cannot attain the desired solution of the l₀-norm regularizer problem and may lead to prediction accuracy loss. In contrast, various non-convex approximations of l₀-norm have been proposed and perform better in many linear feature selection methods. In this paper, we propose a novel l₀-norm based MKL method (l₀-MKL) for feature selection with non-convex approximations constraint on kernel combination coefficients to select features automatically. Considering the better empirical performance of indefinite kernels than positive kernels, our l₀-MKL is built on the primal form of multiple indefinite kernel learning for feature selection. The non-convex optimization problem of l₀-MKL is further refumated as a difference of convex functions (DC) programming and solved by DC algorithm (DCA). Experiments on real-world datasets demonstrate that l₀-MKL is superior to some related state-of-the-art methods in both feature selection and classification performance.