A group VISA algorithm for variable selection

Abstract

We consider the problem of selecting grouped variables in a linear regression model based on penalized least squares. The group-Lasso and the group-Lars procedures are designed for automatically performing both the shrinkage and the selection of important groups of variables. However, since the same tuning parameter is used (as in Lasso or Lars ) for both group variable selection and shrinkage coefficients, it can lead to over shrinkage the significant groups of variables or inclusion of many irrelevant groups of predictors. This situation occurs when the true number of non-zero groups of coefficients is small relative to the number $p$ of variables. We introduce a novel sparse regression method, called the Group-VISA (GVISA), which extends the VISA effect to grouped variables. It combines the idea of VISA algorithm which avoids the over shrinkage problem of regression coefficients and the idea of the GLars-type estimator which shrinks and selects the members of the group together. Hence, GVISA is able to select a sparse group model by avoiding the over shrinkage of GLars-type estimator. We distinguish two variants of the GVISA algorithm, each one is associated with each version of GLars (I and II). Moreover, we provide a path algorithm, similar to GLars, for efficiently computing the entire sample path of GVISA coefficients. We establish a theoretical property on sparsity inequality of GVISA estimator that is a non-asymptotic bound on the estimation error. A detailed simulation study in small and high dimensional settings is performed, which illustrates the advantages of the new approach in relation to several other possible methods. Finally, we apply GVISA on two real data sets.