Degrees of freedom in submodular regularization: A computational perspective of Stein’s unbiased risk estimate

https://doi.org/10.1016/j.jmva.2019.104546Get rights and content
Under an Elsevier user license
open archive

Abstract

Degrees of freedom is a covariance penalty related to penalized model selection procedures such as Mallows’ Cp and AIC. We study the degrees of freedom of two polyhedral convex regularization classes defined through submodular functions called the Lovász extension regularization and submodular norm regularization. It has been pointed out that submodular regularization contains many existing penalties that induce structural sparsity. In this paper, we show that the degrees of freedom of submodular regularization estimators can be represented in terms of partitions induced by the estimators. Our formula does not depend on the choice of the design matrix and the penalty function. Moreover, if the design matrix has full column rank, calculating an unbiased estimator of the degrees of freedom requires an additional computational cost of only O(plogp) after a solution for the estimator is obtained, where p is the dimension of the parameter. Existing results for some regularization and projection type estimators, such as the lasso, the fused lasso, and the isotonic regression, are also recovered.

AMS 2010 subject classifications

primary
62J05
secondary
62J07

Keywords

Degrees of freedom
Fused lasso
Lasso
Mallows’ Cp
SLOPE
Stein’s unbiased risk estimate
Structured sparse estimation
Submodular function

Cited by (0)