Skip to main content
SpringerLink
Log in

Quantile regression feature selection and estimation with grouped variables using Huber approximation

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

This paper considers model selection and estimation for quantile regression with a known group structure in the predictors. For the median case the model is estimated by minimizing a penalized objective function with Huber loss and the group lasso penalty. While, for other quantiles an M-quantile approach, an asymmetric version of Huber loss, is used which approximates the standard quantile loss function. This approximation allows for efficient implementation of algorithms which rely on a differentiable loss function. Rates of convergence are provided which demonstrate the potential advantages of using the group penalty and that bias from the Huber-type approximation vanishes asymptotically. An efficient algorithm is discussed, which provides fast and accurate estimation for quantile regression models. Simulation and empirical results are provided to demonstrate the effectiveness of the proposed algorithm and support the theoretical results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Alfo, M., Salvati, N., Ranallli, M.G.: Finite mixtures of quantile and m-quantile regression models. Stat. Comput. 27(2), 547–570 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Belloni, A., Chernozhukov, V.: L1-penalized quantile regression in high-dimensional sparse models. Ann. Statist. 39(1), 82–130 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Bianchi, A., Salvati, N.: Asymptotic properties and variance estimators of the m-quantile regression coefficients estimators. Commun. Stat. Theory Methods 44(11), 2416–2429 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Bianchi, A., Fabrizi, E., Salvati, N., et al.: Estimation and testing in m-quantile regression with applications to small area estimation. Int. Stat. Rev. 86(3), 541–570 (2018)

    Article  MathSciNet  Google Scholar 

  • Bickel, P.J., Ritov, Y., Tsybakov, A.B.: Simultaneous analysis of lasso and dantzig selector. Ann. Statist 37(4), 1705–1732 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Breckling, J., Chambers, R.: M-quantiles. Biometrika 75(4), 761–771 (1988)

    Article  Google Scholar 

  • Breheny, P., Huang, J.: Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors. Stat. Comput. 25, 173–187 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Breheny, P., Zeng, Y.: grpreg: regularization paths for regression models with grouped covariates 3.1-2. R package version 3.3.1 (2017)

  • Chambers, R., Tzavidis, N.: M-quantile models for small area estimation. Biometrika 93(2), 255–268 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Ciuperca, G.: Adaptive group lasso selection in quantile models. Statist. Papers 60(1), 173–197 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • De Cock, D.: Ames, Iowa: alternative to the boston housing data as an end of semester regression project. J. Stat. Educ. 19(3), 1–15 (2011)

    Google Scholar 

  • Del Sarto, S., Marino, M.F., Ranalli, M.G., et al.: Using finite mixtures of m-quantile regression models to handle unobserved heterogeneity in assessing the effect of meteorology and traffic on air quality. Stoch. Environ. Res. Risk Assess 33(7), 1345–1359 (2019)

    Article  Google Scholar 

  • Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Assoc. 96(456), 1348–1360 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Fan, J., Li, Q., Wang, Y.: Estimation of high dimensional mean regression in the absence of symmetry and light tail assumptions. J. R. Stat. Soc. Ser. B Stat Methodol. 79(1), 247–265 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Fasiolo, M., Wood, S.N., Zaffran, M., et al.: Fast calibrated additive quantile regression. J. Am. Statist. Assoc. 116(535), 1402–1412 (2021)

    Article  MathSciNet  Google Scholar 

  • Friedman, J., Hastie, T., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33(1), 1–22 (2010)

    Article  Google Scholar 

  • He, X., Shao, Q.M.: On parameters of increasing dimensions. J. Multivariate Anal. 73, 120–135 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Huang, J., Zhang, T.: The benefit of group sparsity. Ann. Statist. 38(4), 1978–2004 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Huang, J., Horowitz, J.L., Wei, F.: Variable selection in nonparametric additive models. Ann. Statist. 38(4), 2282–2313 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Huber, P.J.: Robust estimation of a location parameter. Ann. Math. Statist. 35(1), 73–101 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  • Huber, P.J.: Robust regression: asymptotics, conjectures and monte carlo. Ann. Statist. 1(5), 799–821 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  • Hunter, D.R., Lange, K.: A tutorial on mm algorithms. Am. Statist. 58(1), 30–37 (2004)

    Article  MathSciNet  Google Scholar 

  • Kato, K.: Group lasso for high dimensional sparse quantile regression models. https://arxiv.org/abs/1103.1458 (2011)

  • Koenker, R.: Quantile Regress. Cambridge University Press, UK (2005)

    Book  MATH  Google Scholar 

  • Koenker, R., Bassett, G.: Regress. Quantiles. Econometrica 46(1), 33–50 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  • Koenker, R., Mizera, I.: Convex optimization in R. J. Stat. Softw. 60(5), 1–23 (2014)

    Article  MATH  Google Scholar 

  • Koenker, R.W., D’Orey, V.: Computing regression quantiles. J. R. Stat. Soc.: Ser. C: Appl. Stat. 36(3), 383–393 (1987)

    Google Scholar 

  • Kokic, P., Chambers, R., Breckling, J., et al.: A measure of production performance. J. Bus. Econ. Stat. 15(4), 445–451 (1997)

    Google Scholar 

  • Kuhn, M.: AmesHousing: The Ames Iowa Housing Data. R package version 0.0.4 (2020)

  • Lee, Y., MacEachern, S.N., Jung, Y.: Regularization of case-specific parameters for robustness and efficiency. Statist Sci 27(3), 350–372 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Li, S., Sherwood, B.: hrqglas: group variable selection for quantile and robust mean regression. R. Package Version 1, 1 (2021)

    Google Scholar 

  • Liu, L.Z., Wu, F.X., Zhang, W.J.: A group lasso-based method for robustly inferring gene regulatory networks from multiple time-course datasets. BMC Syst. Biol. 8, 1–12 (2014)

    Article  Google Scholar 

  • Lounici, K., et al.: Oracle inequalities and optimal inference under group sparsity. Ann. Statist. 39(4), 2164–2204 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Meier, L., Van De Geer, S., Bühlmann, P.: The group lasso for logistic regression. J. R. Stat. Soc. Ser. B Stat Methodol. 70(1), 53–71 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Muggeo, V.M., Sciandra, M., Augugliaro, L.: Quantile regression via iterative least squares computations. J. Stat. Comput. Simul. 82(11), 1557–1569 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Negahban, S.N., et al.: A unified framework for high-dimensional analysis fo \(m\)-estimators with decomposable regualrizers. Statist. Sci. 27(4), 538–557 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Newey, W.K., Powell, J.L.: Asymmetric least squares estimation and testing. Econometrica 55(4), 819–847 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  • Portnoy, S., Koenker, R.: The gaussian hare and the laplacian tortoise: computability of squared-error versus absolute-error estimators. Statist. Sci. 12(4), 279–300 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  • Pratesi, M., Ranalli, M.G., Salvati, N.: Nonparametric m-quantile regression using penalised splines. J. Nonparametr. Stat. 21(3), 287–304 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Sherwood, B., Maidman, A., Li, S.: rqPen: Penalized Quantile Regression. R package version 3.0 (2022)

  • Sun, Q., Zhou, W.X., Fan, J.: Adaptive huber regression. J. Am. Statist. Assoc. 115(529), 254–265 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  • Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat Methodol. 58(1), 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  • Tibshirani, R., et al.: Strong rules for discarding predictors in lasso-type problems. J. R. Stat. Soc. Ser. B Stat Methodol. 74(2), 245–266 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Tzavidis, N., Salvati, N., Schmid, T., et al.: Longitudinal analysis of the strengths and difficulties questionnaire scores of the millennium cohort study children in england using m-quantile random-effects regression. J. R. Stat. Soc. Ser. A Stat. Soc. 179(2), 427–452 (2016)

    Article  MathSciNet  Google Scholar 

  • Wang, L., Wu, Y., Li, R.: Quantile regression of analyzing heterogeneity in ultra-high dimension. J. Am. Statist. Assoc. 107(497), 214–222 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Wu, T.T., Lange, K., et al.: The MM alternative to EM. Statist. Sci. 25(4), 492–505 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Wu, Y., Liu, Y.: Variable selection in quantile regression. Statist. Sinica 19(2), 801–817 (2009)

    MathSciNet  MATH  Google Scholar 

  • Xu, J., Ying, Z.: Simultaneous estimation and variable selection in median regression using lasso-type penalty. Ann. Inst. Statist. Math. 62, 487–514 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Yang, Y., Zou, H.: A fast unified algorithm for solving group-lasso penalize learning problems. Stat. Comput. 25(6), 1129–1141 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  • Yi, C.: hqreg: Regularization Paths for Lasso or Elastic-Net Penalized Huber Loss Regression and Quantile Regression. R Package Version 1, 4 (2017)

    Google Scholar 

  • Yi, C., Huang, J.: Semismooth newton coordinate descent algorithm for elastic-net penalized huber loss regression and quantile regression. J. Comput. Graph. Statist. 26(3), 547–557 (2017)

  • Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat Methodol. 68(1), 49–67 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Zhou, W.X., et al.: A new perspective on robust \(m\)-estimation: finite sample theory and applications to dependence-adjusted multiple testing. Ann. Statist. 46(5), 1904–1931 (2018)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Business, University of Kansas, 1654 Naismith Dr., Lawrence, KS, 66045, USA

    Ben Sherwood & Shaobo Li

Authors
  1. Ben Sherwood
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaobo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben Sherwood.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (zip 186 KB)

Supplementary file 2 (pdf 203 KB)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sherwood, B., Li, S. Quantile regression feature selection and estimation with grouped variables using Huber approximation. Stat Comput 32, 75 (2022). https://doi.org/10.1007/s11222-022-10135-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11222-022-10135-w

Keywords

Search

Navigation