Skip to main content
SpringerLink
Log in

Solving constrained nonsmooth group sparse optimization via group Capped-\(\ell _1\) relaxation and group smoothing proximal gradient algorithm

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, we study the constrained group sparse regularization optimization problem, where the loss function is convex but nonsmooth, and the penalty term is the group sparsity which is then proposed to be relaxed by the group Capped-\(\ell _1\) for the convenience of computation. Firstly, we introduce three kinds of stationary points for the continuous relaxation problem and describe the relationship of the three kinds of stationary points. We give the optimality conditions for the group Capped-\(\ell _1\) problem and the original group sparse regularization problem, and investigate the link between the relaxation problem and the original problem in terms of global and local optimal solutions. The results reveal the equivalence of the original problem and the relaxation problem in some sense. Secondly, we propose a group smoothing proximal gradient (GSPG) algorithm for the constrained group sparse optimization, and prove that the proposed GSPG algorithm globally converges to a lifted stationary point of the relaxation problem. Finally, we present some numerical results on recovery of the simulated group sparse signals and the real group sparse images to illustrate the efficiency of the continuous relaxation model and the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability Statement

The Matlab Codes and the data included in this paper are available on the website of this journal or at http://math.gzu.edu.cn/2017/0414/c9635a59268/page.htm.

References

  1. Ahn, M., Pang, J.-S., Xin, J.: Difference-of-convex learning: directional stationarity, optimality, and sparsity. SIAM J. Opt. 27, 1637–1665 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bech, A., Hallak, N.: Optimization problems involving group sparsity terms. Math. Prog. 178(1–2), 39–67 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bian, W., Chen, X.: A smoothing proximal gradient algorithm for nonsmooth convex regression with cardinality penalty. SIAM J. Numer. Anal. 58(1), 858–883 (2020)

    Article  MathSciNet  Google Scholar 

  4. Bian, W., Chen, X.: Optimality and complexity for constrained optimization problems with nonconvex regularization. Math. Oper. Res. 42, 1063–1084 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  5. Breheny, P., Huang, J.: Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors. Stat. Comput. 25, 173–187 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of \(\ell _{2}\)-\(\ell _{p}\) minimization. SIAM J. Sci. Comput. 32, 2832–2852 (2010)

    Article  MathSciNet  Google Scholar 

  7. Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM, Philadelphia (1990)

    Book  MATH  Google Scholar 

  8. Eren, A.M., Vidyasagar, M.: Error bounds for compressed sensing algorithms with group sparsity: a unified approach. Appl. Comput. Harmon. Anal. 43, 212–232 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  9. Ewout, V.D.B., Friedlander, M.: Probing the pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31, 890–912 (2008)

    MathSciNet  MATH  Google Scholar 

  10. Fan, J., Li, R.: Statistical challenges with high dimensionality: feature selection in knowledge discovery. Proc. Int. Cong. Math. 3, 595–622 (2006)

    MathSciNet  MATH  Google Scholar 

  11. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96, 1348–1360 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  12. Fan, J., Peng, H.: Nonconcave penalized likelihood with a diverging number of parameters. Ann. Stat. 32, 928–961 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gong, P., Zhang, C., Lu, Z.: A general iterative shrinkage and thresholding algorithm for nonconvex regularized optimization problems. In: Proceedings of the 30th International Conference on Machine Learning, 28, 37–45 (2013)

  14. Gramfort, A., Kowalski, M.: Improving M/EEG source localization with an inter-condition sparse prior. In: IEEE International Symposium on Biomedical Imaging(ISBI) (2009)

  15. Hu, Y., Li, C., Meng, K., Qin, J., Yang, X.: Group sparse optimization via \(L_{p, q}\) regularization. J. Mach. Learn. Res. 18, 1–52 (2017)

    Google Scholar 

  16. Huang, J., Breheny, P., Ma, S.: A selective review of group selection in high-dimensional models. Stat. Sci. 27(4), 481–499 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Huang, J., Zhang, T.: The benefit of group sparsity. Ann. Stat. 38, 1978–2004 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  18. Jiao, Y., Jin, B., Lv, X.: Group sparse recovery via the \(\ell _{0}(l_{2})\) penalty: theory and algorithm. IEEE Trans. Signal Process. 65(4), 998–1012 (2017)

    Article  MathSciNet  Google Scholar 

  19. Le, T.H.A., Pham, D.T., Le, H.-M., Vo, X.-T.: DC approximation approaches for sparse optimization. Eur. J. Oper. Res. 244, 26–46 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  20. Lee, S., Oh, M., Kim, Y.: Sparse optimization for nonconvex group penalized estimation. J. Stat. Comput. Simul. 86, 597–610 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  21. Li W., Bian W., Toh K.-C.: DC algorithms for a class of sparse group \(\ell _0\) regularized optimization problems. arXiv:2109.05251 (2021)

  22. Liu, H., Yao, T., Li, R., Ye, Y.: Folded concave penalized sparse linear regression: sparsity, statistical performance, and algorithmic theory for local solutions. Math. Prog. 166(1–2), 207–240 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  23. Meier, L., van, de. Geer. S., Buhlmann, P.: The group Lasso for logistic regression. J. R. Stat. Soc. Ser. B 70, 53–71 (2008)

  24. Natarajan, B.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 227–234 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  25. Pan, L., Chen, X.: Group sparse optimization for images recovery using capped folded concave functions. SIAM J. Imag. Sci. 14(1), 1–25 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  26. Pang, J.-S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  27. Peng, D., Chen, X.: Computation of second-order directional stationary points for group sparse optimization. Opt. Methods Softw. 35(2), 348–376 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  28. Rockafellar, R.: Convex analysis. Princeton University Press, Princeton (1970)

  29. Rockafellar, R., Wets, R.-J.: Variational Analysis. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  30. Soubies, E., Blanc-Férand, L., Aubert, G.: A continuous exact \(L_{0}\) penalty (CE\(L_{0}\)) for least squares regularized problem. SIAM J. Imag. Sci. 8, 1607–1639 (2015)

    Article  MATH  Google Scholar 

  31. Soubies, E., Blanc-Férand, L., Aubert, G.: A unified view of exact continuous penalties for \(\ell _{2}\)-\(\ell _{0}\) minimization. SIAM J. Opt. 27, 2034–2060 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Wei, F., Zhu, H.: Group coordinate descent algorithms for nonconvex penalized regression. Comput. Stat. Data Anal. 56, 316–326 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  33. Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B 68(1), 49–67 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  34. Zhang, C.-H., Huan, J.: The sparsity and bias of the LASSO selection in high dimensional linear regression. Ann. Stat. 36, 1567–1594 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  35. Zhang, C.-H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38, 894–942 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  36. Zhang, T.: Analysis of multi-stage convex relaxation for sparse regularization. J. Mach. Learn. Res. 11, 1081–1107 (2010)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are deeply grateful to the anonymous reviewers for their valuable suggestions and comments that help us to revise the paper into present form. They also thank one of the reviewers for bringing the reference [21] to their attention, in which the authors considered a class of very interesting sparse plus group sparse regularization optimization problems.

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Guizhou University, Guiyang, 550025, China

    Xian Zhang & Dingtao Peng

Authors
  1. Xian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dingtao Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dingtao Peng.

Ethics declarations

Conflicts of Interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by the National Natural Science Foundation of China (11861020, 12261020), the Growth Project of Education Department of Guizhou Province for Young Talents in Science and Technology ([2018]121), the Foundation for Selected Excellent Project of Guizhou Province for High-level Talents Back from Overseas ([2018]03), the Guizhou Provincial Science and Technology Program (ZK[2021]009), and the Research Foundation for Postgraduates of Guizhou Province (YJSCXJH[2020]085).

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Peng, D. Solving constrained nonsmooth group sparse optimization via group Capped-\(\ell _1\) relaxation and group smoothing proximal gradient algorithm. Comput Optim Appl 83, 801–844 (2022). https://doi.org/10.1007/s10589-022-00419-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-022-00419-2

Keywords

Mathematics Subject Classification

Search

Navigation