Skip to main content
Log in

A residual-based algorithm for solving a class of structured nonsmooth optimization problems

Journal of Global Optimization Aims and scope Submit manuscript

Abstract

In this paper, we consider a class of structured nonsmooth optimization problem in which the first component of the objective is a smooth function while the second component is the sum of one-dimensional nonsmooth functions. We first verify that every minimizer of this problem is a solution of an equation \(h(x)=0\), where h is continuous but not differentiable, and moreover \(-h(x)\) is a descent direction of the objective at \(x\in \mathbb {R}^n\) if \(h(x)\ne 0\). Then by using \(-h(x)\) as a search direction, we propose a residual-based algorithm for solving this problem. Under proper conditions, we verify that any accumulation point of the sequence of iterates generated by our algorithm is a first-order stationary point of the problem. Additionally, we prove that the worst-case iteration-complexity for finding an \(\epsilon \) first-order stationary point is \(O(\epsilon ^{-2})\). Numerical results have shown the efficiency of this algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Bian, W., Chen, X.: Worst-case complexity of smoothing quadratic regularization methods for non-Lipschitzian optimization. SIAM J. Optim. 23, 1718–1741 (2013)

    Article  MathSciNet  Google Scholar 

  2. Borwein, J.M., Lewis, A.S.: Convex Analysis and Nonlinear Optimization: Theory and Examples. Springer, New York (2000)

    Book  Google Scholar 

  3. Chen, X., Lu, Z., Pong, T.K.: Penalty methods for a class of non-Lipschitz optimization problems. SIAM J. Optim. 26, 1465–1492 (2016)

    Article  MathSciNet  Google Scholar 

  4. Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3, 765–790 (2010)

    Article  MathSciNet  Google Scholar 

  5. Chen, X., Zhou, W.: Convergence of the reweighted \(L_1\) minimization algorithm for \(L_2\)-\(L_p\) minimization. Comput. Optim. Appl. 59, 47–61 (2014)

    Article  MathSciNet  Google Scholar 

  6. Fan, J.: Comments on ‘Wavelets in statistics: a review’ by A. Antoniadis. J. Ital. Stat. Soc. 6, 131–138 (1997)

    Article  Google Scholar 

  7. Gong, P., Zhang, C., Lu, Z., Huang, J., Ye, J.: A general iterative shrinkage and thresholding algorithm for nonconvex regularized optimization problems. In: ICML, pp. 37–45 (2013)

  8. Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986)

    Article  MathSciNet  Google Scholar 

  9. Hale, E., Yin, W., Zhang, Y.: Fixed-point continuation for \(\ell _1\)-minimization: methodology and convergence. SIAM J. Optim. 19, 1107–1130 (2008)

    Article  MathSciNet  Google Scholar 

  10. Huang, J., Horowitz, J.L., Ma, S.: Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Ann. Stat. 36, 587–613 (2008)

    Article  MathSciNet  Google Scholar 

  11. Lu, Z.: Iterative reweighted minimization methods for \(L_p\) regularized unconstrained nonlinear programming. Math. Program. 147, 277–307 (2014)

    Article  MathSciNet  Google Scholar 

  12. Lu, Z., Li, X.: Sparse recovery via partial regularization: models, theory and algorithms. Math. Oper. Res. 43, 1290–1316 (2018)

    Article  MathSciNet  Google Scholar 

  13. Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I: Basic Theory. Springer, Berlin (2006)

    Book  Google Scholar 

  14. Nikolova, M., Ng, M.K., Zhang, S., Ching, W.-K.: Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 1, 2–25 (2008)

    Article  MathSciNet  Google Scholar 

  15. Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)

    Article  MathSciNet  Google Scholar 

  16. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, New York (1998)

    Book  Google Scholar 

  17. Tibshirani, R.: Shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  18. Wright, S.J., Nowak, R., Figueiredo, M.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57, 2479–2493 (2009)

    Article  MathSciNet  Google Scholar 

  19. Wu, L., Sun, Z., Li, D.H.: A Barzilai–Borwein-like iterative half thresholding algorithm for the \(L_{1/2}\) regularized problem. J. Sci. Comput. 67, 581–601 (2016)

    Article  MathSciNet  Google Scholar 

  20. Yu, P., Pong, T.K.: Iteratively reweighted \(\ell _1\) algorithms with extrapolation. https://arxiv.org/pdf/1710.07886.pdf

  21. Zhang, C.-H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38, 894–942 (2010)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous referee and the editor for their valuable suggestions and comments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lei Wu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The work was supported by the National Nature Science Foundation of People’s Republic of China (11501265 and 11761037), the Nature Science Foundation of Jiangxi (No. 20161BAB211011), and the Foundation of Department of Education Jiangxi Province (No. GJJ150314)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, L. A residual-based algorithm for solving a class of structured nonsmooth optimization problems. J Glob Optim 76, 137–153 (2020). https://doi.org/10.1007/s10898-019-00776-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-019-00776-z

Keywords

Mathematics Subject Classification

Navigation