Abstract
In this paper, we consider a class of structured nonsmooth optimization problem in which the first component of the objective is a smooth function while the second component is the sum of one-dimensional nonsmooth functions. We first verify that every minimizer of this problem is a solution of an equation \(h(x)=0\), where h is continuous but not differentiable, and moreover \(-h(x)\) is a descent direction of the objective at \(x\in \mathbb {R}^n\) if \(h(x)\ne 0\). Then by using \(-h(x)\) as a search direction, we propose a residual-based algorithm for solving this problem. Under proper conditions, we verify that any accumulation point of the sequence of iterates generated by our algorithm is a first-order stationary point of the problem. Additionally, we prove that the worst-case iteration-complexity for finding an \(\epsilon \) first-order stationary point is \(O(\epsilon ^{-2})\). Numerical results have shown the efficiency of this algorithm.
References
Bian, W., Chen, X.: Worst-case complexity of smoothing quadratic regularization methods for non-Lipschitzian optimization. SIAM J. Optim. 23, 1718–1741 (2013)
Borwein, J.M., Lewis, A.S.: Convex Analysis and Nonlinear Optimization: Theory and Examples. Springer, New York (2000)
Chen, X., Lu, Z., Pong, T.K.: Penalty methods for a class of non-Lipschitz optimization problems. SIAM J. Optim. 26, 1465–1492 (2016)
Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3, 765–790 (2010)
Chen, X., Zhou, W.: Convergence of the reweighted \(L_1\) minimization algorithm for \(L_2\)-\(L_p\) minimization. Comput. Optim. Appl. 59, 47–61 (2014)
Fan, J.: Comments on ‘Wavelets in statistics: a review’ by A. Antoniadis. J. Ital. Stat. Soc. 6, 131–138 (1997)
Gong, P., Zhang, C., Lu, Z., Huang, J., Ye, J.: A general iterative shrinkage and thresholding algorithm for nonconvex regularized optimization problems. In: ICML, pp. 37–45 (2013)
Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986)
Hale, E., Yin, W., Zhang, Y.: Fixed-point continuation for \(\ell _1\)-minimization: methodology and convergence. SIAM J. Optim. 19, 1107–1130 (2008)
Huang, J., Horowitz, J.L., Ma, S.: Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Ann. Stat. 36, 587–613 (2008)
Lu, Z.: Iterative reweighted minimization methods for \(L_p\) regularized unconstrained nonlinear programming. Math. Program. 147, 277–307 (2014)
Lu, Z., Li, X.: Sparse recovery via partial regularization: models, theory and algorithms. Math. Oper. Res. 43, 1290–1316 (2018)
Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I: Basic Theory. Springer, Berlin (2006)
Nikolova, M., Ng, M.K., Zhang, S., Ching, W.-K.: Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 1, 2–25 (2008)
Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)
Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, New York (1998)
Tibshirani, R.: Shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1996)
Wright, S.J., Nowak, R., Figueiredo, M.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57, 2479–2493 (2009)
Wu, L., Sun, Z., Li, D.H.: A Barzilai–Borwein-like iterative half thresholding algorithm for the \(L_{1/2}\) regularized problem. J. Sci. Comput. 67, 581–601 (2016)
Yu, P., Pong, T.K.: Iteratively reweighted \(\ell _1\) algorithms with extrapolation. https://arxiv.org/pdf/1710.07886.pdf
Zhang, C.-H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38, 894–942 (2010)
Acknowledgements
The authors would like to thank the anonymous referee and the editor for their valuable suggestions and comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The work was supported by the National Nature Science Foundation of People’s Republic of China (11501265 and 11761037), the Nature Science Foundation of Jiangxi (No. 20161BAB211011), and the Foundation of Department of Education Jiangxi Province (No. GJJ150314)
Rights and permissions
About this article
Cite this article
Wu, L. A residual-based algorithm for solving a class of structured nonsmooth optimization problems. J Glob Optim 76, 137–153 (2020). https://doi.org/10.1007/s10898-019-00776-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-019-00776-z