Abstract
This paper studies a general form problem in which a lower bounded continuously differentiable function is minimized over a block separable set incorporating a group sparsity expression as a constraint or a penalty (or both) in the group sparsity setting. This class of problems is generally hard to solve, yet highly applicable in numerous practical settings. Particularly, we study the proximal mapping that includes group-sparsity terms, and derive an efficient method to compute it. Necessary optimality conditions for the problem are devised, and a hierarchy between stationary-based and coordinate-wised based conditions is established. Methods that obtain points satisfying the optimality conditions are presented, analyzed and tested in applications from the fields of investment and graph theory.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
If \(I_1(\mathbf{y}) \cap \{j : \omega (\mathbf{x})_j < 2\lambda \}=\emptyset \), then the sum equals 0, and otherwise it is negative.
We use a convention that if \(T = \emptyset \), then \(\mathbf{U}_{{{{\mathcal {A}}}} (T)} \mathbf{y}=\mathbf{0}\).
This result assumes that \(\mathbf{x}^{j,+}\) exists, which happens only if \(\Vert g(\mathbf{x}) \Vert _0 < s\), see Remark 4.11.
The data was acquired using Matlab’s built-in functions, see www.mathworks.com/help/datafeed/retrieve-bloomberg-historical-data.html for details.
References
Baldassarre, L., Bhan, N., Cevher, V., Kyrillidis, A., Satpathi, S.: Group-sparse model selection: hardness and relaxations. IEEE Trans. Inf. Theory 62(11), 6508–6534 (2016)
Baraniuk, R.G., Cevher, V., Duarte, M.F., Hegde, C.: Model-based compressive sensing. IEEE Trans. Inf. Theory 56(4), 1982–2001 (2010)
Beck, A.: Introduction to Nonlinear Optimization: Theory, Algorithms, and Applications with MATLAB. MOS-SIAM Series on Optimization. Society for Industrial and Applied Mathematics, Philadelphia (2014)
Beck, A., Eldar, Y.C.: Sparsity constrained nonlinear optimization: optimality conditions and algorithms. SIAM J. Optim. 23(3), 1480–1509 (2013)
Beck, A., Hallak, N.: Proximal mapping for symmetric penalty and sparsity. SIAM J. Optim. 28, 496–527 (2018)
Beck, A., Hallak, N.: On the minimization over sparse symmetric sets: projections, optimality conditions, and algorithms. Math. Oper. Res. 41(1), 196–223 (2016)
Beck, A., Vaisbourd, Y.: The sparse principal component analysis problem: optimality conditions and algorithms. J. Optim. Theory Appl. 170, 119–143 (2016)
Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)
Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009)
Blumensath, T., Davies, M.E., Mike, E.: Sampling theorems for signals from the union of finite-dimensional linear subspaces. IEEE Trans. Inf. Theory 55(4), 1872–1882 (2009)
Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1), 459–494 (2014)
Bomze, I.M., Budinich, M., Pardalos, P.M., Pelillo, M.: The maximum clique problem. In: Du, D.Z., Pardalos, P. (eds.) Handbook of Combinatorial Optimization, pp. 1–74. Springer, Boston (1999)
Davenport, M.A., Duarte, M.F., Eldar, Y.C., Kutyniok, G.: Introduction to compressed sensing. Preprint, pp. 1–68 (2011)
Duarte, M.F., Cevher, V., Baraniuk, R.G.: Model-based compressive sensing for signal ensembles. In: Communication, Control, and Computing, 2009. Allerton 2009. 47th Annual Allerton Conference on, pp. 244–250. IEEE (2009)
Duarte, M.F., Eldar, Y.C.: Structured compressed sensing: from theory to applications. IEEE Trans. Signal Process. 59(9), 4053–4085 (2011)
Elad, M.: Sparse and Redundant Representations: from Theory to Applications in Signal and Image Processing. Springer, Berlin (2010)
Eldar, Y.C., Kuppinger, P., Bolcskei, H.: Block-sparse signals: uncertainty relations and efficient recovery. IEEE Trans. Signal Process. 58(6), 3042–3054 (2010)
Eldar, Y.C., Mishali, M.: Robust recovery of signals from a structured union of subspaces. IEEE Trans. Inf. Theory 55(11), 5302–5316 (2009)
Jain, P., Rao, N., Dhillon, I.S.: Structured sparse regression via greedy hard thresholding. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29, pp. 1516–1524. Curran Associates Inc., New York (2016)
Jenatton, R., Audibert, J., Bach, F.: Structured variable selection with sparsity-inducing norms. J. Mach. Learn. Res. 12(Oct), 2777–2824 (2011)
Kochenberger, G., Hao, J., Glover, F., Lewis, M., Lü, Z., Wang, H., Wang, Y.: The unconstrained binary quadratic programming problem: a survey. J. Comb. Optim. 28(1), 58–81 (2014)
Kroese, D.P., Rubinstein, R.Y.: Simulation and the Monte Carlo Method. Wiley, New York (2008)
Meier, L., Van De Geer, S., Bühlmann, P.: The group lasso for logistic regression. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 70(1), 53–71 (2008)
Moreau, J.-J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)
Nardi, Y., Rinaldo, A.: On the asymptotic properties of the group lasso estimator for linear models. Electron. J. Stat. 2, 605–633 (2008)
Prigent, J.L.: Portfolio Optimization and Performance Analysis. CRC Press, Boca Raton (2007)
Sra, S., Nowozin, S., Wright, S.J.: Optimization for Machine Learning. Mit Press, Cambridge (2012)
Stojnic, M., Parvaresh, F., Hassibi, B.: On the reconstruction of block-sparse signals with an optimal number of measurements. IEEE Trans. Signal Process. 57(8), 3075–3085 (2009)
Tropp, J.A., Wright, S.J.: Computational methods for sparse solution of linear inverse problems. Proc. IEEE 98(6), 948–958 (2010)
Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 68(1), 49–67 (2006)
Author information
Authors and Affiliations
Corresponding author
Additional information
The research of the first author was partially supported by the Israel Science Foundation Grant 1821/16.
Rights and permissions
About this article
Cite this article
Beck, A., Hallak, N. Optimization problems involving group sparsity terms. Math. Program. 178, 39–67 (2019). https://doi.org/10.1007/s10107-018-1277-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-018-1277-1