Skip to main content
SpringerLink
Log in

Dykstra’s Splitting and an Approximate Proximal Point Algorithm for Minimizing the Sum of Convex Functions

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

We show that Dykstra’s splitting for projecting onto the intersection of convex sets can be extended to minimize the sum of convex functions and a regularizing quadratic function. We give conditions for which convergence to the primal minimizer holds so that more than one convex function can be minimized at a time, the convex functions are not necessarily sampled in a cyclic manner, and the SHQP strategy for problems involving the intersection of more than one convex set can be applied. When the sum does not involve the regularizing quadratic function, we discuss an approximate proximal point method combined with Dykstra’s splitting to minimize this sum.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Dykstra, R.: An algorithm for restricted least-squares regression. J. Am. Stat. Assoc. 78, 837–842 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  2. Boyle, J., Dykstra, R.: A method for finding projections onto the intersection of convex sets in Hilbert spaces. In: Dykstra, R., Robertson, T., Wright, F.T. (eds.) Advances in Order Restricted Statistical Inference. Lecture notes in Statistics, pp. 28–47. Springer, New York (1985)

  3. Han, S.: A successive projection method. Math. Program. 40, 1–14 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  4. Gaffke, N., Mathar, R.: A cyclic projection algorithm via duality. Metrika 36, 29–54 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  5. Iusem, A., Pierro, A.D.: On the convergence of Han’s method of convex programming with quadratic objective. Math. Program. 52, 265–284 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  6. Pierra, G.: Decomposition through formalization in a product space. Math. Program. 28, 96–115 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  7. Hundal, H., Deutsch, F.: Two generalizations of Dykstra’s cyclic projections algorithm. Math. Program. 77, 335–355 (1997)

    MathSciNet  MATH  Google Scholar 

  8. Pang, C.: The supporting halfspace—quadratic programming strategy for the dual of the best approximation problem. SIAM J. Optim. 26(4), 2591–2619 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  9. Deutsch, F.: Accelerating the convergence of the method of alternating projections via a line search: a brief survey. In: Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, pp. 203–217. Elsevier, Amsterdam (2001)

    Chapter  Google Scholar 

  10. Deutsch, F.: Best Approximation in Inner Product Spaces. CMS Books in Mathematics. Springer, Berlin (2001)

    Book  Google Scholar 

  11. Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Berlin (2011)

    Book  MATH  Google Scholar 

  12. Escalante, R., Raydan, M.: Alternating Projection Methods. SIAM, Philadelphia, PA (2011)

  13. Beck, A., Tetruashvili, L.: On the convergence of block coordinate descent type methods. SIAM J. Optim. 23(4), 2037–2060 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  14. Beck, A.: On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes. SIAM J. Optim. 25(1), 185–209 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  15. Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Program. Ser. B 117(117), 387–423 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  16. Tseng, P., Yun, S.: Block-coordinate gradient descent method for linearly constrained nonsmooth separable optimization. J. Optim. Theory Appl. 140, 513–535 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  17. Wright, S.: Coordinate descent algorithms. Math. Program. 151, 3–34 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hong, M., Wang, X., Razaviyayn, M., Luo, Z.: Iteration complexity analysis of block coordinate descent methods. Math. Program. 163, 85–114 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  19. Martinet, B.: Régularisation d’inéquations variationnelles par approximations successives. Rev. Française Informat. Rech. Opér. 4, 154–158 (1970)

    MATH  Google Scholar 

  20. Rockafellar, R.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  21. Han, S.: A decomposition method and its application to convex programming. Math. Oper. Res. 14, 237–248 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  22. Bauschke, H., Combettes, P.: A Dykstra-like algorithm for two monotone operators. Pac. J. Optim. 4, 383–391 (2008)

    MathSciNet  MATH  Google Scholar 

  23. Tseng, P.: Dual coordinate ascent methods for non-strictly convex minimization. Math. Program. 59, 231–247 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  24. Combettes, P., Dũng, D., Vũ, B.: Proximity for sums of composite functions. J. Math. Anal. Appl. 380(2), 680–688 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  25. Combettes, P., Dũng, D., Vũ, B.: Dualization of signal recovery problems. Set-Valued Var. Anal. 18, 373–404 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  26. Abboud, F., Chouzenoux, E., Pesquet, J.C., Chenot, J.H., Laborelli, L.: Dual block-coordinate forward–backward algorithm with application to deconvolution and deinterlacing of video sequences. J. Math. Imaging Vis. 59(3), 415–431 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  27. Nedić, A.: Random algorithms for convex minimization problems. Math. Program. Ser. B 225, 225–253 (2011)

    MathSciNet  MATH  Google Scholar 

  28. Nesterov, Y.: Introductory Lectures on Convex Optimization. Kluwer, London (2004)

    Book  MATH  Google Scholar 

  29. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  30. Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization (2008)

  31. Bauschke, H., Borwein, J.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  32. Censor, Y., Chen, W., Combettes, P.L., Davidi, R., Herman, G.: On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints. Comput. Optim. Appl. 51, 1065–1088 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  33. Neto, E.H., Pierro, A.D.: Incremental subgradients for constrained convex optimization: a unified framework and new methods. SIAM J. Optim. 20, 1547–1572 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  34. Ram, S., Nedić, A., Veeravalli, V.: Incremental stochastic subgradient algorithms for convex optimization. SIAM J. Optim. 20, 691–717 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  35. Censor, Y., Davidi, R., Herman, G.: Perturbation resilience and superiorization of iterative algorithms. Inverse Probl. 26(6), 065008 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  36. Censor, Y., Davidi, R., Herman, G., Schulte, R., Tetruashvili, L.: Projected subgradient minimization versus superiorization. J. Optim. Theory Appl. 160, 730–747 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  37. Bauschke, H., Borwein, J., Li, W.: Strong conical hull intersection property, bounded linear regularity, Jameson’s property (G), and error bounds in convex optimization. Math. Program., Ser. A 86(1), 135–160 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  38. Burke, J., Deng, S.: Weak sharp minima revisited. II. Application to linear regularity and error bounds. Math. Program., Ser. B 104(2–3), 235–261 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  39. Ng, K., Yang, W.: Regularities and their relations to error bounds. Math. Program., Ser. A 99, 521–538 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  40. Kruger, A.: About regularity of collections of sets. Set-Valued Anal. 14, 187–206 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  41. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2010)

    Article  MATH  Google Scholar 

  42. Combettes, P., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H., Burachik, R., Combettes, P., Elser, V., Luke, D., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pp. 185–212. Springer, New York, NY (2011)

  43. Pang, C.H.J.: Distributed deterministic asynchronous algorithms in time-varying graphs through Dykstra splitting. SIAM J. Optim. 29(1), 484–510 (2018)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We acknowledge Grant R-146-000-214-112 from the Faculty of Science, National University of Singapore. We gratefully acknowledge discussions with Ting-Kei Pong on Dykstra’s splitting which led to this paper. We also thank the two anonymous referees and the editorial staff.

Author information

Authors and Affiliations

  1. Department of Mathematics, National University of Singapore, Block S17 08-11, 10 Lower Kent Ridge Road, Singapore, 119076, Singapore

    Chin How Jeffrey Pang

Authors
  1. Chin How Jeffrey Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chin How Jeffrey Pang.

Additional information

Communicated by Panos M. Pardalos.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pang, C.H.J. Dykstra’s Splitting and an Approximate Proximal Point Algorithm for Minimizing the Sum of Convex Functions. J Optim Theory Appl 182, 1019–1049 (2019). https://doi.org/10.1007/s10957-019-01512-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-019-01512-z

Keywords

Mathematics Subject Classification

Search

Navigation