Skip to main content
SpringerLink
Log in

An acceleration scheme for Dykstra’s algorithm

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Dykstra’s algorithm is an iterative alternating projection procedure for solving the best approximation problem: find the closest point, to a given one, in the intersection of a finite number of closed and convex sets. The main drawback of Dykstra’s algorithm is its frequent slow convergence. In this work we develop an acceleration scheme with a strong geometrical flavor, which guarantees termination at the solution in two cycles of projections in the case of two closed subspaces. The proposed scheme can also be applied to any other alternating projection algorithm that solves the best approximation problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Abdalla, M.O., Grigoriadis, K.M., Zimmerman, D.C.: Enhanced structural damage detection using alternating projection methods. AIAA J. 36, 1305–1311 (1998)

    Article  Google Scholar 

  2. Appleby, G., Smolarski, D.: A linear acceleration row action method for projecting onto subspaces. Electron. Trans. Numer. Anal. 20, 253–275 (2005)

    MATH  MathSciNet  Google Scholar 

  3. Bauschke, H.H., Borwein, J.M.: Dykstra’s alternating projection algorithm for two sets. J. Approx. Theory 79, 418–443 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  5. Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York (2011)

    Book  MATH  Google Scholar 

  6. Bauschke, H.H., Combettes, P.L., Kruk, S.G.: Extrapolation algorithm for affine-convex feasibility problems. Numer. Algorithms 41, 239–274 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  7. Bauschke, H.H., Deutsch, F., Hundal, H., Park, S.H.: Accelerating the convergence of the method of alternating projections. Trans. Am. Math. Soc. 355, 3433–3461 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  8. Birgin, E.G., Raydan, M.: Robust stopping criteria for Dykstra’s algorithm, SIAM. J. Sci. Comput. 26, 1405–1414 (2005)

    MATH  MathSciNet  Google Scholar 

  9. Boyle, J.P., Dykstra, L.: A method for finding projections onto the intersections of convex sets in Hilbert spaces. In: Advances in Order Restricted Statistical Inference. Lecture Notes in Statistics, vol. 37, pp. 28–47. Springer, Berlin (1985)

  10. Borwein, J.M., Tam, M.K.: A cyclic Douglas–Rachford iteration scheme. J. Optim. Theory Appl. 160, 1–29 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  11. Bregman, L.M., Censor, Y., Reich, S.: Dykstra’s algorithm as the nonlinear extension of Bregman’s optimization method. J. Convex Anal. 6, 319–333 (1999)

    MATH  MathSciNet  Google Scholar 

  12. Bregman, L.M., Censor, Y., Reich, S., Zepkowitz-Malachi, Y.: Finding the projection of a point onto the intersection of convex sets via the projection onto halfspaces. J. Approx. Theory 124, 194–218 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  13. Cegielski, A.: Iterative Methods for Fixed Point Problems in Hilbert Spaces. Lecture Notes in Mathematics vol. 2057. Springer, Heidelberg (2012)

  14. Cegielski, A., Censor, Y.: Extrapolation and local acceleration of an iterative process for common fixed point problems. J. Math. Anal. Appl. 394, 809–818 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  15. Cegielski, A., Dylewski, R.: Variable target value relaxed alternating projection method. Comput. Optim. Appl. 47, 455–476 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  16. Cegielski, A., Suchocka, A.: Relaxed alternating projection methods, SIAM. J. Optim. 19, 1093–1106 (2008)

    MATH  MathSciNet  Google Scholar 

  17. Censor, Y.: Computational acceleration of projection algorithms for the linear best approximation problem. Linear Algebra Appl. 416, 111–123 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  18. Censor, Y., Herman, G.T.: On some optimization techniques in image reconstruction from projections. Appl. Numer. Math. 3, 365–391 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  19. Censor, Y., Reich, S.: The Dykstra’s algorithm with Bregman projections. Commun. Appl. Anal. 2, 407–419 (1998)

    MATH  MathSciNet  Google Scholar 

  20. Censor, Y., Zenios, S.A.: Parallel Optimization: Theory, Algorithms, and Applications. Oxford University Press, New York (1997)

    MATH  Google Scholar 

  21. De Pierro, A., Iusem, A.N.: A parallel projection method for finding a common point of a family of convex sets. Pesqui. Oper. 5, 1–20 (1985)

    Google Scholar 

  22. Deutsch, F.: Accelerating the convergence of the method of alternating projections via a line search: a brief survey. In: Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently Parallel Algorithms in Feasibility and Optimization and Their Applications, pp. 203–217. Elsevier, Amsterdam (2001)

    Chapter  Google Scholar 

  23. Deutsch, F.: Best Approximation in Inner Product Spaces. Springer, New York (2001)

    Book  MATH  Google Scholar 

  24. Deutsch, F., Hundal, H.: The rate of convergence of Dykstra’s cyclic projections algorithm: the polyhedral case. Numer. Funct. Anal. Optim. 15, 537–565 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  25. Dos Santos, L.T.: A parallel subgradient projections method for the convex feasibility problem. J. Comput. Appl. Math. 18, 307–320 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  26. Dykstra, R.L.: An algorithm for restricted least-squares regression. J. Am. Stat. Assoc. 78, 837–842 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  27. Eberle, M.G., Maciel, M.C.: Finding the closest Toeplitz matrix. Comput. Appl. Math. 22, 1–18 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  28. Echebest, N., Guardarucci, M.T., Scolnik, H., Vacchino, M.C.: An accelerated iterative method with diagonally scaled oblique projections for solving linear feasibility problems. Ann. Oper. Res. 138, 235–257 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  29. Escalante, R., Raydan, M.: Dykstra’s algorithm for a constrained least-squares matrix problem. Numer. Linear Algebra Appl. 3, 459–471 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  30. Escalante, R., Raydan, M.: On Dykstra’s algorithm for constrained least-squares rectangular matrix problems. Comput. Math. Appl. 35, 73–79 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  31. Escalante, R., Raydan, M.: Alternating Projection Methods. SIAM, Philadelphia (2011)

    Book  Google Scholar 

  32. Galántai, A.: Projectors and Projection Methods. Kluwer Academic Publishers, Dordrecht (2004)

    Book  MATH  Google Scholar 

  33. García-Palomares, U.M.: Preconditioning projection methods for solving algebraic linear systems. Numer. Algorithms 21, 157–164 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  34. Gearhart, W., Koshy, M.: Acceleration schemes for the method of alternating projection. J. Comput. Appl. Math. 26, 235–249 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  35. Glunt, W., Hayden, T.L., Hong, S., Wells, J.: An alternating projection algorithm for computing the nearest Euclidean distance matrix. SIAM J. Matrix Anal. Appl. 11, 589–600 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  36. Grigoriadis, K.M., Skelton, R.: Low order control design for LMI problems using alternating projection methods. Automatica 32, 1117–1125 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  37. Gubin, L.G., Polyak, B.T., Raik, E.V.: The method of projections for finding the common point of convex sets. USSR Comput. Math. Math. Phys. 7(6), 1–24 (1967)

    Article  Google Scholar 

  38. Higham, N.: Computing the nearest correlation matrix - a problem from finance, IMA. J. Numer. Anal. 22, 329–343 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  39. Hundal, H., Deutsch, F.: Two generalizations of Dykstra’s cyclic projections algorithm. Math. Program. 77, 335–355 (1997)

    MATH  MathSciNet  Google Scholar 

  40. Hernández-Ramos, L.M.: Alternating oblique projections for coupled linear systems. Numer. Algorithms 38, 285–303 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  41. Hernández-Ramos, L.M., Escalante, R., Raydan, M.: Unconstrained optimization techniques for the acceleration of alternating projection methods. Numer. Funct. Anal. Optim. 32, 1041–1066 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  42. Iusem, A.N., De Pierro, A.: Convergence results for an accelerated nonlinear Cimmino algorithm. Numer. Math. 49, 367–378 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  43. López, G., Martín-Márquez, V., Wang, F., Xu, H.-K.: Solving the split feasibility problem without prior knowledge of matrix norms. Inverse Probl. 28(8), 1–18 (2012). doi:10.1088/0266-5611/28/8/085004

    Article  Google Scholar 

  44. Luenberger, D.G.: Optimization by Vector Space Methods. Wiley, New York (1969)

    MATH  Google Scholar 

  45. Martínez, J.M.: An accelerated successive orthogonal projection method for solving large-scale linear feasibility problems. Comput. Math. Appl. 15, 367–373 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  46. Mendoza, M., Raydan, M., Tarazaga, P.: Computing the nearest diagonally dominant matrix. Numer. Linear Algebra Appl. 5, 461–474 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  47. Moreno, J., Datta, B., Raydan, M.: A symmetry preserving alternating projection method for matrix model updating. Mech. Syst. Signal Process. 23, 1784–1791 (2009)

    Article  Google Scholar 

  48. Morillas, P.M.: Dykstra’s algorithm with strategies for projecting onto certain polyhedral cones. Appl. Math. Comp. 167, 635–649 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  49. Pang, C.H.J.: Set intersection problems: supporting hyperplanes and quadratic programming. Math. Program., Ser. A, (2014). doi:10.1007/s10107-014-0759-z

  50. Perkins, C.: A convergence analysis of Dykstra’s algorithm for polyhedral sets. SIAM J. Numer. Anal. 40, 792–804 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  51. Pierra, G.: Decomposition through formalization in a product space. Math. Prog. 28, 96–118 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  52. Raydan, M., Tarazaga, P.: Primal and polar approach for computing the symmetric diagonally dominant projection. Numer. Linear Algebra Appl. 9, 333–345 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  53. Strohmer, T., Vershynin, R.: A randomized Kaczmarz algorithm with exponential convergence. J. Fourier Anal. Appl. 15, 262–278 (2009)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

We thank the three anonymous referees for constructive remarks and additional references. Williams López Partially supported by Universidad de Los Andes

Marcos Raydan Partially supported by the Cesma Center at USB

Author information

Authors and Affiliations

  1. Departamento de Física y Matemáticas, Universidad de Los Andes, Trujillo, Venezuela

    Williams López

  2. Departamento de Cómputo Científico y Estadística, Universidad Simón Bolívar, Ap. 89000, Caracas, 1080-A, Venezuela

    Marcos Raydan

Authors
  1. Williams López
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcos Raydan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcos Raydan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

López, W., Raydan, M. An acceleration scheme for Dykstra’s algorithm. Comput Optim Appl 63, 29–44 (2016). https://doi.org/10.1007/s10589-015-9768-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-015-9768-y

Keywords

Search

Navigation