Skip to main content
Log in

A Simplified Form of Block-Iterative Operator Splitting and an Asynchronous Algorithm Resembling the Multi-Block Alternating Direction Method of Multipliers

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

This paper develops what is essentially a simplified version of the block-iterative operator splitting method already proposed by the author and P. Combettes, but with more general initialization conditions. It then describes one way of implementing this algorithm asynchronously under a computational model inspired by modern high-performance computing environments, which consist of interconnected nodes each having multiple processor cores sharing a common local memory. The asynchronous implementation framework is then applied to derive an asynchronous algorithm which resembles the alternating direction method of multipliers with an arbitrary number of blocks of variables. Unlike earlier proposals for asynchronous variants of the alternating direction method of multipliers, the algorithm relies neither on probabilistic control nor on restrictive assumptions about the problem instance, instead making only standard convex-analytic regularity assumptions. It also allows the proximal parameters to range freely between arbitrary positive bounds, possibly varying with both iterations and subproblems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Fortin, M., Glowinski, R.: On decomposition-coordination methods using an augmented Lagrangian. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, Studies in Mathematics and its Applications, vol. 15, pp. 97–146. North-Holland Publishing Co., Amsterdam (1983)

    Google Scholar 

  2. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary Value Problems, pp. 299–340. North-Holland, Amsterdam (1983). chap. IX

    Chapter  Google Scholar 

  3. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(3), 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  4. Eckstein, J., Yao, W.: Understanding the convergence of the alternating direction method of multipliers: theoretical and computational perspectives. Pac. J. Optim. 11(4), 619–644 (2015)

    MathSciNet  MATH  Google Scholar 

  5. Iutzeler, F., Bianchi, P., Ciblat, P., Hachem, W.: Asynchronous distributed optimization using a randomized alternating direction method of multipliers. In: Astolfi, A. (ed.) 52nd IEEE Conference on Decision and Control, pp. 3671–3676. IEEE, Piscataway (2013)

    Chapter  Google Scholar 

  6. Wei, E., Ozdaglar, A.: On the O\((1/k)\) convergence of asynchronous distributed alternating direction method of multipliers. In: Tewfik, A. (ed.) 2013 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 551–554. IEEE, Piscataway (2013)

    Chapter  Google Scholar 

  7. Zhang, R., Kwok, J.: Asynchronous distributed ADMM for consensus optimization. In: Jebara, T., Xing, E.P. (eds.) Proceedings of the 31st International Conference on Machine Learning (ICML-14) pp. 1701–1709. (2014)

  8. Mota, J.F.C., Xavier, J.M.F., Aguiar, P.M.Q., Püschel, M.: D-ADMM: a communication efficient distributed algorithm for separable optimization. IEEE Trans. Signal Proc. 61(10), 2718–2723 (2013)

    Article  MathSciNet  Google Scholar 

  9. Chang, T.H., Hong, M., Liao, W.C., Wang, X.: Asynchronous distributed ADMM for large-scale optimization—part I: algorithm and convergence analysis. IEEE Trans. Signal Proc. 64(12), 3118–3130 (2016)

    Article  MathSciNet  Google Scholar 

  10. Combettes, P.L., Eckstein, J.: Asynchronous block-iterative primal-dual decomposition methods for monotone inclusions. Math. Program. (2016). doi:10.1007/s10107-016-1044-0

  11. Chen, C., Sun, R., Ye, Y.: On convergence of the multi-block alternating direction method of multipliers. In: Proceedings of the 8th International Congress on Industrial and Applied Mathematics, pp. 3–15. Higher Ed. Press, Beijing (2015)

  12. Eckstein, J., Svaiter, B.F.: A family of projective splitting methods for the sum of two maximal monotone operators. Math. Program. 111(1–2, Ser. B), 173–199 (2008)

    MathSciNet  MATH  Google Scholar 

  13. Eckstein, J., Svaiter, B.F.: General projective splitting methods for sums of maximal monotone operators. SIAM J. Control Optim. 48(2), 787–811 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Alotaibi, A., Combettes, P.L., Shahzad, N.: Solving coupled composite monotone inclusions by successive Fejér approximations of their Kuhn-Tucker set. SIAM J. Optim. 24(4), 2076–2095 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  15. Briceño-Arias, L.M., Combettes, P.L.: A monotone + skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21(4), 1230–1250 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. Combettes, P.L.: Féjer monotonicity in convex optimization. In: Floudas, C.A., Pardalos, P.M. (eds.) Encyclopedia of Optimization, pp. 106–114. Springer, Berlin (2001)

    Google Scholar 

  17. Bauschke, H.H.: A note on the paper by Eckstein and Svaiter on General projective splitting methods for sums of maximal monotone operators. SIAM J. Control Optim. 48(4), 2513–2515 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  18. Svaiter, B.F.: On weak convergence of the Douglas–Rachford method. SIAM J. Control Optim. 49(1), 280–287 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  19. Rockafellar, R.T.: Convex analysis. Princeton Mathematical Series, No. 28. Princeton University Press, Princeton, NJ (1970)

  20. Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res. 1(2), 97–116 (1976)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported in part by National Science Foundation (NSF) Grants 1115638 and 1617617, Computing and Communications Foundations, CISE directorate. The author would also like to thank Patrick Combettes, as this work grew out the same discussions with him that led to the joint work [10].

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jonathan Eckstein.

Additional information

Communicated by Amir Beck.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eckstein, J. A Simplified Form of Block-Iterative Operator Splitting and an Asynchronous Algorithm Resembling the Multi-Block Alternating Direction Method of Multipliers. J Optim Theory Appl 173, 155–182 (2017). https://doi.org/10.1007/s10957-017-1074-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-017-1074-7

Keywords

Mathematics Subject Classification

Navigation