Skip to main content
SpringerLink
Log in

Decomposition Methods for Differentiable Optimization Problems over Cartesian Product Sets

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

This paper presents a unified analysis of decomposition algorithms for continuously differentiable optimization problems defined on Cartesian products of convex feasible sets. The decomposition algorithms are analyzed using the framework of cost approx imation algorithms. A convergence analysis is made for three decomposition algorithms: a sequential algorithm which extends the classical Gauss-Seidel scheme, a synchronized parallel algorithm which extends the Jacobi method, and a partially asynchronous parallel algorithm. The analysis validates inexact computations in both the subproblem and line search phases, and includes convergence rate results. The range of feasible step lengths within each algorithm is shown to have a direct correspondence to the increasing degree of parallelism and asynchronism, and the resulting usage of more outdated information in the algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. L. Armijo, “Minimization of functions having Lipschitz continuous first partial derivatives,” Pacific J. Math., vol. 16, pp. 1-3, 1966.

    Google Scholar 

  2. K.J. Arrow and L. Hurwicz (Eds.), Studies in Resource Allocation Processes, Cambridge University Press: Cambridge, 1977.

    Google Scholar 

  3. G.M. Baudet, “Asynchronous iterative methods for multiprocessors,” J. ACM, vol. 25, pp. 226-244, 1978.

    Google Scholar 

  4. M.S. Bazaraa, H.D. Sherali, and C.M. Shetty, Nonlinear Programming: Theory and Algorithms, second edition, John Wiley & Sons: New York, NY, 1993.

    Google Scholar 

  5. D.P. Bertsekas, “Distributed dynamic programming,” IEEE Trans. Automat. Control, vol. AC-27, pp. 610- 616, 1982.

    Google Scholar 

  6. D.P. Bertsekas, “Distributed asynchronous computation of fixed points,” Math. Programming, vol. 27, pp. 107-120, 1983.

    Google Scholar 

  7. D.P. Bertsekas, “Auction algorithms for network flow problems: A tutorial introduction,” Comput. Optim. Appl., vol. 1, pp. 7-66, 1992.

    Google Scholar 

  8. D.P. Bertsekas and D. El Baz, “Distributed asynchronous relaxation methods for convex network flow problems,” SIAM J. Control Optim., vol. 25, pp. 74-85, 1987.

    Google Scholar 

  9. D.P. Bertsekas and J. Eckstein, “Dual coordinate step methods for linear network flow problems,” Math. Programming, vol. 42, pp. 203-243, 1988.

    Google Scholar 

  10. D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall: London, 1989.

    Google Scholar 

  11. D.P. Bertsekas and D.A. Castañon, “Parallel synchronous and asynchronous implementations of the auction algorithm,” Parallel Comput., vol. 17, pp. 707-732, 1991.

    Google Scholar 

  12. Y. Censor, “Row-action methods for huge and sparse systems and their applications,” SIAM Rev., vol. 23, pp. 444-466, 1981.

    Google Scholar 

  13. E.D. Chajakis and S.A. Zenios, “Synchronous and asynchronous implementations of relaxation algorithms for nonlinear network optimization,” Parallel Comput., vol. 17, pp. 873-894, 1991.

    Google Scholar 

  14. D. Chazan and W. Miranker, “Chaotic relaxation,” Linear Algebra Appl., vol. 2, pp. 199-222, 1969.

    Google Scholar 

  15. R.J. Chen and R.R. Meyer, “Parallel optimization for traffic assignment,” Math. Programming, vol. 42, pp. 327-345, 1988.

    Google Scholar 

  16. G. Cohen, “Optimization by decomposition and coordination: A unified approach,” IEEE Trans. Automat. Control, vol. AC-23, pp. 222-232, 1978.

    Google Scholar 

  17. G. Cohen, “Auxiliary problem principle and decomposition of optimization problems,” J. Optim. Theory Appl., vol. 32, pp. 277-305, 1980.

    Google Scholar 

  18. G. Cybenko, “Dynamic load balancing for distributed memory multiprocessors,” Technical Report 87-1, Department of Computer Science, Tufts University, Medford, MA, 1987.

    Google Scholar 

  19. G.B. Dantzig and P. Wolfe, “Decomposition principle for linear programs,” Oper. Res., vol. 8, pp. 101-111, 1960.

    Google Scholar 

  20. R.S. Dembo, J.M. Mulvey, and S.A. Zenios, “Large-scale nonlinear network models and their application,” Oper. Res., vol. 37, pp. 353-372, 1989.

    Google Scholar 

  21. Y.M.I. Dirickx and L.P. Jennergren, “Systems analysis by multilevel methods: With applications to economics and management,” Vol. 6 of International Series on Applied Systems Analysis, John Wiley & Sons: Chichester, 1979.

    Google Scholar 

  22. R.W. Eash, B.N. Janson, and D.E. Boyce, “Equilibrium trip assignment: Advantages and implications for practice,” Transp. Res. Rec., vol. 728, pp. 1-8, 1979.

    Google Scholar 

  23. D. El Baz, “A computational experience with distributed asynchronous iterative methods for convex network flow problems,” in Proc. The 28th IEEE Conference on Decision and Control, Tampa, FL, 1989, pp. 590-591.

  24. B. Feijoo and R.R. Meyer, “Piecewise-linear approximation methods for nonseparable convex optimization,” Management Sci., vol. 34, pp. 411-419, 1988.

    Google Scholar 

  25. J.-Ch. Fiorot and P. Huard, “Composition and union of general algorithms of optimization,” Math. Programming Study, vol. 10, pp. 69-85, 1979.

    Google Scholar 

  26. H. Frank and W. Chou, “Routing in computer networks,” Networks, vol. 1, pp. 99-122, 1971.

    Google Scholar 

  27. A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V. Sunderam, PVM: Parallel Virtual Machine. A User’s Guide and Tutorial for Networked Parallel Computing, The MIT Press: Cambridge, MA, 1994. Also available in html form on the Internet, URL http://www.netlib.org/pvm3/book/pvm-book.html.

    Google Scholar 

  28. G. Golub and J.M. Ortega, Scientific Computing: An Introduction with Parallel Computing, Academic Press: San Diego, CA, 1993.

    Google Scholar 

  29. A. Greenbaum, “Synchronization costs on multiprocessors,” Parallel Comput., vol. 10, pp. 3-14, 1989.

    Google Scholar 

  30. J.L. Gustafson, G.R. Montry, and R.E. Banner, “Development of parallel methods for a 1024-processor hypercube,” SIAM J. Sci. Statist. Comput., vol. 9, pp. 609-638, 1988.

    Google Scholar 

  31. P.T. Harker and J.-S. Pang, “Finite-dimensional variational inequality and nonlinear complementarity problems: A survey of theory, algorithms and applications,” Mathematical Programming, vol. 48, pp. 161-220, 1990.

    Google Scholar 

  32. J.-B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms, Springer-Verlag: Berlin, 1993.

    Google Scholar 

  33. R.W. Hockney and C.R. Jesshope, Parallel Computers 2: Architecture, Programming and Algorithms, Adam Hilger: Bristol, 1988.

    Google Scholar 

  34. K. Hwang and F.A. Briggs, Computer Architecture and Parallel Processing, McGraw-Hill: Singapore, 1985.

    Google Scholar 

  35. H.T. Kung, “Synchronized and asynchronous parallel algorithms for multiprocessors,” in Algorithms and Complexity: New Directions and Recent Results, J.F. Traub (Ed.), Academic Press: New York, NY, pp. 153-200, 1976.

    Google Scholar 

  36. T. Larsson, A. Migdalas, and M. Patriksson, “A partial linearization method for the traffic assignment problem,” Optimization, vol. 28, pp. 47-61, 1993.

    Google Scholar 

  37. T. Larsson and M. Patriksson, “A class of gap functions for variational inequalities,” Math. Programming, vol. 64, pp. 53-79, 1994.

    Google Scholar 

  38. T. Larsson and M. Patriksson, “Equilibrium characterizations of solutions to side constrained asymmetric traffic assignment models,” Le Matematiche, vol. 49, pp. 249-280, 1994.

    Google Scholar 

  39. T. Larsson and M. Patriksson, “Price-directive traffic management: Applications of side constrained traffic equilibrium models,” report, Department of Mathematics, Linköping University, Linköping, Sweden, 1996.

    Google Scholar 

  40. L.S. Lasdon, Optimization Theory for Large Systems, McMillan: New York, NY, 1970.

    Google Scholar 

  41. F.A. Lootsma and K.M. Ragsdell, “State-of-the-art in parallel nonlinear optimization,” Parallel Comput., vol. 6, pp. 133-155, 1988.

    Google Scholar 

  42. D.G. Luenberger, Linear and Nonlinear Programming, second edition, Addison-Wesley, Reading, MA, 1984.

    Google Scholar 

  43. Z.-Q. Luo and P. Tseng, “Error bounds and convergence analysis of feasible descent methods: A general approach,” Ann. Oper. Res., vol. 46, pp. 157-178, 1993.

    Google Scholar 

  44. G.G.L. Meyer, “Asymptotic properties of sequences iteratively generated by point-to-set maps,” Math. Programming Study, vol. 10, pp. 115-127, 1979.

    Google Scholar 

  45. J.C. Miellou, Itérations chaotiques à retards: études de la convergence dans le cas d’espaces partiellment ordonnés, Comptes Rendus Hebdomadaires des Séances de l’Académie des Sciences (Paris), Série A, vol. 280, pp. 233-236, 1975.

    Google Scholar 

  46. J.-C. Miellou, P. Cortey-Dumont, and M. Boulbrachêne, “Perturbation of fixed-point iterative methods,” in Advances in Parallel Computing, D.J. Evans (Ed.), JAI Press: Greenwich, CT, vol. 1, pp. 81-122, 1990.

    Google Scholar 

  47. A. Migdalas, Cyclic Linearization vs. Frank-Wolfe Decomposition for Nonlinear Problems over Cartesian Product Sets, unpublished note, Department of Mathematics, Linköping University, Linköping, Sweden, 1990.

    Google Scholar 

  48. A. Nagurney, Network Economics: A Variational Inequality Approach, Kluwer Academic Publishers: Dordrecht, The Netherlands, 1993.

    Google Scholar 

  49. S.S. Nielsen and S.A. Zenios, “Massively parallel algorithms for singly constrained convex programs,” ORSA J. Comput., vol. 4, pp. 166-181, 1992.

    Google Scholar 

  50. J.M. Ortega and W.C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press: New York, NY, 1970.

    Google Scholar 

  51. P.M. Pardalos, A.T. Phillips, and J.B. Rosen, Topics in Parallel Computing in Mathematical Programming, Vol. 2 of Applied Discrete Mathematics and Theoretical Computer Science Series, Science Press: New York, NY, 1992.

    Google Scholar 

  52. M. Patriksson, “Partial linearization methods in nonlinear programming,” J. Optim. Theory Appl., vol. 78, pp. 227-246, 1993.

    Google Scholar 

  53. M. Patriksson, “A unified description of iterative algorithms for traffic equilibria,” European J. Oper. Res., vol. 71, pp. 154-176, 1993.

    Google Scholar 

  54. M. Patriksson, “A unified framework of descent algorithms for nonlinear programs and variational inequalities,” Doctoral dissertation, Department of Mathematics, Linköping University, Linköping, Sweden, 1993.

    Google Scholar 

  55. M. Patriksson, “Cost approximation: A unified framework of descent algorithms for nonlinear programs,” report, Department of Mathematics, University of Washington, Seattle, WA, 1994. Revised for possible publication in SIAM J. Optim.

    Google Scholar 

  56. M. Patriksson, The Traffic Assignment Problem: Models and Methods, VSP: Utrecht, 1994.

    Google Scholar 

  57. M. Patriksson, “On the convergence of descent methods for monotone variational inequalities,” Oper. Res. Lett., vol. 16, pp. 265-269, 1994.

    Google Scholar 

  58. M. Patriksson, A Taxonomy of Classes of Descent Algorithms for Nonlinear Programs and Variational Inequalities, report, Department of Mathematics, University of Washington, Seattle, WA, 1994.

    Google Scholar 

  59. E.R. Petersen, “A primal-dual traffic assignment algorithm,” Management Sci., vol. 22, pp. 87-95, 1975.

    Google Scholar 

  60. M.Ç. Pinar and S.A. Zenios, “Parallel decomposition of multicommodity network flows using a linear-quadratic penalty algorithm,” ORSA J. Comput., vol. 4, pp. 235-249, 1992.

    Google Scholar 

  61. R.T. Rockafellar, “Local boundedness of nonlinear, monotone operators,” Michigan Math. J., vol. 16, pp. 397- 407, 1969.

    Google Scholar 

  62. R.T. Rockafellar, Convex Analysis, Princeton University Press: Princeton, NJ, 1970.

    Google Scholar 

  63. R.T. Rockafellar, “On the maximality of sums of nonlinear monotone operators,” Trans. Amer. Math. Soc., vol. 149, pp. 75-88, 1970.

    Google Scholar 

  64. A. Ruszczyński, “An augmented Lagrangian decomposition method for block diagonal linear programming problems,” Oper. Res. Lett., vol. 8, pp. 287-294, 1989.

    Google Scholar 

  65. G.L. Schultz and R.R. Meyer, “An interior point method for block angular optimization,” SIAM J. Optim., vol. 1, pp. 583-602, 1992.

    Google Scholar 

  66. P. Tseng, “Dual ascent methods for problems with strictly convex costs and linear constraints: A unified approach,” SIAM J. Control Optim., vol. 28, pp. 214-242, 1990.

    Google Scholar 

  67. P. Tseng, “Decomposition algorithm for convex differentiable minimization,” J. Optim. Theory Appl., vol. 70, pp. 109-135, 1991.

    Google Scholar 

  68. P. Tseng, “On the rate of convergence of a partially asynchronous gradient projection algorithm,” SIAM J. Optim., vol. 1, pp. 603-619, 1992.

    Google Scholar 

  69. P. Tseng, D.P. Bertsekas, and J.N. Tsitsiklis, “Partially asynchronous, parallel algorithms for network flow and other problems,” SIAM J. Control Optim., vol. 28, pp. 678-710, 1990.

    Google Scholar 

  70. P. Tseng and D.P. Bertsekas, “Relaxation methods for problems with strictly convex costs and linear constraints,” Math. Oper. Res., vol. 16, pp. 462-481, 1991.

    Google Scholar 

  71. J.N. Tsitsiklis and D.P. Bertsekas, “Distributed asynchronous optimal routing in data networks,” IEEE Trans. Automat. Control, vol. AC-31, pp. 325-332, 1986.

    Google Scholar 

  72. A. Üresin and M. Dubois, “Sufficient conditions for the convergence of asynchronous iterations,” Parallel Comput., vol. 10, pp. 83-92, 1989.

    Google Scholar 

  73. A. Üresin and M. Dubois, “Asynchronous iterative algorithms: Models and convergence,” in Advances in Parallel Algorithms, L. Kronsjö and D. Shumsheruddin (Eds.), Blackwell Scientific Publications: Oxford, Chap. 10, pp. 302-342, 1992.

    Google Scholar 

  74. J. van Tiel, Convex Analysis: An Introductory Text, John Wiley & Sons: Chichester, U.K., 1984.

    Google Scholar 

  75. N. Zadeh, “A note on the cyclic coordinate ascent method,” Management Sci., 16 (1969/1970), pp. 642-644.

    Google Scholar 

  76. W.I. Zangwill, Nonlinear Programming: A Unified Approach, Prentice-Hall: Englewood Cliffs, NJ, 1969.

    Google Scholar 

  77. E. Zeidler, Nonlinear Functional Analysis and Its Applications II/B: Nonlinear Monotone Operators, Springer-Verlag: New York, NY, 1990.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Washington, Seattle, WA

    Michael Patriksson

Authors
  1. Michael Patriksson
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Patriksson, M. Decomposition Methods for Differentiable Optimization Problems over Cartesian Product Sets. Computational Optimization and Applications 9, 5–42 (1998). https://doi.org/10.1023/A:1018358602892

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1018358602892

Search

Navigation