Skip to main content
SpringerLink
Log in

Parallel cyclic reduction strategies for linear systems that arise in dynamic optimization problems

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Dynamic optimization problems are constrained by differential and algebraic equations and are found everywhere in science and engineering. A well-established method to solve these types of problems is direct transcription, where the differential equations are replaced with discretized approximations based on finite-difference or Runge–Kutta schemes. However, for problems with thousands of state variables and discretization points, direct transcription may result in nonlinear optimization problems which are too large for general-purpose optimization solvers to handle. Also, when an interior-point solver is applied, the dominant computational cost is solving the linear systems resulting from the Newton step. For large-scale nonlinear programming problems, these linear systems may become prohibitively expensive to solve. Furthermore, the systems also become too large to formulate and store in memory of a standard computer. On the other hand, direct transcription can exploit sparsity and structure of the linear systems in order to overcome these challenges. In this paper we investigate and compare two parallel linear decomposition algorithms, Cyclic Reduction (CR) and Schur complement decomposition, which take advantage of structure and sparsity. We describe the numerical conditioning of the CR algorithm when applied to the linear systems arising from dynamic optimization problems, and then compare CR with Schur complement decomposition on a number of test problems. Finally, we propose conditions under which each should be used, and describe future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Amodio, P., Mazzia, F.: Backward error analysis of cyclic reduction for the solution of tridiagonal systems. Math. Comput. 62(206), 601–617 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  2. Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users’ Guide, 3rd edn. Society for Industrial and Applied Mathematics, Philadelphia (1999)

    Book  MATH  Google Scholar 

  3. Betts, J.: Practical Methods for Optimal Control and Estimation Using Nonlinear Programming, vol. 19. SIAM Series on Advances in Design and Control, Philadelphia (2010)

    Book  MATH  Google Scholar 

  4. Biegler, L.T.: Nonlinear Programming: Concepts, Algorithms, and Applications to Chemical Processes. SIAM, Philadelphia (2010)

    Book  MATH  Google Scholar 

  5. Biegler, L.T., Cervantes, A.M., Wächter, A.: Advances in simultaneous strategies for dynamic process optimization. Chem. Eng. Sci. 57(4), 575–593 (2002)

    Article  Google Scholar 

  6. Binder, T., Blank, L., Bock, H.G., Bulirsch, R., Dahmen, W., Diehl, M., Kronseder, T., Marquardt, W., Schlöder, J.P., von Stryk, O.: Introduction to Model Based Optimization of Chemical Processes on Moving Horizons. In: Grötschel, M., Krumke, S.O., Rambau, J. (eds.) Online Optimization of Large Scale Systems, pp. 295–339. Springer, Berlin (2001)

    Chapter  Google Scholar 

  7. Bini, D.A., Meini, B.: The cyclic reduction algorithm: from poisson equation to stochastic processes and beyond. Num. Algorithms 51(1), 23–60 (2009)

    Article  MATH  Google Scholar 

  8. Borzi, A., Schulz, V.: Computational Optimization of Systems Governed by Partial Differential Equations. SIAM, Philadelphia (2012)

    MATH  Google Scholar 

  9. Bryson, A.E., Ho, Y.C.: Applied Optimal Control: Optimization, Estimation and Control. CRC Press, Boca Raton (1975)

    Google Scholar 

  10. Buneman, O.: Compact non-iterative Poisson solver. Technical Reports, Stanford University, California Institute for Plasma Research (1969)

  11. Byrd, R.H., Hribar, M.E., Nocedal, J.: An interior point algorithm for large scale nonlinear programming. SIAM J. Opt. 9, 877–900 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  12. Cannataro, B.Ş., Rao, A.V., Davis, T.A.: State-defect constraint pairing graph coarsening method for Karush–Kuhn–Tucker matrices arising in orthogonal collocation methods for optimal control. Comput. Optim. Appl. 64(3), 793–819 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  13. Chang, S.C., Chang, T.S., Luh, P.B.: A hierarchical decomposition for large-scale optimal control problems with parallel processing structure. Automatica 25(1), 77–86 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  14. Chiang, N., Petra, C.G., Zavala, V.M.: Structured nonconvex optimization of large-scale energy systems using PIPS-NLP. Power Syst. Comput. Conf. (PSCC) 2014, 1–7 (2014). https://doi.org/10.1109/PSCC.2014.7038374

    Google Scholar 

  15. Davis, T.A.: Algorithm 832: UMFPACK v4.3-an unsymmetric-pattern multifrontal method. ACM Trans. Math. Softw. 30(2), 196–199 (2004). https://doi.org/10.1145/992200.992206

    Article  MathSciNet  MATH  Google Scholar 

  16. Diamond, M.A., Ferreira, D.L.V.: On a cyclic reduction method for the solution of Poisson’s equations. SIAM J. Numer. Anal. 13(1), 54–70 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  17. Diehl, M., Bock, H.G., Schlöder, J.P., Findeisen, R., Nagy, Z., Allgöwer, F.: Real-time optimization and nonlinear model predictive control of processes governed by differential-algebraic equations. J. Process Control 12(4), 577–585 (2002)

    Article  Google Scholar 

  18. Duff, I.S.: MA57–a code for the solution of sparse symmetric definite and indefinite systems. ACM Trans. Math. Softw. 30(2), 118–144 (2004). https://doi.org/10.1145/992200.992202

    Article  MathSciNet  MATH  Google Scholar 

  19. Feng, D., Schnabel, R.B.: Globally convergent parallel algorithms for solving block bordered systems of nonlinear equations. Optim. Methods Softw. 2(3–4), 269–295 (1993)

    Article  Google Scholar 

  20. Fiacco, A., McCormick, G.: Nonlinear Programming: Sequential Unconstrained Minimization Techniques. Wiley, New York (1968)

    MATH  Google Scholar 

  21. Frasch, J.: Parallel Algorithms for Optimization of Dynamic Systems in Real-Time. Ph.D. thesis, Otto-von-Guericke University Magdeburg (2014). http://www.mathopt.de/PUBLICATIONS/Frasch2014.pdf

  22. Gondzio, J.: Matrix-free interior point method. Comput. Optim. Appl. 51(2), 457–480 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hager, W.W., Hou, H., Rao, A.V.: Lebesgue constants arising in a class of collocation methods. IMA J. Numer. Anal. 37, 1884 (2016)

    MathSciNet  Google Scholar 

  24. Hartwich, A., Stockmann, K., Terboven, C., Feuerriegel, S., Marquardt, W.: Parallel sensitivity analysis for efficient large-scale dynamic optimization. Optim. Eng. 12(4), 489–508 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  25. Heller, D.: Some aspects of the cyclic reduction algorithm for block tridiagonal linear systems. SIAM J. Numer. Anal. 13(4), 484–496 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hirshman, S.P., Perumalla, K.S., Lynch, V.E., Sánchez, R.: Bcyclic: a parallel block tridiagonal matrix cyclic solver. J. Comput. Phys. 229(18), 6392–6404 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  27. Hockney, R., Jesshope, C.: Parallel computers: architecture, programming and algorithms. Adam Hilger (1981). https://books.google.com/books?id=CP5gzotAXI4C

  28. Hockney, R.W.: A fast direct solution of poisson’s equation using fourier analysis. J. ACM 12(1), 95–113 (1965). https://doi.org/10.1145/321250.321259

    Article  MathSciNet  MATH  Google Scholar 

  29. Kang, J., Cao, Y., Word, D.P., Laird, C.D.: An interior-point method for efficient solution of block-structured nlp problems using an implicit schur-complement decomposition. Comput. Chem. Eng. 71, 563–573 (2014)

    Article  Google Scholar 

  30. Kang, J., Chiang, N., Laird, C.D., Zavala, V.M.: Nonlinear programming strategies on high-performance computers. In: IEEE 54th annual conference on decision and control (CDC), pp. 4612–4620 (2015)

  31. Körkel, S., Kostina, E., Bock, H., Schlöder, J.: Numerical methods for optimal control problems in design of robust optimal experiments for nonlinear dynamic processes. Optim. Methods Softw. 19(34), 327338 (2004)

    MathSciNet  MATH  Google Scholar 

  32. Laird, C.D., Wong, A.V., Åkesson, J.: Parallel solution of large-scale dynamic optimization problems. In: 21st European symposium on computer-aided process engineering (2011)

  33. Le, D.D.: Parallelisierung von Innere–Punkte–Verfahren mittels Cyclic Reduction. Bachelor’s thesis, OvGU Magdeburg (2014). http://www.mathopt.de/PUBLICATIONS/Do2014.pdf

  34. Leineweber, D.B., Bauer, I., Bock, H.G., Schlöder, J.P.: An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization. Part 1: theoretical aspects. Comput. Chem. Eng. 27(2), 157–166 (2003)

    Article  Google Scholar 

  35. Leineweber, D.B., Schäfer, A., Bock, H.G., Schlöder, J.P.: An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization: Part II: software aspects and applications. Comput. Chem. Eng. 27(2), 167–174 (2003)

    Article  Google Scholar 

  36. Leugering, G., Engell, S., Griewank, A., Hinze, M., Rannacher, R., Schulz, V., Ulbrich, M., Ulbrich, S.: Constrained Optimization and Optimal Control for Partial Differential Equations. Springer, Basel (2016). https://doi.org/10.1007/978-3-0348-0133-1

    MATH  Google Scholar 

  37. MATLAB: version 8.3.0 (R2012a). The MathWorks Inc., Natick, Massachusetts (2012)

  38. MATLAB: MATLAB function reference, pp. 5702–5709. The MathWorks Incorporated (2015)

  39. Pearson, J.W., Gondzio, J.: Fast interior point solution of quadratic programming problems arising from pde-constrained optimization. Numerische Mathematik pp. 1–41 (2016)

  40. Petra, C.G., Schenk, O., Anitescu, M.: Real-time stochastic optimization of complex energy systems on high-performance computers. Comput. Sci. Eng. 16(5), 32–42 (2014)

    Article  Google Scholar 

  41. Pontryagin, L., Boltyanskii, V., Gamkrelidze, R., Mishchenko, E.: Mathematical Theory of Optimal Processes. Interscience Publishers Inc., New York (1962)

    Google Scholar 

  42. Qin, S.J., Badgwell, T.A.: An Overview of Nonlinear Model Predictive Control Applications. In: Allgöwer, F., Zheng, A. (eds.) Nonlinear Model Predictive Control, pp. 369–392. Springer, Berlin (2000)

    Chapter  Google Scholar 

  43. Rosmond, T.E., Faulkner, F.D.: Direct solution of elliptic equations by block cyclic reduction and factorization. Mon. Weather Rev. 104(5), 641–649 (1976)

    Article  Google Scholar 

  44. Steinbach, M.C.: A structured interior point SQP method for nonlinear optimal control problems. In: Computational Optimal Control, vol. 115 of International Series of Numerical Mathematics, pp. 213–222. Birkhäuser (1994)

  45. Steinbach, M.C.: Structured interior point sqp methods in optimal control. Z. Angew. Math. Mech 76(S3), 59–62 (1996)

    MathSciNet  MATH  Google Scholar 

  46. Swarztrauber, P.N.: A direct method for the discrete solution of separable elliptic equations. SIAM J. Numer. Anal. 11(6), 1136–1150 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  47. Swarztrauber, P.N., Sweet, R.A.: Algorithm 541: efficient fortran subprograms for the solution of separable elliptic partial differential equations [d3]. ACM Trans. Math. Softw. 5(3), 352–364 (1979). https://doi.org/10.1145/355841.355850

    Article  MATH  Google Scholar 

  48. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106, 25–57 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  49. Word, D.P., Kang, J., Akesson, J., Laird, C.D.: Efficient parallel solution of large-scale nonlinear dynamic optimization problems. Comput. Optim. Appl. 59(3), 667–688 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  50. Wright, S.J.: Partitioned dynamic programming for optimal control. SIAM J. Optim. 1(4), 620–642 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  51. Yalamov, P., Pavlov, V.: Stability of the block cyclic reduction. Linear Algebra Appl. 249(1), 341–358 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  52. Zavala, V.M., Laird, C.D., Biegler, L.T.: Interior-point decomposition approaches for parallel solution of large-scale nonlinear parameter estimation problems. Chem. Eng. Sci. 63(19), 4834–4845 (2008)

    Article  Google Scholar 

  53. Zhang, X., Byrd, R.H., Schnabel, R.B.: Parallel methods for solving nonlinear block bordered systems of equations. SIAM J. Sci. Stat. Comput. 13(4), 841–859 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  54. Zhang, Y., Cohen, J., Owens, J.D.: Fast tridiagonal solvers on the GPU. ACM Sigplan Notices 45(5), 127–136 (2010)

    Article  Google Scholar 

Download references

Acknowledgements

Funding from ExxonMobil Research and Engineering Company is gratefully acknowledged.

Author information

Authors and Affiliations

  1. Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Bethany L. Nicholson, Wei Wan & Lorenz T. Biegler

  2. ExxonMobil Research and Engineering Company, Annandale, NJ, 08801, USA

    Shivakumar Kameswaran

Authors
  1. Bethany L. Nicholson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shivakumar Kameswaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lorenz T. Biegler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lorenz T. Biegler.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nicholson, B.L., Wan, W., Kameswaran, S. et al. Parallel cyclic reduction strategies for linear systems that arise in dynamic optimization problems. Comput Optim Appl 70, 321–350 (2018). https://doi.org/10.1007/s10589-018-0001-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-018-0001-7

Keywords

Search

Navigation