Skip to main content
SpringerLink
Log in

The neuro-dynamic scheme for solving general form of discrete time optimal control problems

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we show that recently developed neural network methods for quadratic programming can be put to use in solving discrete time optimal control problems, with general pointwise constraints on states and controls. We describe a high performance recurrent neural network for a discrete time linear quadratic regulator problem with mixed state–control constraints. The equilibrium point of the proposed model is proved to be equivalent to the optimal solution of the discrete time problem. It is also shown that the proposed network model is stable in the Lyapunov sense and it is globally convergent to an exact optimal solution of the original problem. Several practical examples are provided to show the feasibility and the efficiency of the scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32

Similar content being viewed by others

References

  1. Toan NT, Thuy LQ Second-order necessary optimality conditions for a discrete optimal control problem with mixed constraints, Journal of Global Optimization, https://doi.org/10.1007/s10898-015-0333-0

  2. Toan NT, Ansari QH, Yao J-C Second-Order Necessary Optimality Conditions for a Discrete Optimal Control Problem, Journal of Optimization Theory and Applications, DOI https://doi.org/10.1007/s10957-014-0648-x

  3. Marchand PA, Lawrencet PD, Cherchas DB (1989) A discrete time optimal control law for a robot arm. Opt Control Appl Methods 10:1–20

    Article  MathSciNet  MATH  Google Scholar 

  4. Leyendecker S, Ober-Blobaum S, Marsden JE, Ortiz M (2010) Discrete mechanics and optimal control for constrained systems. Opt Control Appl Methods 31:505–528

    Article  MathSciNet  MATH  Google Scholar 

  5. Sandblom C-L, Eiselt HA, Jornasten K (1987) Discrete time optimal contriol of an economic system using differtent objective functions. Opt Control Appl Methods 8:253–269

    Article  Google Scholar 

  6. Tan F, Luo B, Guan X (2015) Finite-horizon 𝜖-optimal tracking control of discrete-time linear systems using iterative approximate dynamic programming. Asian J Control 17(1):176–189

    Article  MathSciNet  MATH  Google Scholar 

  7. Al-Tamimi A, Lewis FL, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof. IEEE Trans Syst Man Cybern 38:943–949

    Article  Google Scholar 

  8. Bemporad A, Borrelli F, Morari M (2002) Model predictive control based on linear programming – the explicit solution. IEEE Trans Autom Control 47:1974–1985

    Article  MathSciNet  MATH  Google Scholar 

  9. Bemporad A, Borrelli F, Morari M (2003) Min-max control of constrained uncertain discrete-time linear systems. IEEE Trans Autom Control 48:1600–1606

    Article  MathSciNet  MATH  Google Scholar 

  10. Boltyanskii VG (1978) Optimal control of discrete systems. Wiley, New York

    Google Scholar 

  11. Ioffe AD, Tikhomirov VM (1979) Theory of extremal problems. North-Holland, Amsterdam

    Google Scholar 

  12. Propoi AI (1973) Elements of the theory of optimal discrete processes. Moscow, Nauka. (in Russian)

    Google Scholar 

  13. Kalman RE (1960) Contributions to the theory of optimal control. Bullet Soc Mex 5:102–119

    MathSciNet  Google Scholar 

  14. Chyung DH (1966) Discrete 1I IWM- optimal control system with essentially quadratic cost functionals. IEEE Trans Autom Control 11:404–413

    Article  Google Scholar 

  15. Deley GW, Franklin GF (1965) Optimal bounded control of linear sampled-data systems with quadratic loss. J Basic Eng 57:135–141

    Article  Google Scholar 

  16. Lee EB (1963) Recurrence equations and the control of their evolution. J Math Anal Appl 7:118–126

    Article  MathSciNet  MATH  Google Scholar 

  17. Eaton JH (1963) An online solution to sampled-data time optimal control. J Electron Control 15:333–341

    Article  Google Scholar 

  18. Koepcke RW (1963) A solution to the sampled minimum-time problem. J Basic Eng 86:145–150

    Article  Google Scholar 

  19. Ogata K (1995) Discrete-time control systems, 2nd edn. Prentice-Hall, New Jersey

    Google Scholar 

  20. Pokoski J (1965) An analysis scheme for suboptimal minimum-time sampled-data systems. Joint Autom Control Conf. 15:270–257

    Google Scholar 

  21. Itoh U (1971) Optimal control of the discrete linear system with the bounded controller and the quadratic cost functional (in Japanese). J Inst Electr Eng Jpn 91:521–530

    Google Scholar 

  22. Halkin H (1964). In: Leondes CT (ed) Optimal control for systems described by difference equations, in Advances in Control Systems. Academic Press, New York

  23. Butkovskii AG (1963) The necessary and sufficient conditions for optimality of discrete control systems. Autom Remote Control 24:963–970

    Google Scholar 

  24. Jordan BW, Polak E (1964) Theory of a class of discrete optimal systems. Theory Class Discret Opt Control Syst 17:697–711

    MathSciNet  Google Scholar 

  25. Katz S, Kranc GM (1969) On the least time control problem with interior output constraints. IEEE Trans Autom Control 14:255–261

    Article  MathSciNet  Google Scholar 

  26. Kranc GM, Shilman MB (1970) An application of functional analysis to time optimal control of linear discrete systems with output constraints. J Frankl Inst 290:137–147

    Article  MathSciNet  MATH  Google Scholar 

  27. Pantoja JFAD, Mayne DQ (1991) Sequential quadratic programming algorithm for discrete optimal control problems with control inequality constraints. Int J Control 53:823–836

    Article  MathSciNet  MATH  Google Scholar 

  28. Wright SJ (1990) Solution of discrete-time optimal control problems on parallel computers. Parallel Comput 16:221–238

    Article  MathSciNet  MATH  Google Scholar 

  29. Rockafellar RT, Wets RJ (1990) Generalized linear-quadratic problems of deterministic and stochastic optimal control in discrete time. SIAM J Control Optim 28:810–822

    Article  MathSciNet  MATH  Google Scholar 

  30. Ohno K (1978) A new approach to differential dynamic programming for discrete- time systems. IEEE Trans Autom Control AC-23:37–47

    Article  MathSciNet  MATH  Google Scholar 

  31. Sage AP, White III CC (1977) Prentice-Hall, New Jersey

  32. Liu X, Li Y, Zhang W (2014) Stochastic linear quadratic optimal control with constraint for discrete-time systems. Appl Math Comput 228:264–270

    MathSciNet  MATH  Google Scholar 

  33. wright SJ (1993) Interior point methods for optimal control of discrete time systems. J Optim Theory Appl 77:161–187

    Article  MathSciNet  MATH  Google Scholar 

  34. Borrelli F, Baoti M, Bemporad A, Morari M (2005) Dynamic programming for constrained optimal control of discrete-time linear hybrid systems. Automatica 41:1709–1721

    Article  MathSciNet  MATH  Google Scholar 

  35. Sontag ED (1981) Nonlinear regulation: The piecewise linear approach. IEEE Trans Autom Control 26:346–358

    Article  MathSciNet  MATH  Google Scholar 

  36. Mayne DQ (2001) Constrained optimal control. In: European control conference. Plenary lecture, Porto

  37. Baotic M, Vasak M, Morari M, Peric N (2003) Hybrid theory based optimal control of electronic throttle. In: Proceeding American Control Conference, Denver

  38. Branicky MS, Borkar VS, Mitter SK (1998) A unified framework for hybrid control: model and optimal control theory. IEEE Trans Autom Control 43:31–45

    Article  MathSciNet  MATH  Google Scholar 

  39. Xu X, Antsaklis PJ (2003) Results and perspectives on computational methods for optimal control of switched systems. In: Maler O, Pnueli A (eds) Hybrid Systems: Computation and Control, HSCC 2003, volume 2623 ofLecture Notes in Computer Science. Springer Verlag, pp 540–556

  40. Bemporad A, Borodani P, Mannelli M (2003) Hybrid control ofan automotive robotized gearbox for reduction ofconsumptions and emissions. In: Maler O, Pnueli A (eds) Hybrid systems: Computation and control, HSCC 2003, Lecture notes in computer science, vol 2623. Springer, Berlin, pp 81—96

  41. Bemporad A, Giorgetti N, Kolmanovsky IV, Hrovat D (2002) Hybrid modeling and control ofa direct injection stratified charge engine. In: Symposium on advanced automotive technologies, ASME international mechanical engineering congress and exposition, New Orleans

  42. Bemporad A, Morari M (1999) Control ofsystems integrating logic, dynamics, and constraints. Automatica 35(3):407–427

    Article  MathSciNet  MATH  Google Scholar 

  43. F Borrelli A, Bemporad M, Fodor D, Hrovat D (2001) A hybrid approach to traction control. In: Sangiovanni-Vincentelli A, Di Benedetto MD (eds) Hybrid systems: Computation and control, Lecture notes in computer science, vol 2034. Springer, Berlin, pp 162–174

  44. Mignone D (2002) Control and estimation of hybrid systems via mathematical optimization. Dr. sc. tech. Thesis, Automatic Control Laboratory - ETH, Zurich. http://control.ee.ethz.ch

    Google Scholar 

  45. Möbus R, Baotic M, Morari M (2003) Multi-object adaptive cruise control. In: Maler O, Pnueli A (eds) Hybrid systems: Computation and control, HSCC 2003, Lecture notes in computer science, vol 2623. Springer, Berlin, pp 359–374

  46. Torrisi FD, Bemporad A (2004) HYSDELA tool for generating computational hybrid models. IEEE Trans Control Syst Technol 12(2):235–249

    Article  Google Scholar 

  47. Borrelli F, Baotic M, Bemporad A, Morari M (2003) Constrained optimal control of discrete-time linear hybrid systems. Technical Report AUT03-05, Automatic Control Laboratory. ETH Zurich, Switzerland

    Google Scholar 

  48. Tank DW, Hopfield JJ (1986) Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming pircuit. IEEE Trans Circ Syst 33:533–541

    Article  Google Scholar 

  49. Effati S, Nazemi AR (2006) Neural network models and its application for solving linear and quadratic programming problems. Appl Math Comput 172:305–331

    MathSciNet  MATH  Google Scholar 

  50. Effati S, Ghomashi A, Nazemi AR (2007) Application of projection neural network in solving convex programming problems. Appl Math Comput 188:1103–1114

    MathSciNet  MATH  Google Scholar 

  51. Forti M, Nistri P, Quincampoix M (2006) Convergence of neural networks for programming problems via a nonsmooth Lojasiewicz inequality. IEEE Trans Neural Netw 17:1471–1486

    Article  Google Scholar 

  52. Gao XB, Liao L-Z, Qi LQ (2005) A novel neural network for variational inequalities with linear and nonlinear constraints. IEEE Trans Neural Netw 16:1305–1317

    Article  Google Scholar 

  53. Hu X (2009) Applications of the general projection neural network in solving extended linear-quadratic programming problems with linear constraints. Neurocomputing 72:1131–1137

    Article  Google Scholar 

  54. Hu X, Wang J (2007) Design of general projection neural networks for solving monotone linear variational inequalities and linear and quadratic optimization problems. IEEE Trans Syst Man Cybern Part B 37:1414–1421

    Article  Google Scholar 

  55. Liu QS, Wang J (2008) A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming. IEEE Trans Neural Netw 19:558–570

    Article  Google Scholar 

  56. Malek A, Hosseinipour-Mahani N, Ezazipour S (2010) Efficient recurrent neural network model for the solution of general nonlinear optimization problems. Optim Methods Softw 25:1–18

    Article  MathSciNet  MATH  Google Scholar 

  57. Nazemi AR (2012) A dynamic system model for solving convex nonlinear optimization problems. Commun Nonlinear Sci Numer Simul 17:1696–1705

    Article  MathSciNet  MATH  Google Scholar 

  58. Nazemi AR (2014) A neural network model for solving convex quadratic programming problems with some applications. Eng Appl Artif Intell 32:54–62

    Article  Google Scholar 

  59. Nazemi AR, Dehghan M (2015) A neural network method for solving support vector classification problems. Neurocomputing 152:369–376

    Article  Google Scholar 

  60. Wu H, Shi R, Qin L, Tao F, He L (2010) A nonlinear projection neural network for solving interval quadratic programming problems and its stability analysis. Math Probl Eng 2010:1–13

    MathSciNet  MATH  Google Scholar 

  61. Xia Y, Feng G (2005) An improved network for convex quadratic optimization with application to real-time beamforming. Neurocomputing 64:359–374

    Article  Google Scholar 

  62. Xue X, Bian W (2007) A project neural network for solving degenerate convex quadratic program. Neurocomputing 70:2449–2459

    Article  Google Scholar 

  63. Yang Y, Cao J (2008) A feedback neural network for solving convex constraint optimization problems. Appl Math Comput 201:340–350

    MathSciNet  MATH  Google Scholar 

  64. Monteiro RDC, Adler I (1989) Interior path-following primal–dual algorithms, Part 2: convex quadratic programming. Math Programm 44:43–66

    Article  MATH  Google Scholar 

  65. Mangasarian OL (1969) Nonlinear programming. McGraw-Hill, New York

    MATH  Google Scholar 

  66. Ferreira JAS, Vidal RVV (1984) Optimization of a pump-pipe system by dynamic programming. Eng Optim 7:241–251

    Article  Google Scholar 

  67. Ritch PS (1973) Discrete optimal control with multiple constraints I: constraint separation and transformation technique. Automatica 9:415–429

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Mathematical Science, Shahrood University of Technology, P.O. Box 3619995161-316, Shahrood, Iran

    Alireza Nazemi, Samira Sukhtsaraie & Marzieh Mortezaee

Authors
  1. Alireza Nazemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samira Sukhtsaraie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marzieh Mortezaee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alireza Nazemi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nazemi, A., Sukhtsaraie, S. & Mortezaee, M. The neuro-dynamic scheme for solving general form of discrete time optimal control problems. Appl Intell 48, 3178–3191 (2018). https://doi.org/10.1007/s10489-017-1131-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-017-1131-9

Keywords

Search

Navigation