Elsevier

Automatica

Volume 101, March 2019, Pages 388-395
Automatica

Brief paper
A new computational approach for optimal control problems with multiple time-delay

https://doi.org/10.1016/j.automatica.2018.12.036Get rights and content

Abstract

The control parameterization method used together with the time-scaling transformation is an effective approach to approximating optimal control problems into optimal parameter selection problems when no time delays are involved. The approximate problems can then be solved by gradient-based optimization algorithms. However, the time-scaling transformation, which works well for optimizing variable switching times of the approximate piecewise constant/linear control functions obtained after the application of the control parameterization method, is not applicable to optimal control problems with time delays. In this paper, we consider a class of nonlinear optimal control problems with multiple time delays subject to canonical equality and inequality constraints. Our aim is to develop a novel transformation procedure that converts a given time-delay system into an equivalent system – defined on a new time horizon – in which the control switching times are fixed, but the dynamic system contains multiple variable time delays expressed in terms of the durations between the switching times for each of the approximate control functions in the original time horizon. On this basis, we show that an optimal control policy for the equivalent system can be obtained efficiently using gradient-based optimization techniques. This optimal control policy can then be used to determine the optimal switching times and optimal control variables for the original system. We conclude the paper by solving two example problems.

Introduction

A time-delay system is a dynamic system, which evolves depending not only on the current state and/or control variables but also on the state and/or control variables at some past time instants. Such systems arise in plethora of real-world applications, including epidemiological modeling (Bashier & Patidar, 2017), vehicle suspension design (Jing, 2016), and spacecraft attitude control (Denis-Vidal, Jauberthie, & Joly-Blanchard, 2006).

Control and stability for systems described by nonlinear delay differential equations have been well studied in the literature. See, for example, Ben and Hammami, 2014, Ben and Hammami, 2015 and Wang, Li, and Xu (2010). Due to its practical significance, finding optimal control strategies for optimal control problems governed by time-delay systems have attracted the attention of many researchers over the past several decades. However, except for few simple cases, it is impossible to find the closed-form analytical solution for time-delay optimal control problems, and hence most of the time-delay optimal control problems can only be solved numerically. To date, there exist many effective numerical methods for solving time-delay optimal control problems (Betts et al., Dadkhah et al., 2017, Dehghan and Keyanpour, 2017, Deshmuk et al., 2006, Göllmann et al., 2009, Göllmann and Maurer, 2014, Gong et al., 2017, Jajarmi and Hajipour, 2017, Marzban and Hoseini, 2016, Teo et al., 1991, Wu et al., 2006, Wu et al., 2015, Yang et al., 2016, Yu et al., 2016).

Control parameterization technique (Chai et al., 2013a, Chai et al., 2013b, Gong et al., 2017, Lin et al., 2014, Liu et al., 2015, Loxton et al., 2008, Teo et al., 1991, Yang et al., 2016) is a popular numerical method for solving general constrained optimal control problem, where the control function is approximated by the summation of a series of basis functions (piecewise constant/linear functions) on a partition of fixed subintervals. This leads to an optimal parameter selection problem, which is a finite-dimensional approximation of the original optimal control problem. Since this approximate problem can be viewed as a nonlinear optimization problem, it can be readily solved by standard gradient-based optimization techniques (e.g. sequential quadratic programming (Luenberger and Ye, 2008, Nocedal and Wright, 2006)). For the traditional control parameterization approach, the times at which the control switches from one value to another are fixed (normally evenly distributed on the time horizon). To obtain an accurate result, the partition of the time horizon is required to be very fine. Consequently, the finite-dimensional approximate optimization problem will consist of a large number of decision variables. To overcome this problem, the switching times should also be regarded as decision variables, but this approach can also be problematic for some situations (Lin et al., 2014).

The time-scaling transformation, which was first developed in Lee, Teo, Rehbock, and Jennings (1997), can be used to circumvent these difficulties. The time-scaling transformation works by introducing a new time variable and expressing the original time variable as a function of the new one, and the durations between the switching times for each of the approximate piecewise constant/linear functions form part of the decision variables to be chosen optimally. These decision variables together with the original decision variables can be optimized using standard numerical optimization techniques, see Lin et al. (2014) for details. The time-scaling transformation has been successfully applied in a wide range of applications (Lee et al., 1997, Lee et al., 1999, Wu and Teo, 2006, Yu et al., 2013). However, difficulties arise when applying the time-scaling transformation to optimal control problems involving systems with time delay, as fixed delays become variable delays in the new time horizon.

In Yu et al. (2016), a hybrid time-scaling transformation is proposed for solving nonlinear time-delay optimal control problems. Control parameterization technique used in conjunction with this hybrid time-scaling transformation can solve nonlinear time-delay optimal control problems. However, there is no closed form expression for the variable delay in the new time horizon. The delay state values in the new time horizon can only be obtained by numerical interpolation. The process is rather complicated. Furthermore, the durations between the switching times for the control vector are required to be greater than or equal to a pre-given positive value, which is not desirable in practical applications.

In this paper, we present a novel method for solving nonlinear optimal control problem with multiple time-delay. Compared with the hybrid time-scaling transformation (Yu et al., 2016), the new computational approach provides an explicit closed form expression for the delay time in the new time horizon, and the constraints on the durations between switching times to be greater than or equal to a pre-set positive number are removed. The rest of the paper is organized as follows. In Section 2, we formulate a combined switching time and parameter optimization problem for a general class of time-delay control systems. Then, in Section 3, we develop a new time-scaling transformation that converts the problem under consideration into an equivalent problem with fixed switching times. A key feature of the new problem is that the time-delay becomes variable. In Section 4, a gradient-based optimization algorithm is developed for solving the equivalent problem. In Section 5, we present two examples to demonstrate the effectiveness of the proposed new method. Finally, in Section 6, we conclude this paper with some remarks.

Section snippets

Problem formulation

Consider the following time-delay system, defined on the fixed time interval (,T]: ẋ(t)=f(x(t),x̄(t),u(t),ū(t)),t[0,T],x(t)=ϕ(t),t0,u(t)=φ(t),t<0, where x(t)=[x1(t),x2(t),,xn(t)]Rn is the state vector; u(t)=[u1(t),u2(t),,ur(t)]Rr is the control vector; x̄(t)=[x1(th1),x2(th2),,xn(thn)] and ū(t)=[u1(thn+1),u2(thn+2),,ur(thn+r)], in which hq>0,q=1,,n+r, are given time delays; f:Rn×Rn×Rr×RrRn and ϕ(t)=[ϕ1(t),,ϕn(t)] are given continuously differentiable functions; and φ(t)

Control parameterization

For linear control theory and methods, the dynamic systems are described by linear differential equations. Thus, they are not applicable to Problem (P). In what follows, we shall propose a numerical method to solve Problem (P). We subdivide the planning horizon [0,T] into p1 subintervals, ti,i=0,1,,p, are the partition points that satisfying 0=t0t1t2tp1tp=T.Let Ξ denote the set of all vectors σ=[t1,,tp] such that (7) is satisfied. Then the control u is approximated as follows: ui=1p

Gradient computation

To solve Problem (Q) using the gradient-based nonlinear optimization algorithms, we require the gradients of the cost and constraint functions with respect to each of their variables. We first rewrite g̃k(θ,δ),k=0,,Ne+Nm, in the following forms: g̃k(θ,δ)=Φk(y(pθ,δ))+0pμ(γ)γLˆk(y(γθ,δ),ȳ(γθ,δ),δ)dγ, where Lˆk(y(γθ,δ),ȳ(γθ,δ),δ)=i=1pLk(γθ,δ),ȳ(γθ,δ,δ(i))χ[i1,i)(γ).

Then we consider the derivative of ζ() with respect to θi,i=1,,p. Let S denote the set of points γ such that ζ(γ){0

Problem 1: optimal control with state delay

Consider the following multiple time-delay optimal control problem: ming0(u)=12x(tf)Sx(tf)+120tf{xQx+uRu}dt,subject to the time-delay dynamic system ẋ(t)=A1(t)x(t)+A2(t)x̄(t)+B(t)u(t),x(t)=[1,0],t0, where x̄=(x1(t1),x2(t0.5)) A1(t)=014π2(a+ccos2πt)0, A2(t)=004π2bcos2πt0,B(t)=01,the parameters of the problem are in Table 1.

and the control constraints are 3ui4,t[0,tf],i=1,2.By choosing different partition numbers, i.e., q=5,7,10, we obtain the corresponding optimal costs of g0(u)

Conclusion

In this paper, we develop a new method for multiple time-delay optimal control problems, where we convert the original multiple time-delay system into an equivalent system defined on the new time horizon with fixed switching times. At the same time, this new method transforms the fixed time-delays into some variables on the new time horizon, for which explicit form for the variable time-delay in the new time horizon is. Numerical results show that the proposed approach is more effective than

Di Wu received her B.Sc. and M.Sc. in Mathematics from Shanghai Normal University, China, in 2015 and Shanghai University, China, in 2018 respectively. She is a Ph.D. student of Department of Mathematics in Shanghai University since 2018. Her research interests focus on optimal control.

References (33)

  • DadkhahM. et al.

    Optimal control of a class of non-linear time-delay systems via hybrid functions

    IMA Journal of Mathematical Control and Information

    (2017)
  • DehghanR. et al.

    A numerical approximation for delay fractional optimal control problems based on the method of moments

    IMA Journal of Mathematical Control and Information

    (2017)
  • Denis-VidalL. et al.

    Identifiability of a nonlinear delayed-differential aerospace model

    IEEE Transactions on Automatic Control

    (2006)
  • DeshmukV. et al.

    Optimal control of parametrically excited linear delay differential system via chebyshev polynomials

    Optimal Control Applications and Methods

    (2006)
  • GöllmannL. et al.

    Optimal control problems with delays in state and control variables subject to mixed control-state constraints

    Optimal Control Applications

    (2009)
  • GöllmannL. et al.

    Theory and applications of optimal control problems with multiple time-delays

    Journal of Industrial and Management Optimization

    (2014)
  • Cited by (39)

    • Robust suboptimal feedback control for a fed-batch nonlinear time-delayed switched system

      2023, Journal of the Franklin Institute
      Citation Excerpt :

      The time-scaling transformation introduced in [38] can only apply to systems without time-delay. To vercome this difficulty, a new time-scaling transformation proposed in [43] is used in this paper for optimizing switching times in systems with time-delay. Problem C becomes Problem D given below.

    • Razumikhin and Krasovskii approaches for safe stabilization

      2022, Automatica
      Citation Excerpt :

      The second contribution of this paper is to address the stabilization and safety objectives simultaneously via the sliding mode control (SMC) approach, which is different from many existing works (Ames et al., 2016; Jankovic, 2018a) based on optimization techniques. The motivation of the proposed approach lies in that (nonlinear) time-delay optimal control problems are difficult to be solved to obtain closed-form analytical solutions (Wu, Bai and Yu, 2019). To begin with, the general sliding surface functional is constructed as the combination of the proposed control Lyapunov and barrier functions/functionals.

    • A computational study of optimal control of Markov jump systems

      2022, Journal of the Franklin Institute
      Citation Excerpt :

      In [29], a time grid refinement-based control parameterization method is proposed for flight level tracking optimal control problems. In [30], a control parameterization-based policy together with a novel time-scaling transformation method is introduced to study a new computational method for optimal control problems governed by time-delay systems. In this paper, we consider an optimal control problem governed by Markov jump system.

    View all citing articles on Scopus

    Di Wu received her B.Sc. and M.Sc. in Mathematics from Shanghai Normal University, China, in 2015 and Shanghai University, China, in 2018 respectively. She is a Ph.D. student of Department of Mathematics in Shanghai University since 2018. Her research interests focus on optimal control.

    Yanqin Bai received her Ph.D. in 1996 at the department of Mathematics of Shanghai University, China. She worked as both research Fellow and post-Doctoral Research during the period of 2001–2004 in Delft University of Technology, Netherland. Currently, Yanqin Bai holds professorship in Department of Mathematics at Shanghai University, China. She is a vice-president of Operations Research Society of China and the president of Operations Research Society of Shanghai. Yanqin Bai’s research interests include Linear Programming and Nonlinear Programming, Conic optimization and Interior-point methods, SVM and Machine Learning.

    Changjun Yu received his Ph.D. degrees in Mathematics from the Curtin University, Australia, 2012. He was Research Fellow with the Department of Mathematics and Statistics, Curtin University, Australia. In 2015, he won the title of “Youth Eastern Scholar” which is a prestigious academic prize for young researchers in Shanghai. In 2016, he joined the Department of Mathematics, Shanghai University, China, as Associate Professor. He was the first principle investigator of two NSFC programs. He has also participated in a number of NSFC Programs, such as an NSFC Key program, and three NSFC General Programs. He was also involved in two ARC Discovery Grants. He was a guest editor of two special issues, one for Journal of Industrial and Management Optimization and one for Pacific Journal of Optimization. His research interests include both the theoretical and practical aspects of optimal control and optimization, and their practical applications such as in signal processing in telecommunications.

    This work is supported by NSFC, China under grants 11871039 and 11771275. The material in this paper was partially presented at the Second Pacific Optimization Conference (POC2017) December 4–7, 2017, Perth, Australia. This paper was recommended for publication in revised form by Associate Editor Akira Kojima under the direction of Editor Ian R. Petersen.

    View full text