An ecologically inspired direct search method for solving optimal control problems with Bézier parameterization

Abstract

An optimal control problem can be formulated through a set of differential equations describing the trajectory of the control variables that minimize the cost functional (related to both state and control variables). Direct solution methods for optimal control problems treat them from the perspective of global optimization: i.e. perform a global search for the control function that optimizes the required objective. In this article we use a recently developed ecologically inspired optimization technique called Invasive Weed Optimization (IWO) for solving such optimal control problems. Usually the direct solution method operates on discrete n-dimensional vectors and not on continuous functions. Consequently it can become computationally expensive for large values of n. Thus, a parameterization technique is required to represent the control functions using a small number of real-valued parameters. Typically, direct methods based on evolutionary computing techniques parameterize control functions with a piecewise constant approximation. This has obvious limitations both for accuracy in representing arbitrary functions, and for optimization efficiency. In this paper a new parameterization is introduced using Bézier curves, which can accurately represent continuous control functions with only a few parameters. It is combined with IWO into a new evolutionary direct method for optimal control. The effectiveness of the new method is demonstrated by solving a wide variety of optimal control problems.

Introduction

Optimal control deals with the problem of finding a control law for a given system such that a certain optimality criterion can be achieved. The control problem includes a cost functional that is a function of state and control variables. The optimal control problem is formulated as a set of differential equations describing the paths of the control variables that minimize the cost function. It was developed by inter alia—a bunch of Russian mathematicians, among whom the leading role was taken by Pontryagin. Just like for other dynamical systems, the state variables, here, are represented by $x(t)\in {\mathbb{R}}^n$. The state variables are controlled by the set of independent functions u(t), called control variables. An obvious goal is to find that optimizes in some sense performance of the dynamical system. Mathematically this problem can be stated as follows:

$$\min F(u)={\int }_{t_0}^{t_f}f(t,x(t),u(t))dt$$

$$\text{subject}\text{to}\{\begin{array}{l}x\prime (t)=g(t,x(t),u(t))\hfill \\ x(t_0)=x_0\hfill \end{array}$$

where t₀ and t_f are the initial and final times and f, g depend on the particular system model. It is to be noted that the optimal control problem, as stated above, may have multiple solutions (i.e., a solution may not be unique). Thus, it is most often the case that any solution to the optimal control problem is only locally minimizing the objective representing system's performance.

There are two general approaches to solve the optimal control problems. These are often labeled as direct and indirect methods (von Stryk and Bulirsch, 1992). An indirect method transforms the problem into another form before solving it. Typically Pontryagin's Maximum Principle (Pontryagin, 1962) is used to find the necessary conditions for the existence of an optimum. This allows the original optimal control problem to be transformed into a Boundary Value Problem (BVP), which can then be solved analytically or numerically using well-known techniques for differential equations. This boundary-value problem actually has a special structure because it arises from taking the derivative of a Hamiltonian. The indirect method is sometimes described as first optimize then discretize because optimality conditions are found before numerical techniques are applied. These techniques were used in early years of optimal control. The disadvantage of indirect methods is that the boundary-value problem is often extremely difficult to solve (particularly for problems that span large time intervals or problems with interior point constraints). An excellent introduction to this method can be found in a recent text by Lenhart and Workman (2007).

In a direct method, optimal control is seen as a standard optimization problem: perform a search for the control function u(t) that optimizes the objective functional. However, optimization routines do not operate on infinite-dimensional spaces. So before optimizing, the state control variables are approximated using an appropriate function approximation like the piecewise constant parameterization. Since the parameterization used is often a straightforward discretization of the continuous space, the direct method has been described as: first discretize then optimize.

Recently a considerable attention has been paid by the researchers towards employing algorithms inspired from natural processes and/or events in order to solve optimization problems. For example, Genetic Algorithms (GAs) (Holland, 1975), which mimics the process of Darwinian evolution and natural genetics, are now a standard optimization tool in engineering. There are also other numerical, direct search and optimization methods, e.g. Ant Colony Optimization (ACO) (Dorigo et al., 1996) and Particle Swarm Optimization (PSO) (Kennedy and Eberhart, 1995). Since past few years, researchers have been attempting to imitate the ecological phenomenon of nature for solving engineering optimization problems. For instance, a novel Evolutionary Algorithm (EA), inspired by the nature of spatial interactions in ecological systems, was proposed by Kirley (2002), where the author examined the response of the evolving population to the process of fragmentation and disturbance cased by natural events (like fire, floods or climate changes). Another ecology-inspired EA for constrained optimization was presented by Yuchi and Kim (2005). In the mentioned research, in each generation, according to the feasibility of the individuals, the whole population is divided into two groups: feasible group and infeasible group. Evaluation and ranking of these two groups are performed in parallel and separately. The best individuals from feasible and infeasible groups are selected together as parents. The number of feasible parents has a sigmoid-type relation with that of the feasible individuals, which is inspired by the natural ecological population growth in a confined space. Following the same tradition, Mehrabian and Lucas (2006) proposed the Invasive Weed Optimization (IWO), a derivative-free, metaheuristic algorithm, mimicking the ecological behavior of colonizing weeds. Weeds have been shown to be very robust and adaptive to the changes in environment. As IWO is designed to capture the properties of the weeds, it has been emerged as a powerful optimization algorithm. Since its inception, IWO has found successful applications in many practical optimization problems like optimization and tuning of a robust controller (Mehrabian and Lucas, 2006), optimal positioning of piezoelectric actuators (Mehrabian and Yousefi-Koma, 2007), developing a recommender system (Rad and Lucas, 2007), antenna array optimization (Roy et al., 2011), design of E-shaped MIMO Antenna (Mallahzadeh et al., 2009) and design of encoding sequences for DNA computing (Zhang et al., 2009).

To use an EA for optimal control, a parameterization strategy is required by which control functions can be represented by the vectors on which the EA may operate. This is commonly known as Control Vector Parameterization (CVP). A wide variety of CVPs have been used with non-evolutionary optimizers, including piecewise constant (Goh and Teo, 1988), Chebyshev polynomials (Vlassenbroeck, 1988), Lagrange polynomials (Biegler, 1984), and piecewise Lagrange polynomials (Vassiliadis et al., 1994). This article presents a new direct evolutionary method to solve optimal control problems where an improved variant of IWO is used along with Bézier curves (Bézier, 1972) to parameterize the control functions. The new method is designed to achieve both accuracy and efficiency simultaneously. Rest of the paper is organized in the following way. Section 2 examines the direct methods for optimal control and the evolutionary direct methods. In Section 3 the Bézier parameterization technique is developed for use in conjunction with IWO. Section 3 also elaborates the IWO algorithm along with its proposed modification. Section 4 discusses the application of this method considering various instantiations of the optimal control problem. The focus here is to confirm that this new direct method is effective and efficient for a wide range of problems. In each case, the examples used can be solved analytically by an indirect method. This permits comparison of the two solutions and validates the proposed method. Section 5 demonstrates the effectiveness of the proposed approach by comparing the result obtained with this approach with the popular ecologically inspired optimization methods. The main focus here is to show that the modified version of IWO performs better as compared to a few significant nature-inspired stochastic algorithms used as evolutionary direct methods. Finally Section 6 concludes the paper uncovering a few future research directions.