Data-driven asymptotic stabilization for discrete-time nonlinear systems☆
Introduction
Lyapunov approach provides a powerful framework for analyzing the stability of nonlinear dynamical systems as well as designing feedback controllers that guarantee closed-loop system stability [1], [2]. The synthesis typically relies on the co-design of a CLF and a state feedback controller. Historically, this is an important but challenging problem for the general class of nonlinear systems [3]. The main bottleneck to the success of these methods lies in the construction of CLF. Moreover, the DOA, an invariant set characterizing stabilizable area around the equilibrium, needs further investigation because global stabilization is difficult to achieve in practical applications [4]. There have been a number of studies to solve this problem. These results can be divided into two categories: model-based and data-driven.
Among model-based approaches, one simple approach to obtain a quadratic CLF is solving the Riccati equation associated with the linearized system of the nonlinear system, which often leads to a small DOA for the closed-loop due to approximation errors. Based on sum-of-squares (SOS) programming [5], [6], a polynomial CLF can be constructed and an enlarged estimate of DOA is simultaneously obtained [7], [8], [9]. However, SOS based methods can handle polynomial or rational systems only. SOS programming is extended to non-polynomial systems by variable transformation with algebraic constraints [10]. In [4], a fractional programming problem is formulated to construct a CLF for non-polynomial systems. The main disadvantage of model-based approaches is that the model of the nonlinear plant is the prerequisite and the model must be control-affine.
In fact, there are lots of plants which are hard to be effectively modeled. So designing controllers directly from data bypassing the modeling step, called Data-driven Control Approach, is promising and has received much attention recently. Many data-driven control approaches could be found, such as unfalsified control (UC) [11], model-free adaptive control (MFAC) [12], [13], etc. The main ideas of these approaches are quite different, but they do not require a plant model and do directly use data.
Among data-driven approaches, the control Lyapunov measure approach [14] deals with a similar problem with the one in this paper. Instead of a point-wise notion of the CLF, a control Lyapunov measure is constructed in the term of measure-theory for a nonlinear plant. This approach is data-driven, even though the authors do not say so. In this approach, the model is used to generate data for computing the Markov matrix. The disadvantage of this approach is that it leads to weaker coarse stability while our conclusion on stability is exact.
In this paper, we propose a data-driven asymptotic stabilization for discrete-time nonlinear systems, where a feedback controller asymptotically stabilizing the plant is obtained directly from data and the DOA of the closed-loop is enlarged. First, sufficient conditions for the feedback controller asymptotically stabilizing the plant are proposed. For a given CLF of the plant, if a feedback controller belongs to an open set consisting of pairs of control input and state of the plant, whose elements make the difference of the CLF to be negative-definite, the feedback controller can asymptotically stabilize the plant. However, under traditional controller design frameworks, it is hard to obtain the set for general nonlinear plants. Then, based on a data set collected from the plant and a given CLF candidate, an estimate of the set can be obtained. The idea of estimating the set, similar to set oriented numerical methods [15], is covering the set by a finite number of cells which contains data points satisfying some specific conditions. From the estimate of the set, it is easy to check whether the candidate is or is not a CLF. If it is, a feedback controller is designed using data, which satisfies sufficient conditions mentioned above. Finally, the estimate of DOA for closed-loop is enlarged by finding an appropriate CLF from a CLF candidate set based on data. An unconstrained nonlinear optimization problem, which can be solved by metaheuristic optimizers, is proposed to find the appropriate CLF. In our method, we directly use data bypassing the modeling step. Hence, complexity in building the model and modeling error are avoided.
This paper is organized as follows. In Section 2 the control problem is formulated. In Section 3, sufficient conditions for asymptotic stabilization and estimation of DOA for the closed-loop are introduced. In Section 4, the data-driven asymptotic stabilization is derived. In Section 5, the estimate of DOA for the closed-loop is enlarged by selecting an appropriate CLF from a CLF candidate set. Finally, in Section 6, the conclusion is drawn and further works are summarized.
Notation: represents the set of real numbers. represents the set of positive real numbers. represents . represents the set of positive integer numbers. represents . represents the set of real vectors with elements. For a vector represents . For a vector represents the -th element of . For a domain represents the Lebesgue measure of (in Euclidean space, it is the volume of ).
Section snippets
Problem formulation
Consider the nonlinear discrete-time system where is the state, is the control input, is an unknown piecewise continuous function satisfying and is asymptotically stabilizable at the origin.
Although is unknown, we have a data set collected from the plant (1) without measurement noises, where consists of the state trajectory and the control sequence, is the
Sufficient conditions for asymptotic stabilization and estimation of DOA for closed-loops
In this section, first, we introduce sufficient conditions for estimation of DOA for nonlinear discrete-time systems without control input in Lemma 1. Since the theory for nonlinear discrete-time systems closely parallels the theory for nonlinear continuous-time systems, many of the results are similar [1]. However, for the estimate of DOA by Lyapunov function, the discrete-time result deviates markedly from its continuous-time counterpart as illuminating in Remark 2. Then, sufficient
Data-driven asymptotic stabilization
The control problem formulated in Section 2 is solved in this section and the next section. In this section, for a given CLF candidate, we propose an algorithm to get an estimate of using the data set. With , it is easy to check whether the candidate is or is not a CLF. If it is, then a feedback controller satisfying conditions in Lemma 2 is designed based on and the data set. It should be noted that, during the above procedure, only data is used. The problem of finding
Enlarging estimate of DOA for closed-loop
In Section 4, we design an asymptotic stabilizer based on data set. However, our control objective is not finished yet. We hope to find an estimate of DOA for the closed-loop, which is as large as possible. This is solved by finding an appropriate CLF from a CLF candidate set by using data set.
According to Lemma 2, if the -level set , of a continuous positive-definite function , satisfies conditions (9), (11), then is an invariant subset of DOA. Based on this idea, the
Conclusion
In this paper, a feedback controller, which asymptotically stabilizes the nonlinear plant, is designed directly from data. Meanwhile, the estimate of DOA for the closed-loop is enlarged. Because our method just uses data directly, complexity in building the model and modeling error are avoided. From Lemma 2, we know that a state feedback controller asymptotically stabilizes the plant if it belongs to an open subset of space ( denotes the control input and denotes the state). In this
References (17)
- et al.
An iterative optimization approach to design of control Lyapunov function
J. Process Control
(2012) - et al.
Local stability analysis using simulations and sum-of-squares programming
Automatica
(2008) - et al.
Direct data-driven recursive controller unfalsification with analytic update
Automatica
(2007) - et al.
Notes on data-driven system approaches
Acta Automat. Sinica
(2009) - et al.
Nonlinear Dynamical Systems and Control: A Lyapunov-Based Approach
(2008) - et al.
Stability and Stabilization of Nonlinear Systems
(2011) Control-Lyapunov function
- et al.
Help on SOS [ask the experts]
IEEE Control Syst.
(2010)
Cited by (17)
Research on improved partial format MFAC greenhouse temperature control method based on low energy consumption optimization
2024, Computers and Electronics in AgricultureData-driven approximate Q-learning stabilization with optimality error bound analysis
2019, AutomaticaCitation Excerpt :And the optimality error bound of the AQL closed-loop is also analyzed. The proof of Lemma 1 is consistent with that of Lemma 2 in Li and Hou (2014). For convenience and clearness of analyzing the optimality error bound, the Q-learning operator is defined in Definition 1 and its properties are given in Theorem 1.
Distributed adaptive dynamic programming for data-driven optimal control
2018, Systems and Control LettersCitation Excerpt :However, the stability of online designed controllers is difficult to predict and requires restrictive assumptions. This shortcoming is overcome by offline approaches (e.g. [12–14]). Combined offline–online approaches (e.g. [15]) have been developed to incorporate both closed-loop stability and performance improvement with online data.
Data-driven approximate value iteration with optimality error bound analysis
2017, AutomaticaCitation Excerpt :In this study, we present the most significant theoretical analysis result of the optimal control method proposed in Li et al. (2014). The proof of Lemma 1 is consistent with that of Lemma 2 in Li and Hou (2014). The proof of Lemma 4 is presented in Appendix B.
Online Adaptive Optimal Control Algorithm of Partial Unknown System with Adding Experience Replay and Safety Check
2022, Lecture Notes in Electrical Engineering
- ☆
This research was supported by the State Key Program (No. 60834001) and the Major Program of International Cooperation and Exchanges (No. 61120106009) of National Natural Science Foundation of China.