Elsevier

Automatica

Volume 36, Issue 5, May 2000, Pages 641-657
Automatica

Model-based iterative learning control with a quadratic criterion for time-varying linear systems

https://doi.org/10.1016/S0005-1098(99)00194-6Get rights and content

Abstract

In this paper, iterative learning control (ILC) based on a quadratic performance criterion is revisited and generalized for time-varying linear constrained systems with deterministic, stochastic disturbances and noises. The main intended area of application for this generalized method is chemical process control, where excessive input movements are undesirable and many process variables are subject to hard constraints. It is shown that, within the framework of the quadratic-criterion-based ILC (Q-ILC), various practical issues such as constraints, disturbances, measurement noises, and model errors can be considered in a rigorous and systematic manner. Algorithms for the deterministic case, the stochastic case, and the case with bounded parameter uncertainties are developed and relevant properties such as the asymptotic convergence are established under some mild assumptions. Numerical examples are provided to demonstrate the performance of the proposed algorithms.

Introduction

Iterative learning control (ILC) was originally proposed in the robotics community (Arimoto, Kawamura & Miyazaki, 1984) as an intelligent teaching mechanism for robot manipulators. The basic idea of ILC is to improve the control signal for the present operation cycle by feeding back the control error in the previous cycle. Even though the mainstream ILC research has thus far been carried out with mechanical systems in mind, chemical and other manufacturing processes could also benefit significantly from it. Batch chemical processes such as the batch reactor, batch distillation, and heat treatment processes for metallic or ceramic products are good examples. Traditionally, operations of these processes have relied exclusively on PID feedback and logic-based controllers. Refinement of input bias signals based on the general concept of ILC can potentially enhance the performance of tracking control systems significantly. Diversification of ILC applications to the above-mentioned problems is already starting to take place, as evidenced by the comprehensive lists of recent ILC papers compiled by Chen (1998).

The classical formulation of ILC design problem has been as follows: Find an update mechanism for the input trajectory of a new cycle based on the information from previous cycles so that the output trajectory converges asymptotically to the desired reference trajectory.

The first-order ILC algorithms update the input trajectory u (defined over the same time interval) in the following way (Moore, 1993):uk+1=uk+Hek.In the above, eydy where y and yd denote the output and output reference trajectories which can be either continuous or discrete signals defined over a finite time interval of [0,T]. The subscript k here represents the batch/cycle index.

In the above, H called “learning filter” is an operator that maps the error signal ek to the input update signal uk+1uk. Within this somewhat restrictive problem setup, the ILC design is reduced to choosing the operator H.

The prevalent approach thus far has been to assume a simple structure for the learning filter H and tune the parameters to achieve the desired learning properties. Examples of this type include D-type (Arimoto et al., 1984), PID-type (Bondi, Casalino & Gambradella, 1988), and their variants. As a straightforward extension of the first-order algorithms, higher-order algorithms have been proposed too (Bien & Huh, 1989). This line of approaches, however, could yield only limited results for general multivariable systems.

Model-based algorithms have also been proposed. However, most algorithms proposed were based on the notion of direct model inversion (Togai & Yamano, 1985; Oh, Bien & Suh, 1988; Lucibello, 1992; Moore, 1993; Lee, Bang & Chang, 1994b; Yamada, Watanabe, Tsuchiya & Kaneko, 1994), that is, H=G−1 where G represents the input–output map of the process. Since G−1 would contain a differentiator(s) (in the continuous-time case), the learning filter based on the model inverse becomes hyper-sensitive to high-frequency components in ek. Since, in most process control applications, smooth manipulation of actuators is at least as important as precise control of outputs, these approaches cannot be used directly.

Furthermore, the zero tracking error objective cannot be satisfied for general nonsquare MIMO processes. Since it is not uncommon for industrial batch processes to render a nonsquare problem for which zero tracking error for all the output variables is impossible, a more general objective appropriate for nonsquare processes is needed.

There are certain additional traits and requirements found in prototypical process control problems that motivate a more general (but perhaps more computationally intensive) approach. First, most process variables are subject to certain constraints that are set by physical or safety considerations. Hence, it is desirable to have algorithms that incorporate the constraint information explicitly into the calculation. Second, dynamics of almost all chemical processes are intrinsically nonlinear, and the nonlinearities become exposed when the processes are operated over a wide range of conditions, as in typical industrial batch operations. For this reason, it is necessary to derive ILC algorithms that can accommodate nonlinear system models, when available. Third, disturbances and noises are integral aspects of most process control problems and must be dealt with in a systematic fashion. Some disturbances, once they occur, tend to repeat themselves in subsequent batches, while others tend to be more specific to a particular batch. Most disturbances exhibit significant time correlation that must be exploited for efficient rejection. Finally, chemical processes have a quite long interval allowed between two adjacent batches and sample times can be chosen relatively large in relation to the total cycle time. These traits should allow us to implement numerically more intensive algorithms, such as those based on mathematical programming techniques.

Some of the aforementioned generalizations have already appeared in the literature. For example, to accommodate the nonsquare MIMO systems, the zero-tracking error requirement has been relaxed to “minimum possible error in the least-squares sense”. This type of approach has been studied by Togai and Yamano (1985) and also by Moore (1993). For the purpose of reducing the noise sensitivity, Tao, Kosut and Aral (1994) proposed a discrete-time ILC algorithm based on the following least-squares objective with an input penalty term:||ek||Q2+||uk||R2min(||e||Q2+||u||R2)ask→∞.A similar objective has also been considered by Sogo and Adachi (1994) but in the continuous-time domain. These algorithms can accommodate nonsquare MIMO systems and mitigate the noise sensitivity by using the input penalty term. However, by adding the quadratic penalty term on the inputs directly, offsets result, i.e., the algorithms fail to attain the minimum achievable error in the limit. In addition, it is unclear how to best trade off the noise sensitivity against the speed of convergence and output offset, using the input weight matrix.

Recently, Amann, Owens and Rogers (1996) and Lee, Kim and Lee (1996) have independently proposed to use the following objective:minuk[Jk={||ek||2Q+||ukuk−1||R2}].Because the input change is penalized instead of the input, the algorithm has an integral action (with respect to the batch index) and achieves the minimum achievable error in the limit. In the unconstrained, deterministic setting, Amann et al. (1996) derived a noncausal input updating lawuk=uk−1+R−1GTQekfrom Jk/∂uk=0, while Lee et al. (1996) obtaineduk=uk−1+(GTQG+R)−1GTQek−1which is indeed a rephrasing of (4) in a pure learning form. Amann et al. (1996) transformed (4) to a causal form by borrowing the idea from the solution of the finite-time quadratic optimal tracking problem. The resulting algorithm is a combination of a state feedback law and a feedforward signal based on the error signal of the previous cycle. In addition to significant reduction in the computational load, the feedback implementation gives some robustness to disturbances and model errors. However, their algorithm is developed entirely in a deterministic setting (without direct references to disturbances) and hence deserves further investigation. In a similar spirit, Lee and Lee (1997) also showed that the Q-ILC algorithm can be implemented as an output feedback algorithm, thus improving the robustness. Their real-time algorithm can be viewed as a combination of the popular model predictive control (Lee, Morari & Garcia, 1994a) and the iterative learning control.

The objective of this paper is to provide a more general and comprehensive framework for quadratic-criterion-based ILCs that is capable of addressing all the issues that were mentioned to be important for process control applications. We focus on the iterative learning control implementation rather than the feedback implementation, keeping in mind the fact that the conversion to the latter type of implementation can always be done in a straightforward manner. We first introduce an error transition model that represents the transition of tracking error trajectories between two adjacent batches. We also discuss how the effects of disturbances of various types can be integrated into the transition model. Based on this model, one-batch-ahead quadratic optimal control algorithms are derived for both the unconstrained and constrained cases. In addition, a robust ILC algorithm that minimizes the worst-case tracking error for the next batch is proposed. For each algorithm, relevant mathematical properties such as the convergence, robustness, and noise sensitivity are investigated.

The rest of the paper is organized as follows: In Section 2, the static gain representation of the dynamic system is introduced and is converted into an error transition model. The ILC design objective is defined based on the model description. In Section 3, the quadratic-criterion-based iterative learning control (Q-ILC) algorithm is derived for the unconstrained case, and the analysis of the relevant properties such as the convergence, noise sensitivity, and robustness follows. A real-time output feedback implementation of the algorithm is discussed and a comprehensive comparison with Amann et al.'s state-feedback/error-feedforward algorithm is made. The constrained Q-ILC algorithm is presented with the convergence proof in Section 4. In Section 5, the robust Q-ILC algorithm is proposed with the convergence proof. Numerical examples are given in Section 6 and conclusions are drawn in Section 7.

Section snippets

Process description and problem statement

We assume that the underlying system is described by the following static map between input and output sequences defined over the time domain of a batch operation of interest:yk=Guk+Gddk+b,whereykT=[ykT(1)ykT(2)⋯ykT(N)],ukT=[ukT(0)ukT(1)⋯ukT(N−1)],dkT=[dkT(1)dkT(2)⋯dkT(N)].In the above, y,u and d represent the output, input and disturbance signals repspectively. Subscript k denotes the batch index. b is a constant vector. Eq. (6) is linear affine and is assumed to be invariant with respect to

Derivation of algorithms

We consider solving the following quadratic subproblem upon the completion of the kth batch to update the input trajectory for the k+1th batch:minΔuk+112{ek+1TQek+1+Δuk+1TRΔuk+1},where Q and R are PD (positive-definite) matrices. We may need an expectation operator for the quadratic cost function depending on whether we are dealing with the deterministic case or the stochastic case. Note the cost function has a penalty term on the input change between two adjacent batches. This term does not

Direct-error-feedback algorithm

In many industrial process control applications, certain restrictions need to be placed on the process variables in order to ensure safe, smooth operations. Commonly, constraints imposed on the inputs (its raw values as well as changes with respect to the time and batch index) and the outputs. These constraints are often expressed in the form of mathematical inequalities.

  • For the raw values of inputs,ulowuk+1uhi.

  • For the rate of input changes with respect to the time index,δulow≤δuk+1≤δuhi,

Derivation of algorithm

In the previous sections, we have not incorporated any model uncertainty information explicitly into the algorithm. In this section, we propose a robust Q-ILC algorithm that guarantees convergence and provides optimal performance for a certain class of model uncertainty.

Let us consider the case where our error update model is given asek+1(θ)=ekG(θ)Δuk+1.In the above, the gain matrix is parameterized in terms of an uncertain vector θ. We discuss the deterministic case only here, but nonzero w

Numerical illustrations

Performance of the proposed Q-ILC algorithms are demonstrated through four numerical examples. In the first three examples, we considered batch systems whose input–output map, G, is derived by sampling the zero-order-hold (ZOH) equivalent of a continuous-time model over [0,40]. Sampling period differs depending on the examples.

Example 1 Unconstrained Q-ILC

It was assumed that G for the true process is derived from the following continuous-time model with sampling period of h=0.25:Gp(s)=0.8(5s+1)(3s+1).Filtered square wave

Conclusions

In this paper, it was argued that the existing ILC algorithms, despite their successes in controlling mechanical systems, are not well-suited for process control applications. Motivated by this, we presented new model-based iterative learning control algorithms that were tailored specifically for this type of application. The algorithms were based on quadratic performance criteria and were designed to consider the issues relevant to process control, such as disturbances, noises, nonlinearities,

Acknowledgements

The first author (JHL) gratefully acknowledges the financial support from the National Science Foundation's Young Investigator Program (USA) under the Grant CTS #9357827. The second author would like to acknowledge LG Yonam Foundation (Korea), the Automation Research Center (Korea) at POSTECH and Korea Science and Engineering Foundation for financial support.

Jay H. Lee was born in Seoul, Korea, in 1965. He obtained his B.S. degree in Chemical Engineering from the University of Washington, Seattle, in 1986, and his Ph.D. degree in Chemical Engineering from California Institute of Technology, Pasadena, in 1991. From 1991 to 1998, he was with the Department of Chemical Engineering at Auburn University, AL, as an Assistant Professor and an Associate Professor. Since 1998, he has been with School of Chemical Engineering at Purdue University, West

References (23)

  • J.H. Lee et al.

    State space interpretation of model predictive control

    Automatica

    (1994)
  • K.S. Lee et al.

    Feedback-assisted iterative learning control based on an inverse process model

    Journal of Process and Control

    (1994)
  • N. Amann et al.

    Iterative learning control for discrete-time system with exponential rate of convergence

    IEE Proceedings Control Theory Applications

    (1996)
  • S. Arimoto et al.

    Bettering operation of robots by learning

    Journal of Robotic Systems

    (1984)
  • Z. Bien et al.

    Higher-order iterative learning control algorithm

    IEE Proceedings Part D

    (1989)
  • P. Bondi et al.

    On the iterative learning control theory for robotic manipulators

    IEEE Journal of Robotics Automation

    (1988)
  • Boyd, S. P., & Barratt, C. H. (1991). Linear controller design–limits of performance. Englewood Cliffs:...
  • Chen, Y. (1998). A bibliographical library for ILC research....
  • Campo, P. J., & Morari, M. (1987). Robust model predictive control. Proceedings of ACC, Minneapolis, MN (pp....
  • Lee, J. H., & Cooley, B. (1997). Stable min–max control of state-space systems with bounded input matrix. Proceedings...
  • K.S. Lee et al.

    Model-based iterative learning control with quadratic criterion for linear batch processes

    Journal of Control Automation Systems Engineering

    (1996)
  • Cited by (426)

    • Markov parameters identification and adaptive iterative learning control for linear discrete-time MIMO systems with higher-order relative degree

      2023, Journal of the Franklin Institute
      Citation Excerpt :

      Though the tracking performance such as the impact of the learning ratio on the convergence rate was not exploited yet in-depth, the paradigmatic modes of both the POILC and the NOILC contributed to intensive optimized ILC themes. The focuses turned to the NOILC for the systems with specifications [23–25], the constraint [26], the frequency-domain analysis [27], the auxiliary optimization [28], the non-lifted technique [29,30], and so on. In recent, by benefiting from the learning-gain argument of the POILC and the input assessment of the NOILC, an innovative optimal iterative learning control scheme has been exploited that optimizes the iteration-time-varying learning gains for a given D-type ILC rule in minimizing a criterion of a quadratic-linear combination of the tracking error and the inputs increment [31].

    View all citing articles on Scopus

    Jay H. Lee was born in Seoul, Korea, in 1965. He obtained his B.S. degree in Chemical Engineering from the University of Washington, Seattle, in 1986, and his Ph.D. degree in Chemical Engineering from California Institute of Technology, Pasadena, in 1991. From 1991 to 1998, he was with the Department of Chemical Engineering at Auburn University, AL, as an Assistant Professor and an Associate Professor. Since 1998, he has been with School of Chemical Engineering at Purdue University, West Lafayette, where he currently holds the rank of Associate Professor. He has held visiting appointments at E. I. Du Pont de Numours, Wilmington, in 1994 and at Seoul National University, Seoul, Korea, in 1997. He was a recipient of the National Science Foundation's Young Investigator Award in 1993. His research interests are in the areas of system identification, robust control, model predictive control and nonlinear estimation.

    Dr. Kwang Soon Lee was born in Korea in 1955. He graduated from the Seoul National University in 1977 with B.S. in Chemical Engineering. He then entered KAIST and obtained Ph.D. in Chemical Engineering in 1983 in the area of process control. Since 1983, he joined the faculty of the Department of Chemical Engineering at Sogang University, Seoul, Korea, where he is currently a Professor and the Department Chair. He held visiting appointments at the University of Waterloo, Canada, in 1986 and at Auburn University, Alabama, in 1995. His research has covered the topics of batch process control, model predictive control, model reduction, and system identification.

    Won Cheol Kim was born in Yeosu, Koea in 1963. He received MSc and PhD degrees in Chemical Engineering from Sogang University, Seoul, Korea, in 1990 and 1997, respectively. He is currently with Conwell Co.,Ltd. His main research interests are iterative learning control, model predictive control, and statistical process control.

    This paper was presented in ’96 AIChE Annual Meeting in Chicago. This paper was recommended for publication in revised form by Associate Editor P.J. Fleming under the direction of Editor S. Skogestad.

    View full text