Skip to main content
SpringerLink
Log in

Learning-based robust model predictive control with data-driven Koopman operators

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

This paper presents a data-driven control strategy for nonlinear dynamical systems, which fully exploits the advantages of the Koopman operator in globally linearizing nonlinear dynamical systems. We first generalize the Koopman operator framework to the controlled nonlinear systems, enabling comprehensive linear analysis and control methods to be valid for nonlinear systems. When extracting the Koopman operator approximation from data, model uncertainty always arises due to the variation of the data-driven setting. We next present a hierarchical neural network (HNN) approach to approximate the finite-dimensional Koopman operator representations and construct multiple Koopman-based lifted models for original controlled nonlinear systems in a polytope set construction. Based on that, a robust Koopman-based model predictive control (rKMPC) approach considering state and input constraints is constructed to realize the control of the original nonlinear systems. In particular, we extend the proposed rKMPC framework to a Koopman operator-based reduced-order model, thereby achieving the nonlinear control using only a few given inputs. Finally, several numerical examples and a physical experiment are provided to demonstrate the effectiveness of the proposed data-driven control approach, and numerical comparisons are carried out with existing Koopman-based control methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Abbreviations

Notation:

Expression

k :

Discrete time index.

\(x_k\) :

Original state at the \(k\text {th}\) instant.

\(u_k\) :

Control input at the \(k\text {th}\) instant.

f :

Discrete dynamical map.

\({\mathcal {M}}\) :

Smooth manifold.

\({\mathcal {H}}\) :

Hilbert space.

\({\mathbb {R}}\) :

Real number space.

\({\mathcal {K}}\) :

Koopman operator.

g :

Observable function.

\(\phi\) :

The vector-valued function.

\(\psi _i\) :

The ith lifting function contained in \(\phi\).

\(z_k\) :

Lifted state at the \(k\text {th}\) instant.

A :

State matrix of the lifted linear system.

B :

Control matrix of the lifted linear system.

C :

Output matrix of the lifted linear system.

\({\tilde{A}}_i\) :

The ith vertex matrix corresponding to the matrix A.

\({\tilde{B}}_i\) :

The ith vertex matrix corresponding to the matrix B.

\(N(x\vert \theta )\) :

Representation of the prediction module, where \(\theta\) denotes the network parameters.

\(L(\theta )\) :

Loss function.

O :

The output of each DNN in predictor module and tag module.

E :

The output of each DNN in deviation module.

\(z_{k+1}\) :

Lifted state at the \((k+1)\)th instant, the output of the linear module, initial lifted state estimation.

\({\bar{z}}_{k+1}\) :

The output of the tag module, lifted state tag.

\(\Delta {z}_k\) :

The output of the deviation module, deviation estimation.

\(\Delta {r}_k\) :

Deviation tag.

\({\hat{z}}_{k+1}\) :

Ultimate lifted state estimation.

X :

State constraint set.

U :

Control constraint set.

\(N_k\) :

Prediction steps.

Q :

Cost matrix for the state

P :

Cost matrix for the terminal state.

R :

Cost matrix for the control input.

\(u_k^*\) :

The optimal control rule at the kth instant.

\(\gamma _1\) :

Training episodes.

\(\gamma _2\) :

The number of iterations.

n :

Dimension of the original state.

m :

Dimension of the control input.

N :

Dimension of the lifted state.

M :

Latent dimension of each hidden layer.

T :

Time domain of the training sample (Training time domian), continuous time index.

\(N_d\) :

The number of collected datasets.

h :

The number of vertices of a polytope set.

\(n_c\) :

The number of constant inputs.

\(\text {diag}(\cdots )\) :

A diagnal matrix with elements \(\cdot\).

\(\text {bdiag}(\cdots )\) :

A block diagnal matrix with elements \(\cdot\).

References

  1. Kamb M, Kaiser E, Brunton SL, Kutz JN (2020) Time-delay observables for Koopman: theory and applications. SIAM J Appl Dyn Syst 19(2):886–917. https://doi.org/10.1137/18M1216572

    Article  MathSciNet  MATH  Google Scholar 

  2. Koopman BO (1931) Hamiltonian systems and transformation in Hilbert space. Proc Natl Acad Sci 17(5):315–318. https://doi.org/10.1073/pnas.17.5.315

    Article  MATH  Google Scholar 

  3. Koopman BO, Neumann JV (1932) Dynamical systems of continuous spectra. Proc Natl Acad Sci USA 18(3):255–263. https://doi.org/10.1073/pnas.18.3.255

    Article  MATH  Google Scholar 

  4. Brunton SL, Brunton BW, Proctor JL, Nathan KJ, Kestler HA (2016) Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control. PLoS ONE 11(2):0150171. https://doi.org/10.1371/journal.pone.0150171

    Article  Google Scholar 

  5. Arbabi H, Korda M, Mezic I (2018) A data-driven Koopman model predictive control framework for nonlinear partial differential equations. In: 2018 IEEE Conference on Decision and Control (CDC), pp. 6409–6414. https://doi.org/10.1109/CDC.2018.8619720

  6. Zhang X, Pan W, Scattolini R, Yu S, Xu X (2022) Robust tube-based model predictive control with Koopman operators. Automatica 137:110114. https://doi.org/10.1016/j.automatica.2021.110114

    Article  MathSciNet  MATH  Google Scholar 

  7. Lusch B, Kutz J, Brunton S (2018) Deep learning for universal linear embeddings of nonlinear dynamics. Nat Commun 9:4950. https://doi.org/10.1038/s41467-018-07210-0

    Article  Google Scholar 

  8. Schmid PJ, Sesterhenn J (2010) Dynamic mode decomposition of numerical and experimental data. J Fluid Mech 656(10):5–28. https://doi.org/10.1017/S0022112010001217

    Article  MathSciNet  MATH  Google Scholar 

  9. Rowley C, Mezic I, Bagheri S, Schlatter P, Henningson DS (2009) Spectral analysis of nonlinear flows. J Fluid Mech 641:115–127. https://doi.org/10.1017/S0022112009992059

    Article  MathSciNet  MATH  Google Scholar 

  10. Kutz JN, Brunton SL, Brunton BW, Proctor JL (2016) Dynamic mode decomposition: data-driven modeling of complex systems. SIAM Press, Berlin

    Book  MATH  Google Scholar 

  11. Williams MO, Kevrekidis IG, Rowley CW (2015) A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. J Nonlinear Sci 25:1307–1346. https://doi.org/10.1007/s00332-015-9258-5

    Article  MathSciNet  MATH  Google Scholar 

  12. Noé F, Nüske F (2013) A variational approach to modeling slow processes in stochastic dynamical systems. Multiscale Model Simul 11(2):635–655. https://doi.org/10.1137/110858616

    Article  MathSciNet  MATH  Google Scholar 

  13. Kevrekidis I, Rowley C, Williams M (2016) A kernel-based method for data-driven Koopman spectral analysis. J Comput Dyn 2:247–265. https://doi.org/10.3934/jcd.2015005

    Article  MathSciNet  MATH  Google Scholar 

  14. Yeung E, Kundu S, Hodas N (2019) Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. In: 2019 American Control Conference (ACC), pp. 4832–4839 . https://doi.org/10.23919/ACC.2019.8815339

  15. Otto SE, Rowley CW (2019) Linearly recurrent autoencoder networks for learning dynamics. SIAM J Appl Dyn Syst 18(1):558–593. https://doi.org/10.1137/18M1177846

    Article  MathSciNet  MATH  Google Scholar 

  16. Li S, Yang Y (2021) Data-driven identification of nonlinear normal modes via physics-integrated deep learning. Nonlinear Dyn 106:3231–3246. https://doi.org/10.1007/s11071-021-06931-0

    Article  Google Scholar 

  17. N N, Chakraborty S (2022) Koopman operator for time-dependent reliability analysis. Eprint Arxiv . https://doi.org/10.48550/ARXIV.2203.02658

  18. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press. http://www.deeplearningbook.org

  19. Wang X, Zhao Y, Pourpanah F (2020) Recent advances in deep learning. Int J Mach Learn Cybern 11:747–750. https://doi.org/10.1007/s13042-020-01096-5

    Article  Google Scholar 

  20. Balcazar R, Rubio JDJ, Orozco E, Andres Cordova D, Ochoa G, Garcia E, Pacheco J, Gutierrez GJ, Mujica-Vargas D, Aguilar-Ibanez C (2022) The regulation of an electric oven and an inverted pendulum. Symmetry 14(4):759. https://doi.org/10.3390/sym14040759

    Article  Google Scholar 

  21. Rubio JDJ, Orozco E, Cordova DA, Islas MA, Pacheco J, Gutierrez GJ, Zacarias A, Soriano LA, Meda-Campana JA, Mujica-Vargas D (2022) Modified linear technique for the controllability and observability of robotic arms. IEEE Access 10:3366–3377. https://doi.org/10.1109/ACCESS.2021.3140160

    Article  Google Scholar 

  22. Villasenor Rios CA, Luviano-Juarez A, Lozada-Castillo NB, Carvajal-Gamez BE, Mujica-Vargas D, Gutierrez-Frias O (2022) Flatness-based active disturbance rejection control for a pvtol aircraft system with an inverted pendular load. Machines 10(7):595. https://doi.org/10.3390/machines10070595

    Article  Google Scholar 

  23. Soriano LA, Rubio JDJ, Orozco E, Cordova DA, Ochoa G, Balcazar R, Cruz DR, Meda-Campana JA, Zacarias A, Gutierrez GJ (2021) Optimization of sliding mode control to save energy in a scara robot. Mathematics 9(24):3160. https://doi.org/10.3390/math9243160

    Article  Google Scholar 

  24. Soriano LA, Zamora E, Vazquez-Nicolas JM, Hernandez G, Barraza Madrigal JA, Balderas D (2020) PD control compensation based on a cascade neural network applied to a robot manipulator. Front Neurorobot 14:2. https://doi.org/10.3389/fnbot.2020.577749

    Article  Google Scholar 

  25. Silva-Ortigoza R, Hernandez-Marquez E, Roldan-Caballero A, Tavera-Mosqueda S, Marciano-Melchor M, Garcia-Sanchez JR, Hernandez-Guzman VM, Silva-Ortigoza G (2021) Sensorless tracking control for a full-bridge buck inverter-DC motor system: passivity and flatness-based design. IEEE Access 9:132191–132204. https://doi.org/10.1109/ACCESS.2021.3112575

    Article  Google Scholar 

  26. Proctor JL, Brunton SL, Kutz JN (2014) Dynamic mode decomposition with control. SIAM J Appl Dyn Syst 15(1):1101–1109. https://doi.org/10.1137/15M1013857

    Article  MathSciNet  MATH  Google Scholar 

  27. Korda M, Mezic I (2016) Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control. Automatica 93:149–160. https://doi.org/10.1016/j.automatica.2018.03.046

    Article  MathSciNet  MATH  Google Scholar 

  28. Korda M, Mezic I (2020) Optimal construction of Koopman eigenfunctions for prediction and control. IEEE Trans Autom Control 65(12):5114–5129. https://doi.org/10.1109/TAC.2020.2978039

    Article  MathSciNet  MATH  Google Scholar 

  29. Abraham I, Murphey TD (2019) Active learning of dynamics for data-driven control using Koopman operators. IEEE Trans Rob 35(5):1071–1083. https://doi.org/10.1109/TRO.2019.2923880

    Article  Google Scholar 

  30. Uchida D, Yamashita A, Asama H (2021) Data-driven Koopman controller synthesis based on the extended \(H_2\) norm characterization. IEEE Control Syst Lett 5(5):1795–1800. https://doi.org/10.1109/LCSYS.2020.3042827

    Article  MathSciNet  Google Scholar 

  31. Peitz S, Klus S (2019) Koopman operator-based model reduction for switched-system control of PDEs. Automatica 106:184–191. https://doi.org/10.1016/j.automatica.2019.05.016

    Article  MathSciNet  MATH  Google Scholar 

  32. Proctor JL, Brunton SL, Kutz JN (2016) Generalizing Koopman theory to allow for inputs and control. SIAM J Appl Dyn Syst 17:909–930. https://doi.org/10.1137/16M1062296

    Article  MathSciNet  MATH  Google Scholar 

  33. Williams MO, Hemati MS, Dawson STM, Kevrekidis IG, Rowley CW (2016) Extending data-driven Koopman analysis to actuated systems. IFAC-Papers OnLine 49(18):704–709. https://doi.org/10.1016/j.ifacol.2016.10.248

    Article  Google Scholar 

  34. Han Y, Hao W, Vaidya U (2020) Deep learning of Koopman representation for control. In: 2020 59th IEEE Conference on Decision and Control (CDC), pp. 1890–1895. https://doi.org/10.1109/CDC42340.2020.9304238

  35. Shi H, Meng MQH (2022) Deep Koopman operator with control for nonlinear systems. IEEE Robot Autom Lett 7(3):7700–7707. https://doi.org/10.1109/LRA.2022.3184036

    Article  MathSciNet  Google Scholar 

  36. Bao J, Ye M (2016) Scale invariant constrained deep network for head pose estimation. Adv Model Anal B 59(1):113–130

    Google Scholar 

  37. Takeishi N, Kawahara Y, Yairi T (2017) Learning Koopman invariant subspaces for dynamic mode decomposition. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 1130–1140

  38. Ferreau HJ, Kirches C, Potschka A, Bock HG, Diehl M (2014) qpoases: a parametric active-set algorithm for quadratic programming. Math Progr Comput 6:327–363. https://doi.org/10.1007/s12532-014-0071-1

    Article  MathSciNet  MATH  Google Scholar 

  39. Mayne DQ, Rawlings JB, Rao CV, Scokaert POM (2000) Constrained model predictive control: stability and optimality. Automatica 36(6):789–814. https://doi.org/10.1016/S0005-1098(99)00214-9

    Article  MathSciNet  MATH  Google Scholar 

  40. Limon D, Alamo T, Camacho EF (2003) Stable constrained MPC without terminal constraint. In: Proceedings of the 2003 American Control Conference 6:4893–4898. https://doi.org/10.1109/ACC.2003.1242498

  41. Rawlings J, Mayne DQ, Diehl M (2017) Model predictive control: theory, computation, and design. Madison, WI

    Google Scholar 

  42. Camacho EF, Bordons C (2007) Model predictive control, 2nd edn. Springer, London

    Book  MATH  Google Scholar 

  43. Mayne DQ, De Dona JA, Goodwin GC (2000) Improved stabilising conditions for model predictive control. In: Proceedings of the 39th IEEE Conference on Decision and Control 1:172–1771. https://doi.org/10.1109/CDC.2000.912752

  44. Mayne DQ (2001) Control of constrained dynamic systems. Eur J Control 7(2):87–99. https://doi.org/10.3166/ejc.7.87-99

    Article  MATH  Google Scholar 

  45. Gilbert EG, Tan KT (1991) Linear systems with state and control constraints: the theory and application of maximal output admissible sets. IEEE Trans Autom Control 36(9):1008–1020. https://doi.org/10.1109/9.83532

    Article  MathSciNet  MATH  Google Scholar 

  46. Scokaert POM, Rawlings JB (1998) Constrained linear quadratic regulation. IEEE Trans Autom Control 43(8):1163–1169. https://doi.org/10.1109/9.704994

    Article  MathSciNet  MATH  Google Scholar 

  47. Sznaier M, Damborg MJ (1987) Suboptimal control of linear systems with state and control inequality constraints. In: 26th IEEE Conference on Decision and Control 26:761–762. https://doi.org/10.1109/CDC.1987.272491

  48. Biegler LT (2000) Efficient solution of dynamic optimization and NMPC problems. In: Allgöwer F, Zheng A (eds) Nonlinear model predictive control 26:219–243. https://doi.org/10.1007/978-3-0348-8407-5_13

  49. Leith DJ, Leithead WE (2000) Survey of gain-scheduling analysis and design. Int J Control 73(11):1001–1025. https://doi.org/10.1080/002071700411304

    Article  MathSciNet  MATH  Google Scholar 

  50. Paszke A, Gross S, Chintala S, Chanan G, Yang E, Devito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch. In: 31st Conference on Neural Information Processing Systems

  51. Kingma D, Ba J (2014) Adam: A method for stochastic optimization. In: International Conference on Learning Representations

  52. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1026–1034 . https://doi.org/10.1109/ICCV.2015.123

  53. Van der Pol B (1920) A theory of the amplitude of free and forced triode vibrations. Radio Rev 1:701–710

    Google Scholar 

  54. Guckenheimer J, Holmes P (1983) Nonlinear oscillations, dynamical systems, and bifurcations of vector fields. Appl Math Sci 42:2. https://doi.org/10.1007/978-1-4612-1140-2

    Article  MathSciNet  MATH  Google Scholar 

  55. Mayne DQ, Kerrigan EC, van Wyk EJ, Falugi P (2011) Tube-based robust nonlinear model predictive control. Int J Robust Nonlinear Control 21:1341–1353. https://doi.org/10.1002/rnc.1758

    Article  MathSciNet  MATH  Google Scholar 

  56. Klus S, Nüske F, Peitz S, Niemann JH, Schütte C (2020) Data-driven approximation of the Koopman generator: Model reduction, system identification, and control. Phys D 406:132416. https://doi.org/10.1016/j.physd.2020.132416

    Article  MathSciNet  MATH  Google Scholar 

  57. Kaiser E, Kutz JN, Brunton SL (2021) Data-driven discovery of Koopman eigenfunctions for control. Mach Learn Sci Technol 2(3):035023. https://doi.org/10.1088/2632-2153/abf0f5

    Article  Google Scholar 

  58. Lu B, Fang Y, Ning S (2017) Sliding mode control for underactuated overhead cranes suffering from both matched and unmatched disturbances. Mechatronics 47:116–125. https://doi.org/10.1016/j.mechatronics.2017.09.006

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Natural Science Foundation of Jiangsu Province (BK20201340) and 333 High-level Talents Training Project of Jiangsu Province.

Author information

Authors and Affiliations

  1. Key Laboratory of Advanced Process Control for Light Industry (Ministry of Education), Jiangnan University, Wuxi, 214122, China

    Meixi Wang, Xuyang Lou & Baotong Cui

Authors
  1. Meixi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuyang Lou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baotong Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuyang Lou.

Ethics declarations

Conflict of interest

All authors disclosed no relevant relationships.

Availability of data and materials

The processed data required to reproduce these findings cannot be shared at this time as the data also forms part of an ongoing study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, M., Lou, X. & Cui, B. Learning-based robust model predictive control with data-driven Koopman operators. Int. J. Mach. Learn. & Cyber. 14, 3295–3321 (2023). https://doi.org/10.1007/s13042-023-01834-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01834-5

Keywords

Search

Navigation