Approximate optimal hybrid control synthesis by classification-based derivative-free optimization
Article No.: 7, Pages 1 - 11
Abstract
Hybrid systems are widely used in safety-critical areas. Hybrid optimal control synthesis, which aims to generate an optimal sequence of control inputs for a given task, is one of the most important problems in the field. The classical Gradient-based methods are efficient but they require the system under control should be differentiable. Sampling-based methods have no such limitations, but the ability of existing ones to solve complex control missions is restricted.
In this paper, we propose a practical and efficient method to solve a general class of hybrid optimal control problems. Basically, we transform the control synthesis problem into a derivative-free optimization (DFO) problem. Then, we adapt a start-of-art classification-based DFO method to solve the optimization problems based on sampled variables efficiently. Furthermore, for complex state space, which is difficult to solve, we present a piecewise control synthesis method to make a tradeoff between optimality and efficiency by generating feasible and piecewise optimal control inputs instead. The empirical results on two complex real-world hybrid systems: a vehicle and a quadcopter drone system, demonstrate that our method outperforms existing methods significantly.
References
[1]
Usman Ali and Magnus Egerstedt. 2018. Hybrid optimal control under mode switching constraints with applications to pesticide scheduling. ACM Transactions on Cyber-Physical Systems 2, 1 (2018), 2.
[2]
Rajeev Alur, Costas Courcoubetis, Thomas A Henzinger, and Pei-Hsin Ho. 1992. Hybrid automata: An algorithmic approach to the specification and verification of hybrid systems. In Hybrid systems. Springer, 209--229.
[3]
Henrik Axelsson, Y Wardi, and Magnus Egerstedt. 2008. Gradient descent approach to optimal mode scheduling in hybrid dynamical systems. Journal of Optimization Theory and Applications (2008), 167--186.
[4]
Sorin C Bengea and Raymond A DeCarlo. 2005. Optimal control of switching systems. automatica 41, 1 (2005), 11--27.
[5]
Michael S Branicky. 2005. Introduction to hybrid systems. In Handbook of networked and embedded control systems. Springer, 91--116.
[6]
Michael S Branicky, Vivek S Borkar, and Sanjoy K Mitter. 1998. A unified framework for hybrid control: Model and optimal control theory. IEEE transactions on automatic control 43, 1 (1998), 31--45.
[7]
Michael S Branicky and Michael M Curtiss. 2002. Nonlinear and hybrid control via RRTs. In Proc. Intl. Symp. on Mathematical Theory of Networks and Systems, Vol. 750. Citeseer.
[8]
Michael S Branicky, Michael M Curtiss, Joshua A Levine, and Stuart B Morgan. 2003. RRTs for nonlinear, discrete, and hybrid planning and control. In 42nd IEEE International Conference on Decision and Control (IEEE Cat. No. 03CH37475), Vol. 1. IEEE, 657--663.
[9]
Michael S Branicky and Sanjoy K Mitter. 1995. Algorithms for optimal hybrid control. In Proceedings of CDC 1995. IEEE, 2661--2666.
[10]
Eric Brochu, Vlad M Cora, and Nando De Freitas. 2010. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv (2010).
[11]
Timothy M Caldwell and Todd D Murphey. 2010. An adjoint method for second-order switching time optimization. In Proceedings of CDC 2010. IEEE, 2155--2162.
[12]
Xin Chen, Erika Ábrahám, and Sriram Sankaranarayanan. 2013. Flow*: An analyzer for non-linear hybrid systems. In International Conference on Computer Aided Verification. Springer, 258--263.
[13]
Pieter-Tjerk de Boer, Dirk P. Kroese, Shie Mannor, and Reuven Y. Rubinstein. 2005. A Tutorial on the Cross-Entropy Method. Annals OR 134, 1 (2005), 19--67.
[14]
Jerry Ding and Claire J Tomlin. 2010. Robust reach-avoid controller synthesis for switched nonlinear systems. In 49th IEEE Conference on Decision and Control (CDC). IEEE, 6481--6486.
[15]
X-C Ding, Yorai Wardi, and Magnus Egerstedt. 2009. On-line optimization of switched-mode dynamical systems. IEEE Trans. Automat. Control 54, 9 (2009), 2266--2271.
[16]
Magnus Egerstedt, Yorai Wardi, and Henrik Axelsson. 2006. Transition-time optimization for switched-mode dynamical systems. IEEE Trans. Automat. Control 51, 1 (2006), 110--115.
[17]
Magnus Egerstedt, Yorai Wardi, and Florent Delmotte. 2003. Optimal control of switching times in switched dynamical systems. In Proceedings of CDC 2003, Vol. 3. IEEE, 2138--2143.
[18]
Samira S Farahani, Vasumathi Raman, and Richard M Murray. 2015. Robust model predictive control for signal temporal logic synthesis. IFAC-PapersOnLine 48, 27 (2015), 323--328.
[19]
Goran Frehse, Colas Le Guernic, Alexandre Donzé, Scott Cotton, Rajarshi Ray, Olivier Lebeltel, Rodolfo Ripado, Antoine Girard, Thao Dang, and Oded Maler. 2011. SpaceEx: Scalable verification of hybrid systems. In International Conference on Computer Aided Verification. Springer, 379--395.
[20]
Rafal Goebel, Ricardo G Sanfelice, and Andrew R Teel. 2009. Hybrid dynamical systems. IEEE control systems magazine 29, 2 (2009), 28--93.
[21]
David E. Goldberg. 1989. Genetic Algorithms in Search Optimization and Machine Learning. Addison-Wesley.
[22]
Humberto Gonzalez, Ram Vasudevan, and Maryam Kamgarpour. 2010. A descent algorithm for the optimal control of constrained nonlinear switched dynamical systems. In Proceedings of HSCC 2010. ACM, 51--60.
[23]
Humberto Gonzalez, Ram Vasudevan, Maryam Kamgarpour, S Shankar Sastry, Ruzena Bajcsy, and Claire Tomlin. 2010. A numerical method for the optimal control of switched systems. In Proceedings of CDC 2010. IEEE, 7519--7526.
[24]
Sven Hedlund and Anders Rantzer. 2002. Convex dynamic programming for hybrid systems. IEEE Trans. Automat. Control (2002), 1536--1540.
[25]
Thomas A Henzinger. 2000. The theory of hybrid automata. In Verification of Digital and Hybrid Systems. 265--292.
[26]
Thomas A Henzinger, Pei-Hsin Ho, and Howard Wong-Toi. 1997. HyTech: A model checker for hybrid systems. International Journal on Software Tools for Technology Transfer 1, 1--2 (1997), 110--122.
[27]
Elliot R Johnson and Todd D Murphey. 2009. Second order switching time optimization for time-varying nonlinear systems. In Proceedings of CDC 2009. IEEE, 5281--5286.
[28]
Lydia E Kavraki, Mihail N Kolountzakis, and J-C Latombe. 1998. Analysis of probabilistic roadmaps for path planning. IEEE Transactions on Robotics and Automation 14, 1 (1998), 166--171.
[29]
Lydia E Kavraki, Petr Svestka, J-C Latombe, and Mark H Overmars. 1996. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE transactions on Robotics and Automation 12, 4 (1996), 566--580.
[30]
Steven M LaValle and James J Kuffner. 2001. Rapidly-exploring random trees: Progress and prospects. Algorithmic and computational robotics: new directions 5 (2001), 293--308.
[31]
Jinjin Liu, Kanjian Zhang, Changyin Sun, and Haikun Wei. 2013. Second order transition-time optimization for switched dynamical systems. In Proceedings of the 32nd Chinese Control Conference. IEEE, 2382--2386.
[32]
Frank Neumann and Ingo Wegener. 2007. Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theoretical Computer Science 378, 1 (2007), 32--40.
[33]
Petter Nilsson and Necmiye Ozay. 2014. Incremental synthesis of switching protocols via abstraction refinement. In 53rd IEEE Conference on Decision and Control. IEEE, 6246--6253.
[34]
Héctor J Sussmann. 2000. Set-valued differentials and the hybrid maximum principle. In Proceedings of CDC 2000. IEEE, 558--563.
[35]
Cristian-Ioan Vasile, Vasumathi Raman, and Sertac Karaman. 2017. Sampling-based synthesis of maximally-satisfying controllers for temporal logic specifications. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 3840--3847.
[36]
Jiawan Wang, Lei Bu, Shaopeng Xing, and Xuandong Li. 2021. Path-Oriented, Derivative-Free Approach for Safety Falsification of Nonlinear and Nondeterministic CPS. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2021).
[37]
Xuping Xu and Panos J Antsaklis. 2000. Optimal control of switched systems: new results and open problems. In Proceedings of ACC 2000, Vol. 4. IEEE, 2683--2687.
[38]
Xuping Xu and Panos J Antsaklis. 2003. Results and perspectives on computational methods for optimal control of switched systems. In Proceedings of HSCC 2003. Springer, 540--555.
[39]
Yang Yu and Hong Qian. 2014. The sampling-and-learning framework: A statistical view of evolutionary algorithms. In 2014 IEEE Congress on Evolutionary Computation (CEC). IEEE, 149--158.
[40]
Yang Yu and Hong Qian. 2016. Derivative-free optimization via classification. In Proceedings of AAAI 2016.
[41]
Pengcheng Zhao, Shankar Mohan, and Ram Vasudevan. 2017. Optimal control for nonlinear hybrid systems via convex relaxations. arXiv preprint arXiv:1702.04310 (2017).
Index Terms
- Approximate optimal hybrid control synthesis by classification-based derivative-free optimization
Recommendations
A derivative-free comirror algorithm for convex optimization
We consider the minimization of a nonsmooth convex function over a compact convex set subject to a nonsmooth convex constraint. We work in the setting of derivative-free optimization DFO, assuming that the objective and constraint functions are ...
Variational formulation and optimal control of hybrid lagrangian systems
HSCC '11: Proceedings of the 14th international conference on Hybrid systems: computation and controlIn this contribution, we introduce an optimal control problem for hybrid Lagrangian control systems. The dynamics of these systems and their discrete approximations are derived via a hybrid variational principle which is based on the Lagrange-d'Alembert ...
Efficient Solution of Optimal Control Problems Using Hybrid Systems
We consider the synthesis of optimal controls for continuous feedback systems by recasting the problem to a hybrid optimal control problem: synthesize optimal enabling conditions for switching between locations in which the control is constant. An ...
Comments
Information & Contributors
Information
Published In
May 2021
300 pages
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 19 May 2021
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Science Foundation of China
- The Leading-edge Technology Program of Jiangsu Natural Science Foundation
Conference
HSCC '21
Sponsor:
HSCC '21: 24th ACM International Conference on Hybrid Systems: Computation and Control
May 19 - 21, 2021
Tennessee, Nashville
Acceptance Rates
HSCC '21 Paper Acceptance Rate 27 of 77 submissions, 35%;
Overall Acceptance Rate 153 of 373 submissions, 41%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 116Total Downloads
- Downloads (Last 12 months)10
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in