research-article

Public Access

Reachability analysis for neural feedback systems using regressive polynomial rule inference

Authors:

Souradeep Dutta,

Sriram SankaranarayananAuthors Info & Claims

HSCC '19: Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control

Pages 157 - 168

https://doi.org/10.1145/3302504.3311807

Published: 16 April 2019 Publication History

Abstract

We present an approach to construct reachable set overapproximations for continuous-time dynamical systems controlled using neural network feedback systems. Feedforward deep neural networks are now widely used as a means for learning control laws through techniques such as reinforcement learning and data-driven predictive control. However, the learning algorithms for these networks do not guarantee correctness properties on the resulting closed-loop systems. Our approach seeks to construct overapproximate reachable sets by integrating a Taylor model-based flowpipe construction scheme for continuous differential equations with an approach that replaces the neural network feedback law for a small subset of inputs by a polynomial mapping. We generate the polynomial mapping using regression from input-output samples. To ensure soundness, we rigorously quantify the gap between the output of the network and that of the polynomial model. We demonstrate the effectiveness of our approach over a suite of benchmark examples ranging from 2 to 17 state variables, comparing our approach with alternative ideas based on range analysis.

References

[1]

Abadi, Martín et al. 2016. TensorFlow: A System for Large-scale Machine Learning. In Proc. OSDI'16. USENIX, 265--283.

Digital Library

[2]

M. Althoff. 2015. An Introduction to CORA 2015. In Proc. of ARCH'15 (EPiC Series in Computer Science), Vol. 34. EasyChair, 120--151.

[3]

S. Bak and M. Caccamo. 2013. Computing Reachability for Nonlinear Systems with HyCreate. In Demo and Poster Session in HSCC'13.

[4]

S. Bak and P. S. Duggirala. 2017. HyLAA: A Tool for Computing Simulation-Equivalent Reachability for Linear Systems. In Proc. of HSCC'17. ACM, 173--178.

Digital Library

[5]

Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos, Dimitrios Vytiniotis, Aditya Nori, and Antonio Criminisi. 2016. Measuring neural net robustness with constraints. In Advances in Neural Information Processing Systems. 2613--2621.

Digital Library

[6]

M. Berz. 1999. Modern Map Methods in Particle Beam Physics. Advances in Imaging and Electron Physics, Vol. 108. Academic Press.

[7]

M. Berz and K. Makino. 1998. Verified Integration of ODEs and Flows Using Differential Algebraic Methods on High-Order Taylor Models. Reliable Computing 4 (1998), 361--369. Issue 4.

[8]

X. Chen. 2015. Reachability Analysis of Non-Linear Hybrid Systems Using Taylor Models. Ph.D. Dissertation. RWTH Aachen University.

[9]

X. Chen, E. Ábrahám, and S. Sankaranarayanan. 2012. Taylor Model Flowpipe Construction for Non-linear Hybrid Systems. In Proc. of RTSS'12. IEEE Computer Society, 183--192.

Digital Library

[10]

X. Chen, E. Ábrahám, and S. Sankaranarayanan. 2013. Flow*: An Analyzer for Non-linear Hybrid Systems. In Proc. of CAV'13 (LNCS), Vol. 8044. Springer, 258--263.

[11]

X. Chen and S. Sankaranarayanan. 2016. Decomposed Reachability Analysis for Nonlinear Systems. In 2016 IEEE Real-Time Systems Symposium (RTSS). IEEE Press, 13--24.

[12]

Antonio Eduardo Carrilho da Cunha. 2015. Benchmark: Quadrotor Attitude Control. In Proc. of ARCH 2015 (EPiC Series in Computing), Vol. 34. EasyChair, 57--72.

[13]

Luiz H. de Figueiredo and Jorge Stolfi. 1997. Self-Validated Numerical Methods and Applications. In Brazilian Mathematics Colloquium monograph. IMPA, Rio de Janeiro, Brazil. Cf. http://www.ic.unicamp.br/~stolfi/EXPORT/papers/by-tag/fig-sto-97-iaaa.ps.gz.

[14]

P. S. Duggirala, S. Mitra, M. Viswanathan, and M. Potok. 2015. C2E2: A Verification Tool for Stateflow Models. In Proc. of TACAS'15 (LNCS), Vol. 9035. Springer, 68--82.

Digital Library

[15]

Souradeep Dutta, Susmit Jha, Sriram Sankaranarayanan, and Ashish Tiwari. 2018. Learning and verification of feedback control systems using feedforward neural networks. IFAC-PapersOnLine 51, 16 (2018), 151--156.

[16]

Souradeep Dutta, Susmit Jha, Sriram Sankaranarayanan, and Ashish Tiwari. 2018. Output Range Analysis for Deep Feedforward Neural Networks. In NASA Formal Methods, Aaron Dutle, César Muñoz, and Anthony Narkawicz (Eds.). Springer International Publishing, Cham, 121--138.

[17]

Rüdiger Ehlers. 2017. Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks. In ATVA (Lecture Notes in Computer Science), Vol. 10482. Springer, 269--286.

[18]

G. Frehse, C. Le Guernic, A. Donzé, S. Cotton, R. Ray, O. Lebeltel, R. Ripado, A. Girard, T. Dang, and O. Maler. 2011. SpaceEx: Scalable Verification of Hybrid Systems. In Proc. of CAV'11 (LNCS), Vol. 6806. Springer, 379--395.

Digital Library

[19]

LiMin Fu. 1994. Rule generation from neural networks. IEEE Transactions on Systems, Man, and Cybernetics 24, 8 (Aug 1994), 1114--1124.

[20]

E. Hainry. 2008. Reachability in Linear Dynamical Systems. In Proc. of CiE 2008 (LNCS), Vol. 5028. Springer, 241--250.

Digital Library

[21]

Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2016. Safety Verification of Deep Neural Networks. CoRR abs/1610.06940 (2016). http://arxiv.org/abs/1610.06940

[22]

M. Jankovic, D. Fontaine, and P. V. Kokotovic. 1996. TORA example: cascade- and passivity-based control designs. IEEE Transactions on Control Systems Technology 4, 3 (1996), 292--297.

[23]

Kyle Julian and Mykel J. Kochenderfer. 2017. Neural Network Guidance for UAVs. In AIAA Guidance Navigation and Control Conference (GNC).

[24]

R. R. Kadiyala. 1993. A tool box for approximate linearization of nonlinear systems. IEEE Control Systems 13, 2 (April 1993), 47--57.

[25]

Gregory Kahn, Tianhao Zhang, Sergey Levine, and Pieter Abbeel. 2016. PLATO: Policy Learning using Adaptive Trajectory Optimization. arXiv preprint arXiv:1603.00622 (2016).

[26]

Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. Springer International Publishing, Cham, 97--117.

[27]

S. Kong, S. Gao, W. Chen, and E. M. Clarke. 2015. dReach: Δ-Reachability Analysis for Hybrid Systems. In Proc. of TACAS'15 (LNCS), Vol. 9035. Springer, 200--205.

Digital Library

[28]

Lectures. 2014. Nonlinear Systems and Control. http://people.ee.ethz.ch/~apnoco/Lectures2014/.

[29]

Alessio Lomuscio and Lalit Maganti. 2017. An approach to reachability analysis for feed-forward ReLU neural networks. CoRR abs/1706.07351 (2017). arXiv:1706.07351 http://arxiv.org/abs/1706.07351

[30]

K. Makino and M. Berz. 2003. Taylor models and other validated functional inclusion methods. J. Pure and Applied Mathematics 4, 4 (2003), 379--456.

[31]

S. Mitra and Y. Hayashi. 2000. Neuro-fuzzy rule generation: survey in soft computing framework. IEEE Transactions on Neural Networks 11, 3 (May 2000), 748--768.

Digital Library

[32]

R. E. Moore, R. B. Kearfott, and M. J. Cloud. 2009. Introduction to Interval Analysis. SIAM.

Digital Library

[33]

A. Neumaier. 1993. The Wrapping Effect, Ellipsoid Arithmetic, Stability and Confidence Regions. Springer Vienna, 175--190.

[34]

W. Perruquetti, J. P. Richard, and P. Borne. 1996. Lyapunov analysis ofsliding motions: Application to bounded control. Mathematical Problems in Engineering 3, 1 (1996), 1 -- 25.

[35]

Luca Pulina and Armando Tacchella. 2010. An abstraction-refinement approach to verification of artificial neural networks. In Computer Aided Verification. Springer, 243--257.

Digital Library

[36]

Luca Pulina and Armando Tacchella. 2012. Challenging SMT Solvers to Verify Neural Networks. AI Commun. 25, 2 (2012), 117--135.

[37]

Nathalie Revol and Fabrice Rouillier. 2005. Motivations for an Arbitrary Precision Interval Arithmetic and the MPFI Library. Reliable Computing 11, 4 (2005), 275--290.

[38]

Kazumi Saito and Ryohei Nakano. 2002. Extracting regression rules from neural networks. Neural Networks 15, 10 (2002), 1279 -- 1288.

Digital Library

[39]

Hanan J. Samet. 2006. Foundations of Multidimensional and Metric Data Structures. Morgan Kaufmann.

Digital Library

[40]

Mohamed Amin Ben Sassi, Ezio Bartocci, and Sriram Sankaranarayanan. 2017. A Linear Programming-based iterative approach to Stabilizing Polynomial Dynamics. In Proc. IFAC'17. Elsevier.

[41]

Karsten Scheibler, Leonore Winterer, Ralf Wimmer, and Bernd Becker. 2015. Towards verification of artificial neural networks. In MBMV Workshop. 30âĂŞ40.

[42]

Richard S. Sutton and Andrew G. Barto. 2017. Reinforcement Learning: An Introduction. MIT Press.

Digital Library

[43]

H. Tsukimoto. 2000. Extracting rules from trained neural networks. IEEE Transactions on Neural Networks 11, 2 (Mar 2000), 377--389.

Digital Library

[44]

Weiming Xiang, Hoang-Dung Tran, and Taylor T. Johnson. 2017. Output Reachable Set Estimation and Verification for Multi-Layer Neural Networks. CoRR abs/1708.03322 (2017). arXiv:1708.03322 http://arxiv.org/abs/1708.03322

[45]

Weiming Xiang, Hoang-Dung Tran, and Taylor T. Johnson. 2107. Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations. Cf. https://arxiv.org/pdf/1712.08163.pdf, posted on ArXIV Dec. 2017.

[46]

Weiming Xiang, Hoang-Dung Tran, Joel A. Rosenfeld, and Taylor T. Johnson. 2018. Reachable Set Estimation and Verification for a Class of Piecewise Linear Systems with Neural Network Controllers. To Appear in the American Control Conference (ACC), invited session on Formal Methods in Controller Synthesis.

[47]

Dong-Hae Yeom and Young Hoon Joo. 2012. Control Lyapunov Function Design by Cancelling Input Singularity. 12 (06 2012).

Cited By

Zhang HZhang YXu X(2024)Hybrid Zonotope-Based Backward Reachability Analysis for Neural Feedback Systems With Nonlinear Plant Models2024 American Control Conference (ACC)10.23919/ACC60939.2024.10644573(4155-4161)Online publication date: 10-Jul-2024
https://doi.org/10.23919/ACC60939.2024.10644573
Fatnassi WShoukry Y(2024)PolyARBerNN: A Neural Network Guided Solver and Optimizer for Bounded Polynomial InequalitiesACM Transactions on Embedded Computing Systems10.1145/363297023:2(1-26)Online publication date: 24-Jan-2024
https://dl.acm.org/doi/10.1145/3632970
Wang YZhou WFan JWang ZLi JChen XHuang CLi WZhu Q(2024)POLAR-Express: Efficient and Precise Formal Reachability Analysis of Neural-Network Controlled SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.333121543:3(994-1007)Online publication date: Mar-2024
https://doi.org/10.1109/TCAD.2023.3331215
Show More Cited By

Index Terms

Reachability analysis for neural feedback systems using regressive polynomial rule inference

Recommendations

Sherlock - A tool for verification of neural network feedback systems: demo abstract
HSCC '19: Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control

We present an approach for the synthesis and verification of neural network controllers for closed loop dynamical systems, modelled as an ordinary differential equation. Feedforward neural networks are ubiquitous when it comes to approximating functions,...
Simplified adaptive neural control of strict-feedback nonlinear systems

This paper presents a simplified adaptive backstepping neural network control (ABNNC) strategy for a general class of uncertain strict-feedback nonlinear systems. In the backstepping design, all unknown functions at intermediate steps are passed down ...
Adaptive neural network tracking control for a class of switched strict-feedback nonlinear systems with input delay

In this paper, a neural-network-based control scheme is developed for the tracking control problem of a class of switched strict-feedback nonlinear systems with uncertain input delay and external time-varying disturbances. First, the auxiliary signals ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HSCC '19: Proceedings of the 22nd ACM International Conference on Hybrid Systems: Computation and Control

April 2019

299 pages

ISBN:9781450362825

DOI:10.1145/3302504

Program Chairs:
Necmiye Ozay
University of Michigan
,
Pavithra Prabhakar
Kansas State University

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 April 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

US NSF
Air Force Research Laboratory

Conference

HSCC '19

Sponsor:

SIGBED

HSCC '19: 22nd ACM International Conference on Hybrid Systems: Computation and Control

April 16 - 18, 2019

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 153 of 373 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

102
Total Citations
View Citations
1,636
Total Downloads

Downloads (Last 12 months)434
Downloads (Last 6 weeks)29

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HZhang YXu X(2024)Hybrid Zonotope-Based Backward Reachability Analysis for Neural Feedback Systems With Nonlinear Plant Models2024 American Control Conference (ACC)10.23919/ACC60939.2024.10644573(4155-4161)Online publication date: 10-Jul-2024
https://doi.org/10.23919/ACC60939.2024.10644573
Fatnassi WShoukry Y(2024)PolyARBerNN: A Neural Network Guided Solver and Optimizer for Bounded Polynomial InequalitiesACM Transactions on Embedded Computing Systems10.1145/363297023:2(1-26)Online publication date: 24-Jan-2024
https://dl.acm.org/doi/10.1145/3632970
Wang YZhou WFan JWang ZLi JChen XHuang CLi WZhu Q(2024)POLAR-Express: Efficient and Precise Formal Reachability Analysis of Neural-Network Controlled SystemsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2023.333121543:3(994-1007)Online publication date: Mar-2024
https://doi.org/10.1109/TCAD.2023.3331215
Jafarpour SHarapanahalli ACoogan S(2024)Efficient Interaction-Aware Interval Analysis of Neural Network Feedback LoopsIEEE Transactions on Automatic Control10.1109/TAC.2024.342096869:12(8706-8721)Online publication date: Dec-2024
https://doi.org/10.1109/TAC.2024.3420968
Sheikhi SDuggirala PBak S(2024)Testing Autonomous Cyber-Physical Systems with Koopman Surrogate Model Predictive Control2024 IEEE 10th International Conference on Space Mission Challenges for Information Technology (SMC-IT)10.1109/SMC-IT61443.2024.00018(99-108)Online publication date: 15-Jul-2024
https://doi.org/10.1109/SMC-IT61443.2024.00018
Gupta AGhosh SBanerjee AMohalik S(2024)Configuring Safe Spiking Neural Controllers for Cyber-Physical Systems through Formal Verification2024 22nd ACM-IEEE International Symposium on Formal Methods and Models for System Design (MEMOCODE)10.1109/MEMOCODE63347.2024.00017(103-107)Online publication date: 3-Oct-2024
https://doi.org/10.1109/MEMOCODE63347.2024.00017
Dong YZhao XWang SHuang X(2024)Reachability Verification Based Reliability Assessment for Deep Reinforcement Learning Controlled Robotics and Autonomous SystemsIEEE Robotics and Automation Letters10.1109/LRA.2024.33644719:4(3299-3306)Online publication date: Apr-2024
https://doi.org/10.1109/LRA.2024.3364471
Thevendhriya HGhosh SLohar D(2024)Toward Precision-Aware Safe Neural-Controlled Cyber–Physical SystemsIEEE Embedded Systems Letters10.1109/LES.2024.344400416:4(397-400)Online publication date: Dec-2024
https://doi.org/10.1109/LES.2024.3444004
Dong ZOmidshafiei SEverett M(2024)Collision Avoidance Verification of Multiagent Systems With Learned PoliciesIEEE Control Systems Letters10.1109/LCSYS.2024.34001908(652-657)Online publication date: 2024
https://doi.org/10.1109/LCSYS.2024.3400190
Rober NMahesh KPaine TGreene MLee SMonteiro SBenjamin MHow J(2024)Online Data-Driven Safety Certification for Systems Subject to Unknown Disturbances2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610163(9939-9945)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610163
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten