skip to main content
10.1145/2728606.2728640acmconferencesArticle/Chapter ViewAbstractPublication PagescpsweekConference Proceedingsconference-collections
poster

Enforcing temporal logic specifications via reinforcement learning

Published: 14 April 2015 Publication History

Abstract

We consider the problem of controlling a system with unknown, stochastic dynamics to achieve a complex, time-sensitive task. An example of this problem is controlling a noisy aerial vehicle with partially known dynamics to visit a pre-specified set of regions in any order while avoiding hazardous areas. In particular, we are interested in tasks which can be described by signal temporal logic (STL) specifications. STL is a rich logic that can be used to describe tasks involving bounds on physical parameters, continuous time bounds, and logical relationships over time and states. STL is equipped with a continuous measure called the robustness degree that measures how strongly a given sample path exhibits an STL property [4, 3]. This measure enables the use of continuous optimization problems to solve learning [7, 6] or formal synthesis problems [9] involving STL.

References

[1]
A. Abate, A. D'Innocenzo, and M. Di Benedetto. Approximate abstractions of stochastic hybrid systems. Automatic Control, IEEE Transactions on, 56(11): 2688--2694, Nov 2011.
[2]
A. Dokhanchi, B. Hoxha, and G. Fainekos. On-line monitoring for temporal logic robustness. In Runtime Verification, pages 231--246. Springer, 2014.
[3]
A. Donzé and O. Maler. Robust satisfaction of temporal logic over real-valued signals. Springer, 2010.
[4]
G. E. Fainekos and G. J. Pappas. Robustness of temporal logic specifications for continuous-time signals. Theoretical Computer Science, 410(42): 4262--4291, 2009.
[5]
J. Fu and U. Topcu. Probably approximately correct MDP learning and control with temporal logic constraints. CoRR, abs/1404.7073, 2014.
[6]
A. Jones, Z. Kong, and C. Belta. Anomaly detection in cyber-physical systems: A formal methods approach. In IEEE Conference on Decision and Control (CDC), 2014.
[7]
Z. Kong, A. Jones, A. Medina Ayala, E. Aydin Gol, and C. Belta. Temporal logic inference for classification and prediction from data. In Proceedings of the 17th international conference on Hybrid systems: computation and control, pages 273--282. ACM, 2014.
[8]
M. Lahijanian, S. B. Andersson, and C. Belta. Approximate markovian abstractions for linear stochastic systems. In Proc. of the IEEE Conference on Decision and Control, pages 5966--5971, Maui, HI, USA, Dec. 2012.
[9]
V. Raman, A. Donze, M. Maasoumy, R. M. Murray, A. Sangiovanni-Vincentelli, and S. A. Seshia. Model predictive control with signal temporal logic specifications. In Proceedings of IEEE Conference on Decision and Control (CDC), 2014.
[10]
D. Sadigh, E. S. Kim, S. Coogan, S. S. Sastry, and S. A. Seshia. A learning based approach to control synthesis of markov decision processes for linear temporal logic specifications. CoRR, abs/1409.5486, 2014.
[11]
J. N. Tsitsiklis. Asynchronous stochastic approximation and q-learning. Machine Learning, 16(3): 185--202, 1994.

Cited By

View all
  • (2022)Perception-Based Temporal Logic Planning in Uncertain Semantic MapsIEEE Transactions on Robotics10.1109/TRO.2022.314407338:4(2536-2556)Online publication date: Aug-2022
  • (2022)Safe Policy Improvement in Constrained Markov Decision ProcessesLeveraging Applications of Formal Methods, Verification and Validation. Verification Principles10.1007/978-3-031-19849-6_21(360-381)Online publication date: 17-Oct-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HSCC '15: Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control
April 2015
321 pages
ISBN:9781450334334
DOI:10.1145/2728606
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 April 2015

Check for updates

Qualifiers

  • Poster

Funding Sources

  • Office of Naval Research (ONR)

Conference

HSCC '15
Sponsor:

Acceptance Rates

Overall Acceptance Rate 153 of 373 submissions, 41%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)18
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Perception-Based Temporal Logic Planning in Uncertain Semantic MapsIEEE Transactions on Robotics10.1109/TRO.2022.314407338:4(2536-2556)Online publication date: Aug-2022
  • (2022)Safe Policy Improvement in Constrained Markov Decision ProcessesLeveraging Applications of Formal Methods, Verification and Validation. Verification Principles10.1007/978-3-031-19849-6_21(360-381)Online publication date: 17-Oct-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media