Abstract
Natural language is an intuitive way for humans to communicate formal requirements of cyber-physical systems, such as safety specifications, performance requirements, and task objectives with autonomous cyber-physical systems such as robots. While natural language (NL) is ambiguous, real world tasks and their safety requirements need to be communicated unambiguously. Signal Temporal Logic (STL) is a formal logic that can serve as a versatile, expressive, and unambiguous formal language to describe robotic tasks. On one hand, existing work in using STL for the robotics domain typically requires end-users to express task specifications in STL, which is a challenge for non-expert users. On the other, translating from NL to STL specifications is currently restricted to specific fragments. In this work, we propose DialogueSTL, an explainable and interactive approach for learning correct and concise STL formulas from (often) ambiguous NL descriptions. We use a combination of semantic parsing, pre-trained transformer-based language models, and user-in-the-loop clarifications aided by a small number of user demonstrations to predict the best STL formula to encode NL task descriptions. An advantage of mapping NL to STL is that there has been considerable recent work on the use of reinforcement learning (RL) to identify control policies for robots. We show we can use Deep Q-Learning techniques to learn optimal policies from the learned STL specifications. We demonstrate that DialogueSTL is efficient, scalable, and robust, and has high accuracy in predicting the correct STL formula with a few number of demonstrations and a few interactions with an oracle user.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
\(\textit{robotAtWall}\) is a trace that its value at time instance t is computed as distance of the robot from the closest wall at time t.
- 2.
We denote a verb phrase loosely as a group of words that contains a verb, such as “turn on the lamp,” “if fire is on,” and “open the door.”.
- 3.
Users tend not to specify an explicit word that corresponds to the eventually operator \(\textbf{F}\), even if they do expect the robot to perform the task eventually.
- 4.
Each state is a tuple of 16 elements consist of robot and each of the items’ (door key, green and purple cube) positions, state of the lamp and fire (on or off), and state of the door (open or close).
- 5.
We run the experiments on an Intel Core-i7 Macbook Pro with 2.7 GHz processors and 16 GB RAM.
References
Akbik, A., Bergmann, T., Blythe, D., Rasul, K., Schweter, S., Vollgraf, R.: FLAIR: an easy-to-use framework for state-of-the-art NLP. In: NAACL 2019, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pp. 54–59 (2019)
Asarin, E., Donzé, A., Maler, O., Nickovic, D.: Parametric identification of temporal properties. In: Khurshid, S., Sen, K. (eds.) RV 2011. LNCS, vol. 7186, pp. 147–160. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29860-8_12
Autili, M., Grunske, L., Lumpe, M., Pelliccione, P., Tang, A.: Aligning qualitative, real-time, and probabilistic property specification patterns using a structured English grammar. IEEE Trans. Software Eng. 41(7), 620–638 (2015)
Balakrishnan, A., Deshmukh, J.V.: Structured reward shaping using signal temporal logic specifications. In: International Conference on Intelligent Robots and Systems (IROS) (2019)
Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)
Bunk, T., Varshneya, D., Vlasov, V., Nichol, A.: DIET: lightweight language understanding for dialogue systems. arXiv preprint arXiv:2004.09936 (2020)
Chen, Y., Arkin, J., Dawson, C., Zhang, Y., Roy, N., Fan, C.: AutoTAMP: autoregressive task and motion planning with LLMs as translators and checkers (2024)
Chen, Y., Gandhi, R., Zhang, Y., Fan, C.: NL2TL: transforming natural languages to temporal logics using large language models. In: Bouamor, H., Pino, J., Bali, K. (eds.) Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 15880–15903. Association for Computational Linguistics, Singapore, December 2023. https://doi.org/10.18653/v1/2023.emnlp-main.985. https://aclanthology.org/2023.emnlp-main.985
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Donzé, A.: Breach, a toolbox for verification and parameter synthesis of hybrid systems. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 167–170. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_17
Gavran, I., Darulova, E., Majumdar, R.: Interactive synthesis of temporal specifications from examples and natural language. Proc. ACM Programm. Lang. 4(OOPSLA), 1–26 (2020)
He, J., Bartocci, E., Ničković, D., Isakovic, H., Grosu, R.: From English to signal temporal logic. arXiv preprint arXiv:2109.10294 (2021)
Kress-Gazit, H., Fainekos, G.E., Pappas, G.J.: Translating structured English to robot controllers. Adv. Robot. 22(12), 1343–1359 (2008)
Liu, J.X., et al.: Grounding complex natural language commands for temporal tasks in unseen environments (2023)
Maler, O., Nickovic, D.: Monitoring temporal properties of continuous signals. In: Lakhnech, Y., Yovine, S. (eds.) FORMATS/FTRTFT -2004. LNCS, vol. 3253, pp. 152–166. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30206-3_12
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Mohammadinejad, S., Deshmukh, J.V., Puranic, A.G., Vazquez-Chanlatte, M., Donzé, A.: Interpretable classification of time-series data using efficient enumerative techniques. In: Proceedings of HSCC, pp. 1–10 (2020)
Nelken, R., Francez, N.: Automatic translation of natural language system specifications into temporal logic. In: Alur, R., Henzinger, T.A. (eds.) CAV 1996. LNCS, vol. 1102, pp. 360–371. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-61474-5_83
Puranic, A., Deshmukh, J., Nikolaidis, S.: Learning from demonstrations using signal temporal logic. In: Proceedings of the 2020 Conference on Robot Learning (2021)
Ranta, A.: Translating between language and logic: what is easy and what is difficult. In: Bjørner, N., Sofronie-Stokkermans, V. (eds.) CADE 2011. LNCS (LNAI), vol. 6803, pp. 5–25. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22438-6_3
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mohammadinejad, S., Paul, S., Xia, Y., Kudalkar, V., Thomason, J., Deshmukh, J.V. (2025). Systematic Translation from Natural Language Robot Task Descriptions to STL. In: Steffen, B. (eds) Bridging the Gap Between AI and Reality. AISoLA 2024. Lecture Notes in Computer Science, vol 15217. Springer, Cham. https://doi.org/10.1007/978-3-031-75434-0_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-75434-0_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-75433-3
Online ISBN: 978-3-031-75434-0
eBook Packages: Computer ScienceComputer Science (R0)