Skip to main content

Systematic Translation from Natural Language Robot Task Descriptions to STL

  • Conference paper
  • First Online:
Bridging the Gap Between AI and Reality (AISoLA 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15217))

Included in the following conference series:

  • 79 Accesses

Abstract

Natural language is an intuitive way for humans to communicate formal requirements of cyber-physical systems, such as safety specifications, performance requirements, and task objectives with autonomous cyber-physical systems such as robots. While natural language (NL) is ambiguous, real world tasks and their safety requirements need to be communicated unambiguously. Signal Temporal Logic (STL) is a formal logic that can serve as a versatile, expressive, and unambiguous formal language to describe robotic tasks. On one hand, existing work in using STL for the robotics domain typically requires end-users to express task specifications in STL, which is a challenge for non-expert users. On the other, translating from NL to STL specifications is currently restricted to specific fragments. In this work, we propose DialogueSTL, an explainable and interactive approach for learning correct and concise STL formulas from (often) ambiguous NL descriptions. We use a combination of semantic parsing, pre-trained transformer-based language models, and user-in-the-loop clarifications aided by a small number of user demonstrations to predict the best STL formula to encode NL task descriptions. An advantage of mapping NL to STL is that there has been considerable recent work on the use of reinforcement learning (RL) to identify control policies for robots. We show we can use Deep Q-Learning techniques to learn optimal policies from the learned STL specifications. We demonstrate that DialogueSTL is efficient, scalable, and robust, and has high accuracy in predicting the correct STL formula with a few number of demonstrations and a few interactions with an oracle user.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    \(\textit{robotAtWall}\) is a trace that its value at time instance t is computed as distance of the robot from the closest wall at time t.

  2. 2.

    We denote a verb phrase loosely as a group of words that contains a verb, such as “turn on the lamp,” “if fire is on,” and “open the door.”.

  3. 3.

    Users tend not to specify an explicit word that corresponds to the eventually operator \(\textbf{F}\), even if they do expect the robot to perform the task eventually.

  4. 4.

    Each state is a tuple of 16 elements consist of robot and each of the items’ (door key, green and purple cube) positions, state of the lamp and fire (on or off), and state of the door (open or close).

  5. 5.

    We run the experiments on an Intel Core-i7 Macbook Pro with 2.7 GHz processors and 16 GB RAM.

References

  1. Akbik, A., Bergmann, T., Blythe, D., Rasul, K., Schweter, S., Vollgraf, R.: FLAIR: an easy-to-use framework for state-of-the-art NLP. In: NAACL 2019, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pp. 54–59 (2019)

    Google Scholar 

  2. Asarin, E., Donzé, A., Maler, O., Nickovic, D.: Parametric identification of temporal properties. In: Khurshid, S., Sen, K. (eds.) RV 2011. LNCS, vol. 7186, pp. 147–160. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29860-8_12

    Chapter  MATH  Google Scholar 

  3. Autili, M., Grunske, L., Lumpe, M., Pelliccione, P., Tang, A.: Aligning qualitative, real-time, and probabilistic property specification patterns using a structured English grammar. IEEE Trans. Software Eng. 41(7), 620–638 (2015)

    Article  Google Scholar 

  4. Balakrishnan, A., Deshmukh, J.V.: Structured reward shaping using signal temporal logic specifications. In: International Conference on Intelligent Robots and Systems (IROS) (2019)

    Google Scholar 

  5. Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

  6. Bunk, T., Varshneya, D., Vlasov, V., Nichol, A.: DIET: lightweight language understanding for dialogue systems. arXiv preprint arXiv:2004.09936 (2020)

  7. Chen, Y., Arkin, J., Dawson, C., Zhang, Y., Roy, N., Fan, C.: AutoTAMP: autoregressive task and motion planning with LLMs as translators and checkers (2024)

    Google Scholar 

  8. Chen, Y., Gandhi, R., Zhang, Y., Fan, C.: NL2TL: transforming natural languages to temporal logics using large language models. In: Bouamor, H., Pino, J., Bali, K. (eds.) Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 15880–15903. Association for Computational Linguistics, Singapore, December 2023. https://doi.org/10.18653/v1/2023.emnlp-main.985. https://aclanthology.org/2023.emnlp-main.985

  9. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  10. Donzé, A.: Breach, a toolbox for verification and parameter synthesis of hybrid systems. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 167–170. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14295-6_17

    Chapter  MATH  Google Scholar 

  11. Gavran, I., Darulova, E., Majumdar, R.: Interactive synthesis of temporal specifications from examples and natural language. Proc. ACM Programm. Lang. 4(OOPSLA), 1–26 (2020)

    Google Scholar 

  12. He, J., Bartocci, E., Ničković, D., Isakovic, H., Grosu, R.: From English to signal temporal logic. arXiv preprint arXiv:2109.10294 (2021)

  13. Kress-Gazit, H., Fainekos, G.E., Pappas, G.J.: Translating structured English to robot controllers. Adv. Robot. 22(12), 1343–1359 (2008)

    Article  MATH  Google Scholar 

  14. Liu, J.X., et al.: Grounding complex natural language commands for temporal tasks in unseen environments (2023)

    Google Scholar 

  15. Maler, O., Nickovic, D.: Monitoring temporal properties of continuous signals. In: Lakhnech, Y., Yovine, S. (eds.) FORMATS/FTRTFT -2004. LNCS, vol. 3253, pp. 152–166. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30206-3_12

    Chapter  MATH  Google Scholar 

  16. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Google Scholar 

  17. Mohammadinejad, S., Deshmukh, J.V., Puranic, A.G., Vazquez-Chanlatte, M., Donzé, A.: Interpretable classification of time-series data using efficient enumerative techniques. In: Proceedings of HSCC, pp. 1–10 (2020)

    Google Scholar 

  18. Nelken, R., Francez, N.: Automatic translation of natural language system specifications into temporal logic. In: Alur, R., Henzinger, T.A. (eds.) CAV 1996. LNCS, vol. 1102, pp. 360–371. Springer, Heidelberg (1996). https://doi.org/10.1007/3-540-61474-5_83

    Chapter  MATH  Google Scholar 

  19. Puranic, A., Deshmukh, J., Nikolaidis, S.: Learning from demonstrations using signal temporal logic. In: Proceedings of the 2020 Conference on Robot Learning (2021)

    Google Scholar 

  20. Ranta, A.: Translating between language and logic: what is easy and what is difficult. In: Bjørner, N., Sofronie-Stokkermans, V. (eds.) CADE 2011. LNCS (LNAI), vol. 6803, pp. 5–25. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22438-6_3

    Chapter  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jyotirmoy V. Deshmukh .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mohammadinejad, S., Paul, S., Xia, Y., Kudalkar, V., Thomason, J., Deshmukh, J.V. (2025). Systematic Translation from Natural Language Robot Task Descriptions to STL. In: Steffen, B. (eds) Bridging the Gap Between AI and Reality. AISoLA 2024. Lecture Notes in Computer Science, vol 15217. Springer, Cham. https://doi.org/10.1007/978-3-031-75434-0_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-75434-0_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-75433-3

  • Online ISBN: 978-3-031-75434-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics