Abstract
Natural language programming (NLPr) allows people to program in natural language (NL) for specific domains. It poses great potential since it gives non-experts the ability to develop projects without exhaustive training. However, complex descriptions can sometimes have multiple interpretations, making program synthesis difficult. Thus, if the high-level abstractions can be broken down into a sequence of precise low-level steps, existing natural language processing (NLP) and NLPr techniques could be adaptable to handle the tasks. In this paper, we present an algorithm for converting high-level task descriptions into low-level specifications by parsing the sentences into sentence frames and using generated low-level NL instructions to generate executable programs for pathfinding tasks in a LEGO Mindstorms EV3 robot. Our analysis shows that breaking down the high-level pathfinding abstractions into a sequence of low-level NL instructions is effective for the majority of collected sentences, and the generated NL texts are detailed, readable, and can easily be processed by the existing NLPr system.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Balog, M., Gaunt, A.L., Brockschmidt, M., Nowozin, S., Tarlow, D.: Deepcoder: learning to write programs. CoRR abs/1611.01989 (2016)
Dijkstra, E.W.: On the foolishness of “natural language programming”. In: Bauer, F.L., et al. (eds.) Program Construction. LNCS, vol. 69, pp. 51–53. Springer, Heidelberg (1979). https://doi.org/10.1007/BFb0014656
Ernst, M.D.: Natural language is a programming language: applying natural language processing to software development. In: The 2nd Summit on Advances in Programming Languages, SNAPL 2017, CA, USA, pp. 4:1–4:14. Asilomar, May 2017
Hayes, B., Shah, J.A.: Improving robot controller transparency through autonomous policy explanation. In: Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, HRI 2017, New York, NY, USA, pp. 303–312. Association for Computing Machinery (2017)
Hsiao, M.S.: Automated program synthesis from object-oriented natural language for computer games. In: Controlled Natural Language - Proceedings of the Sixth International Workshop, CNL 2018, Maynooth, Co., Kildare, Ireland, 27–28, August 2018, pp. 71–74 (2018)
Jurafsky, D., Martin, J.H.: Speech and Language Processing, 2nd edn. Prentice-Hall Inc., Hoboken (2009)
Kuhn, T.: A survey and classification of controlled natural languages. Comput. Linguist. 40(1), 121–170 (2014)
Lauria, S., Bugmann, G., Kyriacou, T., Klein, E.: Mobile robot programming using natural language. Robot. Auton. Syst. 38(3), 171–181 (2002). Advances in Robot Skill Learning
Lee, C.Y.: An algorithm for path connections and its applications. IRE Trans. Electron. Comput. EC-10(3), 346–365 (1961)
Lin, X.V., Wang, C., Pang, D., Vu, K., Zettlemoyer, L., Ernst, M.D.: Program synthesis from natural language using recurrent neural networks. Technical report, UW-CSE-17-03-01, University of Washington Department of Computer Science and Engineering, Seattle, WA, USA, Mar 2017
Liu, H.: Metafor: visualizing stories as code. In: 10th International Conference on Intelligent User Interfaces, pp. 305–307. ACM Press (2005)
Loper, E., Bird, S.: NLTK: the natural language toolkit. In: Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics, ETMTNLP 2002, USA, vol. 1, pp. 63–70. Association for Computational Linguistics (2002)
Lopes, L.S., Teixeira, A.: Human-robot interaction through spoken language dialogue. In: Proceedings of the 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No. 00CH37113), vol. 1, pp. 528–534 (2000)
Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)
Matuszek, C., Herbst, E., Zettlemoyer, L., Fox, D.: Learning to parse natural language commands to a robot control system. In: Desai, J., Dudek, G., Khatib, O., Kumar, V. (eds.) Experimental Robotics, pp. 403–415. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-319-00065-7_28
Maxemchuk, N.: Routing in the Manhattan street network. IEEE Trans. Commun. 35(5), 503–512 (1987)
Menon, A.K., Tamuz, O., Gulwani, S., Lampson, B., Kalai, A.T.: A machine learning framework for programming by example. In: Proceedings of the 30th International Conference on International Conference on Machine Learning, ICML 2013, vol. 28, pp. I-187–I-195. JMLR.org (2013)
Mihalcea, R., Liu, H., Lieberman, H.: NLP (Natural Language Processing) for NLP (Natural Language Programming). In: Gelbukh, A. (ed.) CICLing 2006. LNCS, vol. 3878, pp. 319–330. Springer, Heidelberg (2006). https://doi.org/10.1007/11671299_34
Perera, V., Selveraj, S.P., Rosenthal, S., Veloso, M.: Dynamic generation and refinement of robot verbalization. In: 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 212–218, August 2016
Rangra, R., Madhusudan: Natural language parsing: using finite state automata. In: 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), pp. 456–463 (2016)
Rosenthal, S., Selvaraj, S.P., Veloso, M.: Verbalization: narration of autonomous robot experience. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, pp. 862–868. AAAI Press (2016). http://dl.acm.org/citation.cfm?id=3060621.3060741
Toutanova, K., Klein, D., Manning, C.D., Singer, Y.: Feature-rich part-of-speech tagging with a cyclic dependency network. In: Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pp. 252–259 (2003). https://www.aclweb.org/anthology/N03-1033
Wang, X., et al.: Reinforced cross-modal matching and self-supervised imitation learning for vision-language navigation. CoRR abs/1811.10092 (2018)
Zhan, Y., Hsiao, M.S.: A natural language programming application for Lego Mindstorms EV3. In: 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), pp. 27–34, December 2018
Zhou, Y., Wang, W., He, D., Wang, Z.: A fewest-turn-and-shortest path algorithm based on breadth-first search. Geo-spatial Inf. Sci. 17(4), 201–207 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhan, Y., Hsiao, M.S. (2021). Breaking Down High-Level Robot Path-Finding Abstractions in Natural Language Programming. In: Baldoni, M., Bandini, S. (eds) AIxIA 2020 – Advances in Artificial Intelligence. AIxIA 2020. Lecture Notes in Computer Science(), vol 12414. Springer, Cham. https://doi.org/10.1007/978-3-030-77091-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-77091-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77090-7
Online ISBN: 978-3-030-77091-4
eBook Packages: Computer ScienceComputer Science (R0)