Skip to main content
SpringerLink
Log in
Book cover

International Conference: Beyond Databases, Architectures and Structures

BDAS 2017: Beyond Databases, Architectures and Structures. Towards Efficient Solutions for Data Analysis and Knowledge Representation pp 288–301Cite as

Reading Comprehension of Natural Language Instructions by Robots

  • Conference paper
  • First Online:

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 716))

Abstract

We address the problem of robots executing instructions written for humans. The goal is to simplify and speed-up the process of robot adaptation to certain tasks, which are described in human language. We propose an approach, where semantic roles are attached to the components of instructions which lead to robotic execution. However, extraction of such roles from the sentence is not trivial due to the prevalent non determinism of human language. We propose algorithms for extracting actions and object names with roles and explain, how it leads to the robotic execution via attached sub-symbolic information of previous execution examples for rotor assembly and bio(technology) laboratory scenarios. The precision for the main action extraction is 0.977, for the main, primary and secondary objects is 0.828, 0.943 and 0.954, respectively.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Aksoy, E.E., Abramov, A., Dörr, J., Ning, K., Dellen, B., Wörgötter, F.: Learning the semantics of object-action relations by observation. The Int. J. Robot. Res. 30(10), 1229–1249 (2011)

    Article  Google Scholar 

  2. Chen, D., Manning, C.D.: A fast and accurate dependency parser using neural networks. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pp. 740–750 (2014)

    Google Scholar 

  3. Daudaravičius, V., Marcinkevičienė, R.: Gravity counts for the boundaries of collocations. Int. J. Corpus Linguist. 9(2), 321–348 (2004)

    Article  Google Scholar 

  4. De Marneffe, M.C., Manning, C.D.: Stanford typed dependencies manual. Technical report, Stanford University (2008)

    Google Scholar 

  5. El Idrissi, B., Baïna, S., Baïna, K.: Ontology learning from relational database: how to label the relationships between concepts? In: Publishing, S.I. (ed.) International Conference: Beyond Databases, Architectures and Structures, pp. 235–244 (2015)

    Google Scholar 

  6. Kumova, B.İ.: Generating ontologies from relational data with fuzzy-syllogistic reasoning. In: Kozielski, S., Mrozek, D., Kasprowski, P., Małysiak-Mrozek, B., Kostrzewa, D. (eds.) BDAS 2015. CCIS, vol. 521, pp. 21–32. Springer, Cham (2015). doi:10.1007/978-3-319-18422-7_2

  7. Marcus, M.P., Marcinkiewicz, M.A., Santorini, B.: Building a large annotated corpus of english: the penn treebank. Computat. Linguist. 19(2), 313–330 (1993)

    Google Scholar 

  8. Markievicz, I., Vitkute-Adzgauskiene, D., Tamosiunaite, M.: Semi-supervised learning of action ontology from domain-specific corpora. In: Skersys, T., Butleris, R., Butkiene, R. (eds.) ICIST 2013. CCIS, vol. 403, pp. 173–185. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41947-8_16

    Chapter  Google Scholar 

  9. Misra, D.K., Sung, J., Lee, K., Saxena, A.: Tell me dave: context-sensitive grounding of natural language to manipulation instructions. Int. J. Robot. Res. 35(1–3), 281–300 (2016)

    Article  Google Scholar 

  10. Nyga, D., Beetz, M.: Cloud-based probabilistic knowledge services for instruction interpretation. In: ISRR, Genoa, Italy (2015)

    Google Scholar 

  11. Tellex, S., Kollar, T., Dickerson, S., Walter, M.R., Banerjee, A.G., Teller, S., Roy, N.: Approaching the symbol grounding problem with probabilistic graphical models. AI Mag. 32(4), 64–76 (2011)

    Google Scholar 

  12. Yang, Y., Guha, A., Fermuller, C., Aloimonos, Y.: A cognitive system for understanding human manipulation actions. Adv. Cogn. Syst. 3, 67–86 (2014)

    Google Scholar 

Download references

Acknowledgments

The research leading to these results has received funding from the European Community’s Seventh Framework Programme FP7/2007–2013 (Programme and Theme: ICT-2011.2.1, Cognitive Systems and Robotics) under grant agreement No. 600578, ACAT.

Author information

Authors and Affiliations

  1. Faculty of Informatics, Vytautas Magnus University, Kaunas, Lithuania

    Irena Markievicz, Minija Tamosiunaite, Daiva Vitkute-Adzgauskiene, Jurgita Kapociute-Dzikiene, Rita Valteryte & Tomas Krilavicius

  2. Bernstein Center for Computational Neuroscience, University of Goettingen, Goettingen, Germany

    Minija Tamosiunaite

Authors
  1. Irena Markievicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minija Tamosiunaite
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daiva Vitkute-Adzgauskiene
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jurgita Kapociute-Dzikiene
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rita Valteryte
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tomas Krilavicius
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irena Markievicz .

Editor information

Editors and Affiliations

  1. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Stanisław Kozielski

  2. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Dariusz Mrozek

  3. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Paweł Kasprowski

  4. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Bożena Małysiak-Mrozek

  5. Institute of Informatics, Silesian University of Technology, Gliwice, Poland

    Daniel Kostrzewa

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Markievicz, I., Tamosiunaite, M., Vitkute-Adzgauskiene, D., Kapociute-Dzikiene, J., Valteryte, R., Krilavicius, T. (2017). Reading Comprehension of Natural Language Instructions by Robots. In: Kozielski, S., Mrozek, D., Kasprowski, P., Małysiak-Mrozek, B., Kostrzewa, D. (eds) Beyond Databases, Architectures and Structures. Towards Efficient Solutions for Data Analysis and Knowledge Representation. BDAS 2017. Communications in Computer and Information Science, vol 716. Springer, Cham. https://doi.org/10.1007/978-3-319-58274-0_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-58274-0_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-58273-3

  • Online ISBN: 978-3-319-58274-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation