Skip to main content
SpringerLink
Log in

Interactive Incremental Online Learning of Objects Onboard of a Cooperative Autonomous Mobile Robot

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2018)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11307))

Included in the following conference series:

Abstract

Detecting objects and referring to them in a dialog is a crucial requirement for robotic systems that cooperate with humans. For this, in an unrestricted natural environment the innate concepts of the robot must be extended and adapted over time. In this paper we describe an autonomous mobile robot system that performs online interactive incremental learning of objects. We argue that this combination strongly contributes to the variation of appearance, context, and labels under which visual concepts are encountered and thus overcomes limitations of existing databases and robotic systems where one or more of these aspects are missing. In the current prototype version, objects are shown to the robot in hand and are learned by a standard classifier on top of pre-trained CNN features. We evaluate the basic feasibility of the current approach on an existing database of hand-held objects, show how it performs online on the robot, and discuss extensions of the system towards life-long learning and data acquisition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andreopoulos, A., Hasler, S., Wersing, H., Janssen, H., Tsotsos, J.K., Körner, E.: Active 3D object localization using a humanoid robot. IEEE Trans. Robot. 27(1), 47–64 (2011)

    Article  Google Scholar 

  2. Borji, A., Izadi, S., Itti, L.: iLab-20M: a large-scale controlled object dataset to investigate deep learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2221–2230. IEEE Computer Society (2016)

    Google Scholar 

  3. Camoriano, R., Pasquale, G., Ciliberto, C., Natale, L., Rosasco, L., Metta, G.: Incremental robot learning of new objects with fixed update time. In: IEEE International Conference on Robotics and Automation, pp. 3207–3214. IEEE (2017)

    Google Scholar 

  4. Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: Valstar, M.F., French, A.P., Pridmore, T.P. (eds.) British Machine Vision Conference. BMVA Press (2014)

    Google Scholar 

  5. Deng, J., Dong, W., Socher, R., Li, L., Li, K., Li, F.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  6. Fäulhammer, T., et al.: Autonomous learning of object models on a mobile robot. IEEE Robot. Autom. Lett. 2(1), 26–33 (2017)

    Article  Google Scholar 

  7. Fischer, L., Hasler, S., Schrom, S., Wersing, H.: Improving online learning of visual categories by deep features. In: Future of Interactive Learning Machines workshop at the Conference on Neural Information Processing Systems (2016)

    Google Scholar 

  8. Goerick, C., et al.: Interactive online multimodal association for internal concept building in humanoids. In: 9th IEEE-RAS International Conference on Humanoid Robots, pp. 411–418. IEEE (2009)

    Google Scholar 

  9. Goerick, C., Wersing, H., Mikhailova, I., Dunn, M.: Peripersonal space and object recognition for humanoids. In: 5th IEEE-RAS International Conference on Humanoid Robots, pp. 387–392. IEEE (2005)

    Google Scholar 

  10. Google: Cloud Speech-To-Text. https://cloud.google.com/speech-to-text/

  11. Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Netw. 15(8–9), 1059–1068 (2002)

    Article  Google Scholar 

  12. Hasler, S., Wersing, H., Kirstein, S., Körner, E.: Large-scale real-time object identification based on analytic features. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009. LNCS, vol. 5769, pp. 663–672. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04277-5_67

    Chapter  Google Scholar 

  13. IBM: Watson Assistant. www.ibm.com/watson/services/conversation/

  14. Jia, Y., et al.: Caffe: convolutional Architecture for Fast Feature Embedding. In: Hua, K.A., Rui, Y., Steinmetz, R., Hanjalic, A., Natsev, A., Zhu, W. (eds.) Proceedings of the ACM International Conference on Multimedia, pp. 675–678. ACM (2014)

    Google Scholar 

  15. Käding, C., Rodner, E., Freytag, A., Denzler, J.: Fine-tuning deep neural networks in continuous learning scenarios. In: Chen, C.-S., Lu, J., Ma, K.-K. (eds.) ACCV 2016. LNCS, vol. 10118, pp. 588–605. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54526-4_43

    Chapter  Google Scholar 

  16. Kaiser, L., Nachum, O., Roy, A., Bengio, S.: Learning to remember rare events. CoRR abs/1703.03129 (2017)

    Google Scholar 

  17. Kinovarobotics: Kinova. www.meetjaco.com/about/

  18. Kirstein, S., Denecke, A., Hasler, S., Wersing, H., Gross, H., Körner, E.: A vision architecture for unconstrained and incremental learning of multiple categories. Memetic Comput. 1(4), 291–304 (2009)

    Article  Google Scholar 

  19. Kirstein, S., Wersing, H., Körner, E.: Rapid online learning of objects in a biologically motivated recognition architecture. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 301–308. Springer, Heidelberg (2005). https://doi.org/10.1007/11550518_38

    Chapter  Google Scholar 

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Bartlett, P.L., Pereira, F.C.N., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25: Proceedings of the 26th Annual Conference on Neural Information Processing Systems, pp. 1106–1114 (2012)

    Google Scholar 

  21. Lee, J., Yoon, J., Yang, E., Hwang, S.J.: Lifelong learning with dynamically expandable networks. CoRR abs/1708.01547 (2017)

    Google Scholar 

  22. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  23. Losing, V., Hammer, B., Wersing, H.: KNN classifier with self adjusting memory for heterogeneous concept drift. In: Bonchi, F., Domingo-Ferrer, J., Baeza-Yates, R.A., Zhou, Z., Wu, X. (eds.) IEEE 16th International Conference on Data Mining, pp. 291–300. IEEE (2016)

    Google Scholar 

  24. Losing, V., Hammer, B., Wersing, H.: Incremental on-line learning: a review and comparison of state of the art algorithms. Neurocomputing 275, 1261–1274 (2018)

    Article  Google Scholar 

  25. MetraLabs: Scitos G5. www.metralabs.com/en/mobile-robot-scitos-g5/

  26. Mudrová, L., Lacerda, B., Hawes, N.: An integrated control framework for long-term autonomy in mobile service robots. In: 2015 European Conference on Mobile Robots, pp. 1–6. IEEE (2015)

    Google Scholar 

  27. Nakadai, K., Takahashi, T., Okuno, H.G., Nakajima, H., Hasegawa, Y., Tsujino, H.: Design and implementation of robot audition system ‘Hark’ - open source software for listening to three simultaneous speakers. Adv. Robot. 24(5–6), 739–761 (2010)

    Article  Google Scholar 

  28. Nuance: Vocalizer Text-To-Speech. www.nuance.com/mobile/mobile-solutions/vocalizer-expressive.html

  29. Oberlin, J., Meier, M., Kraska, T., Tellex, S.: Acquiring object experiences at scale. In: AAAI-RSS Special Workshop on the 50th Anniversary of Shakey: The Role of AI to Harmonize Robots and Humans (2015)

    Google Scholar 

  30. Pasquale, G., Ciliberto, C., Odone, F., Rosasco, L., Natale, L.: Real-world object recognition with off-the-shelf deep Conv Nets: how many objects can iCub learn? CoRR abs/1504.03154 (2015)

    Google Scholar 

  31. Pasquale, G., Ciliberto, C., Rosasco, L., Natale, L.: Object identification from few examples by improving the invariance of a Deep Convolutional Neural Network. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4904–4911. IEEE (2016)

    Google Scholar 

  32. Peng, X., Usman, B., Kaushik, N., Hoffman, J., Wang, D., Saenko, K.: VisDA: the visual domain adaptation challenge. CoRR abs/1710.06924 (2017)

    Google Scholar 

  33. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6517–6525. IEEE Computer Society (2017)

    Google Scholar 

  34. ROS: Robot Operating System. www.ros.org

  35. Sato, A., Yamada, K.: Generalized learning vector quantization. In: Touretzky, D.S., Mozer, M., Hasselmo, M.E. (eds.) Advances in Neural Information Processing Systems 8, pp. 423–429. MIT Press (1995)

    Google Scholar 

Download references

Acknowledgments

We thank MetraLabs for the setup and support of the robots. We got a quick start with our robots by being able to use the software developed in the STRANDS project. For this we thank the whole project team, especially Lenka Mudrová and Nick Hawes. We also thank Manuel Mühlig for establishing and maintaining the basic robot software system at our institute.

Author information

Authors and Affiliations

  1. Honda Research Institute Europe GmbH, Carl-Legien-Str. 30, 63073, Offenbach am Main, Germany

    Stephan Hasler

  2. Frankfurt University of Applied Sciences, Nibelungenplatz 1, 60318, Frankfurt am Main, Germany

    Jennifer Kreger & Ute Bauer-Wersing

Authors
  1. Stephan Hasler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jennifer Kreger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ute Bauer-Wersing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephan Hasler .

Editor information

Editors and Affiliations

  1. The Chinese Academy of Sciences, Beijing, China

    Long Cheng

  2. City University of Hong Kong, Kowloon, Hong Kong

    Andrew Chi Sing Leung

  3. Kobe University, Kobe, Japan

    Seiichi Ozawa

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hasler, S., Kreger, J., Bauer-Wersing, U. (2018). Interactive Incremental Online Learning of Objects Onboard of a Cooperative Autonomous Mobile Robot. In: Cheng, L., Leung, A., Ozawa, S. (eds) Neural Information Processing. ICONIP 2018. Lecture Notes in Computer Science(), vol 11307. Springer, Cham. https://doi.org/10.1007/978-3-030-04239-4_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-04239-4_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-04238-7

  • Online ISBN: 978-3-030-04239-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation