Skip to main content
SpringerLink
Log in

Service Robots: A Unified Framework for Detecting, Opening and Navigating Through Doors

  • Conference paper
  • First Online:
Book cover Software Technologies (ICSOFT 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1250))

Included in the following conference series:

Abstract

For an autonomous robotic system, detecting, opening, and navigating through doors remains a very challenging problem. It involves several hard-to-solve sub-tasks such as recognizing the door frame and the handle, discriminating between different type of doors and their status, and opening and moving through the doorway. Previous works often tackle single individual sub-problems, assuming that the robot is moving in a well-known static environments or it is already facing the door handle. However, ignoring navigation issues, using specialized robots, or restricting the analysis to specific types of doors or handles, reduce the applicability of the proposed approach. In this paper, we present a unified framework for the door opening problem, by taking a navigation scenario as a reference. We implement specific algorithms to solve each sub-task and we describe the hierarchical automata which integrates the control of the robot during the entire process. We build a publicly available data-set which consists in 780 images of doors and handles crawled from Google Images. Using this data-set, we train a deep learning neural network, exploiting the Single Shot MultiBox Detector, to recognize doors and handles. We implement error recovery mechanisms to add robustness and reliability to our robot, and to guarantee a high success rate in every task. We carry-out experiments on a realistic scenario, the “Help Me Carry” task of the RoboCup 2018, using a standard service robot, the Toyota Human Support Robot. Our experiments demonstrate that our framework can successfully detect, open, and navigate through doors in a reliable way, with low error rates, and without adapting the environment to the robot.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The data set is publicly available at https://www.mi.t.u-tokyo.ac.jp/projects/mildoor.

  2. 2.

    http://www.ros.org.

  3. 3.

    https://cloud.google.com/speech-to-text.

  4. 4.

    SMACH is a ROS-independent Python library for building hierarchical state machines.

  5. 5.

    RVIZ is a tool for displaying sensor data using ROS.

  6. 6.

    https://newsroom.toyota.co.jp/jp/detail/8709536.

References

  1. Andreopoulos, J.A., Tsotsos, J.K.: A framework for door localization and door opening using a robotic wheelchair for people living with mobility impairments. In: Robotics: Science and Systems, Workshop: Robot Manipulation: Sensing and Adapting to the Real World (2007)

    Google Scholar 

  2. Aude, E.P.L., Lopes, E.P., Aguiar, C.S., Martins, M.F.: Door crossing and state identification using robotic vision. In: 8th IFAC Symposium on Robot Control IFAC Proceedings, vol. 39(15), 659–664 (2006). https://doi.org/10.3182/20060906-3-IT-2910.00110, http://www.sciencedirect.com/science/article/pii/S1474667016385895

  3. Boston Dynamics: Robots: SPOT. https://www.bostondynamics.com/spot-mini (2019). Accessed 10 Nov 2019

  4. Burgard, W., et al.: The interactive museum tour-guide robot. In: Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence, pp. 11–18. AAAI 1998/IAAI 1998, American Association for Artificial Intelligence, Menlo Park, CA, USA (1998) http://dl.acm.org/citation.cfm?id=295240.295249

  5. Chitta, S., Cohen, B., Likhachev, M.: Planning for autonomous door opening with a mobile manipulator. In: IEEE International Conference on Robotics and Automation, pp. 1799–1806, May 2010. https://doi.org/10.1109/ROBOT.2010.5509475

  6. Kim, D., Kang, J.-H., Hwang, C.-S., Park, G.-T.: Mobile robot for door opening in a house. In: Negoita, M.G., Howlett, R.J., Jain, L.C. (eds.) KES 2004. LNCS (LNAI), vol. 3215, pp. 596–602. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30134-9_80

    Chapter  Google Scholar 

  7. Fernández-Caramés, C., Moreno, V., Curto, B., Rodríguez-Aragón, J.F., Serrano, F.: A real-time door detection system for domestic robotic navigation. J. Intell. Robot. Syst. 76(1), 119–136 (2014). https://doi.org/10.1007/s10846-013-9984-6

    Article  Google Scholar 

  8. Gray, S., Chitta, S., Kumar, V., Likhachev, M.: A single planner for a composite task of approaching, opening and navigating through non-spring and spring-loaded doors. In: IEEE International Conference on Robotics and Automation, pp. 3839–3846, May 2013

    Google Scholar 

  9. Hernandez, K., Bacca, B., Posso, B.: Multi-goal path planning autonomous system for picking up and delivery tasks in mobile robotics. IEEE Latin Am. Trans. 15(2), 232–238 (2017)

    Article  Google Scholar 

  10. Jain, A., Kemp, C.C.: Behaviors for robust door opening and doorway traversal with a force-sensing mobile manipulator. In: RSS Workshop on Robot Manipulation: Intelligence in Human Environments (2008)

    Google Scholar 

  11. Johnson, M., et al.: Team IHMC’s lessons learned from the DARPA robotics challenge trials. J. Field Robot. 32(2), 192–208 (2015). https://doi.org/10.1002/rob.21571. https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.21571

    Article  MathSciNet  Google Scholar 

  12. Khatib, O.: Mobile manipulation: the robotic assistant. Robot. Autonomous Syst. 26(2), 175–183 (1999). https://doi.org/10.1016/S0921-8890(98)00067-0. http://www.sciencedirect.com/science/article/pii/S0921889098000670, field and Service Robotics

    Article  Google Scholar 

  13. Kim, G., Chung, W., Kim, K.R., Kim, M., Han, S., Shinn, R.: The autonomous tour-guide robot Jinny. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). vol. 4, pp. 3450–3455, January 2004. https://doi.org/10.1109/IROS.2004.1389950

  14. Kim, S., Cheong, H., Kim, D.H., Park, S.: Context-based object recognition for door detection. In: 15th International Conference on Advanced Robotics (ICAR), pp. 155–160, June 2011

    Google Scholar 

  15. Klingbeil, E., Saxena, A., Ng, A.Y.: Learning to open new doors. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2751–2757, October 2010. https://doi.org/10.1109/IROS.2010.5649847

  16. Kohlbrecher, S., von Stryk, O., Meyer, J., Klingauf, U.: A flexible and scalable SLAM system with full 3D motion estimation. In: IEEE International Symposium on Safety, Security, and Rescue Robotics, pp. 155–160, November 2011. https://doi.org/10.1109/SSRR.2011.6106777

  17. Krotkov, E., et al.: The DARPA robotics challenge finals: results and perspectives. J. Field Robot. 34(2), 229–240 (2017). https://doi.org/10.1002/rob.21683

    Article  Google Scholar 

  18. Lee, I., Oh, J.H.: Humanoid posture selection for reaching motion and a cooperative balancing controller. J. Intell. Robot. Syst. 81(3), 301–316 (2016). https://doi.org/10.1007/s10846-015-0225-z

    Article  Google Scholar 

  19. Lim, J., et al.: Robot system of DRC-HUBO+ and control strategy of team KAIST in DARPA robotics challenge finals. J. Field Robot. 34(4), 802–829 (2017). https://doi.org/10.1002/rob.21673

    Article  Google Scholar 

  20. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  21. Meeussen, W., et al.: Autonomous door opening and plugging in with a personal robot. In: IEEE International Conference on Robotics and Automation, pp. 729–736. Anchorage, Alaska, USA (2010)

    Google Scholar 

  22. Ott, C., Bäuml, B., Borst, C., Hirzinger, G.: Autonomous opening of a door with a mobile manipulator: a case study. In: 6th IFAC PSymposium on Intelligent Autonomous Vehicles, vol. 40, no. 15, pp. 349–354 (2007). https://doi.org/10.3182/20070903-3-FR-2921.00060, http://www.sciencedirect.com/science/article/pii/S1474667016346857

  23. Peterson, L., Austin, D., Kragic, D.: High-level control of a mobile manipulator for door opening. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). vol. 3, pp. 2333–2338 (2000). https://doi.org/10.1109/IROS.2000.895316

  24. Petrovskaya, A., Ng, A.Y.: Probabilistic mobile manipulation in dynamic environments, with application to opening doors. In: Proceedings of the 20th International Joint Conference on Artifical Intelligence, pp. 2178–2184. IJCAI 2007, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (2007). http://dl.acm.org/citation.cfm?id=1625275.1625627

  25. Rhee, C., Chung, W., Kim, M., Shim, Y., Lee, H.: Door opening control using the multi-fingered robotic hand for the indoor service robot. In: Proceedings IEEE International Conference on Robotics and Automation, vol. 4, pp. 4011–4016. IEEE, January 2004. https://doi.org/10.1109/ROBOT.2004.1308898

  26. RoboCup 2018: http://www.robocup2018.com (2018). Accessed 10 Nov 2019

  27. Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  28. Rusu, R.B., Meeussen, W., Chitta, S., Beetz, M.: Laser-based perception for door and handle identification. In: International Conference on Advanced Robotics, pp. 1–8, June 2009

    Google Scholar 

  29. Savarese, F., Tejero-de-Pablos, A., Quer, S., Harada, T.: Detecting, opening and navigating through doors: a unified framework for human service robots. In: 14th International Conference on Software Technologies (ICSOFT 2019), pp. 416–427, January 2019. https://doi.org/10.5220/0007947604160427

  30. Shalaby, M.M., Salem, M.A., Khamis, A., Melgani, F.: Geometric model for vision-based door detection. In: 9th International Conference on Computer Engineering Systems, pp. 41–46, December 2014. https://doi.org/10.1109/ICCES.2014.7030925

  31. Sunspiral, V., Kunz, C., Nourbakhsh, I.: The History of the Mobot Museum Robot Series: An Evolutionary Study. pp. 514–518, January 2001

    Google Scholar 

  32. Thrun, S., et al.: MINERVA: a second-generation museum tour-guide robot. IEEE Int. Conf. Robot. Automat. 3, 1999–2005 (1999)

    Article  Google Scholar 

  33. Toyota: Partner Robot. https://www.toyota-global.com/in-no-va-tion/partner_ro-bot/ro-bot/#link02 (2019). Accessed 10 Nov 2019

Download references

Acknowledgments

We would like to thank Yusuke Kurose, Yujin Tang, Jen-Yen Chang, James Borg, Takayoshi Takayanagi, Yingy Wen and Reza Motallebi for their help implementing this research. This work was partially supported by JST CREST Grant Number JPMJCR1403, Japan. The authors have been part of the HSR developer community (see footnote 6), and they made use of HSR hardware and software platforms.

Author information

Authors and Affiliations

  1. Machine Intelligence Lab., The University of Tokyo, Tokyo, Japan

    Tatsuya Harada, Antonio Tejero-de-Pablos & Francesco Savarese

  2. Department of Control and Computer Engineering, Politecnico di Torino, Turin, Italy

    Stefano Quer & Francesco Savarese

Authors
  1. Tatsuya Harada
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Tejero-de-Pablos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefano Quer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francesco Savarese
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio Tejero-de-Pablos .

Editor information

Editors and Affiliations

  1. Information Systems Group, University of Twente, Enschede, The Netherlands

    Marten van Sinderen

  2. Institute of Business Informatics, Wrocław University of Economics, Wrocław, Poland

    Leszek A. Maciaszek

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Harada, T., Tejero-de-Pablos, A., Quer, S., Savarese, F. (2020). Service Robots: A Unified Framework for Detecting, Opening and Navigating Through Doors. In: van Sinderen, M., Maciaszek, L. (eds) Software Technologies. ICSOFT 2019. Communications in Computer and Information Science, vol 1250. Springer, Cham. https://doi.org/10.1007/978-3-030-52991-8_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-52991-8_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-52990-1

  • Online ISBN: 978-3-030-52991-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation