Skip to main content
SpringerLink
Log in

Robust Detection for Autonomous Elevator Boarding Using a Mobile Manipulator

  • Conference paper
  • First Online:
Pattern Recognition (ACPR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14406))

Included in the following conference series:

  • 247 Accesses

Abstract

Indoor robots are becoming increasingly prevalent across a range of sectors, but the challenge of navigating multi-level structures through elevators remains largely uncharted. For a robot to operate successfully, it’s pivotal to have an accurate perception of elevator states. This paper presents a robust robotic system, tailored to interact adeptly with elevators by discerning their status, actuating buttons, and boarding seamlessly. Given the inherent issues of class imbalance and limited data, we utilize the YOLOv7 model and adopt specific strategies to counteract the potential decline in object detection performance. Our method effectively confronts the class imbalance and label dependency observed in real-world datasets, Our method effectively confronts the class imbalance and label dependency observed in real-world datasets, offering a promising approach to improve indoor robotic navigation systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chaplot, D.S., Gandhi, D.P., Gupta, A., Salakhutdinov, R.R.: Object goal navigation using goal-oriented semantic exploration. Adv. Neural. Inf. Process. Syst. 33, 4247–4258 (2020)

    Google Scholar 

  2. Dong, Z., Zhu, D., Meng, M.Q.H.: An autonomous elevator button recognition system based on convolutional neural networks. In: 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2533–2539. IEEE (2017)

    Google Scholar 

  3. Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  4. Huang, C., Mees, O., Zeng, A., Burgard, W.: Visual language maps for robot navigation. arXiv preprint arXiv:2210.05714 (2022)

  5. Kang, J.G., An, S.Y., Choi, W.S., Oh, S.Y.: Recognition and path planning strategy for autonomous navigation in the elevator environment. Int. J. Control Autom. Syst. 8, 808–821 (2010)

    Article  Google Scholar 

  6. Kim, H.H., Kim, D.J., Park, K.H.: Robust elevator button recognition in the presence of partial occlusion and clutter by specular reflections. IEEE Trans. Industr. Electron. 59(3), 1597–1611 (2011)

    Article  Google Scholar 

  7. Klingbeil, E., Carpenter, B., Russakovsky, O., Ng, A.Y.: Autonomous operation of novel elevators for robot navigation. In: 2010 IEEE International Conference on Robotics and Automation, pp. 751–758. IEEE (2010)

    Google Scholar 

  8. Law, H., Deng, J.: CornerNet: detecting objects as paired keypoints. Int. J. Comput. Vision 128(3), 642–656 (2019). https://doi.org/10.1007/s11263-019-01204-1

    Article  Google Scholar 

  9. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

    Google Scholar 

  10. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  11. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  12. Macenski, S., Martín, F., White, R., Clavero, J.G.: The marathon 2: a navigation system. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2718–2725. IEEE (2020)

    Google Scholar 

  13. Menini, D., Kumar, S., Oswald, M.R., Sandström, E., Sminchisescu, C., Van Gool, L.: A real-time online learning framework for joint 3D reconstruction and semantic segmentation of indoor scenes. IEEE Robot. Autom. Lett. 7(2), 1332–1339 (2021)

    Article  Google Scholar 

  14. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  15. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems 28 (2015)

    Google Scholar 

  16. Song, J., Patel, M., Ghaffari, M.: Fusing convolutional neural network and geometric constraint for image-based indoor localization. IEEE Robot. Autom. Lett. 7(2), 1674–1681 (2022)

    Article  Google Scholar 

  17. Vidanapathirana, K., Ramezani, M., Moghadam, P., Sridharan, S., Fookes, C.: LoGG3D-Net: locally guided global descriptor learning for 3d place recognition. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 2215–2221. IEEE (2022)

    Google Scholar 

  18. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696 (2022)

  19. Yang, P.Y., Chang, T.H., Chang, Y.H., Wu, B.F.: Intelligent mobile robot controller design for hotel room service with deep learning arm-based elevator manipulator. In: 2018 International Conference on System Science and Engineering (ICSSE), pp. 1–6. IEEE (2018)

    Google Scholar 

  20. Zhou, X., Wang, D., Krähenbühl, P.: Objects as points. arXiv preprint arXiv:1904.07850 (2019)

  21. Zhu, D., Li, T., Ho, D., Zhou, T., Meng, M.Q.: A novel OCR-RCNN for elevator button recognition. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3626–3631. IEEE (2018)

    Google Scholar 

  22. Zhu, D., Min, Z., Zhou, T., Li, T., Meng, M.Q.H.: An autonomous eye-in-hand robotic system for elevator button operation based on deep recognition network. IEEE Trans. Instrum. Meas. 70, 1–13 (2020)

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported by “Research of Elevator Indicator Recognition Technology for Indoor Autonomous Navigation” project funded by ROBOTIS Co. Ltd. and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2019-0-00079, No. 2022-0-00871, No. 2022-0-00612, No. 2022-0-00480).

Author information

Authors and Affiliations

  1. Korea University, 13 Jongam-ro, Seongbuk-gu, Seoul, 02841, Republic of Korea

    Seungyoun Shin, Joon Hyung Lee & Sungjoon Choi

  2. Ewha Womans University, 52 Ewhayeodae-gil, Seodaemun-gu, Seoul, 03760, Republic of Korea

    Junhyug Noh

Authors
  1. Seungyoun Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joon Hyung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junhyug Noh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sungjoon Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sungjoon Choi .

Editor information

Editors and Affiliations

  1. Kyushu Institute of Technology, Kitakyushu, Fukuoka, Japan

    Huimin Lu

  2. The University of Sydney, Sydney, NSW, Australia

    Michael Blumenstein

  3. Yonsei University, Seoul, Korea (Republic of)

    Sung-Bae Cho

  4. Chinese Academy of Sciences, Beijing, China

    Cheng-Lin Liu

  5. Osaka University, Osaka, Ibaraki, Japan

    Yasushi Yagi

  6. Kyushu Institute of Technology, Kitakyushu, Japan

    Tohru Kamiya

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shin, S., Lee, J.H., Noh, J., Choi, S. (2023). Robust Detection for Autonomous Elevator Boarding Using a Mobile Manipulator. In: Lu, H., Blumenstein, M., Cho, SB., Liu, CL., Yagi, Y., Kamiya, T. (eds) Pattern Recognition. ACPR 2023. Lecture Notes in Computer Science, vol 14406. Springer, Cham. https://doi.org/10.1007/978-3-031-47634-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47634-1_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47633-4

  • Online ISBN: 978-3-031-47634-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation