Abstract
Indoor robots are becoming increasingly prevalent across a range of sectors, but the challenge of navigating multi-level structures through elevators remains largely uncharted. For a robot to operate successfully, it’s pivotal to have an accurate perception of elevator states. This paper presents a robust robotic system, tailored to interact adeptly with elevators by discerning their status, actuating buttons, and boarding seamlessly. Given the inherent issues of class imbalance and limited data, we utilize the YOLOv7 model and adopt specific strategies to counteract the potential decline in object detection performance. Our method effectively confronts the class imbalance and label dependency observed in real-world datasets, Our method effectively confronts the class imbalance and label dependency observed in real-world datasets, offering a promising approach to improve indoor robotic navigation systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chaplot, D.S., Gandhi, D.P., Gupta, A., Salakhutdinov, R.R.: Object goal navigation using goal-oriented semantic exploration. Adv. Neural. Inf. Process. Syst. 33, 4247–4258 (2020)
Dong, Z., Zhu, D., Meng, M.Q.H.: An autonomous elevator button recognition system based on convolutional neural networks. In: 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2533–2539. IEEE (2017)
Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Huang, C., Mees, O., Zeng, A., Burgard, W.: Visual language maps for robot navigation. arXiv preprint arXiv:2210.05714 (2022)
Kang, J.G., An, S.Y., Choi, W.S., Oh, S.Y.: Recognition and path planning strategy for autonomous navigation in the elevator environment. Int. J. Control Autom. Syst. 8, 808–821 (2010)
Kim, H.H., Kim, D.J., Park, K.H.: Robust elevator button recognition in the presence of partial occlusion and clutter by specular reflections. IEEE Trans. Industr. Electron. 59(3), 1597–1611 (2011)
Klingbeil, E., Carpenter, B., Russakovsky, O., Ng, A.Y.: Autonomous operation of novel elevators for robot navigation. In: 2010 IEEE International Conference on Robotics and Automation, pp. 751–758. IEEE (2010)
Law, H., Deng, J.: CornerNet: detecting objects as paired keypoints. Int. J. Comput. Vision 128(3), 642–656 (2019). https://doi.org/10.1007/s11263-019-01204-1
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Macenski, S., Martín, F., White, R., Clavero, J.G.: The marathon 2: a navigation system. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2718–2725. IEEE (2020)
Menini, D., Kumar, S., Oswald, M.R., Sandström, E., Sminchisescu, C., Van Gool, L.: A real-time online learning framework for joint 3D reconstruction and semantic segmentation of indoor scenes. IEEE Robot. Autom. Lett. 7(2), 1332–1339 (2021)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems 28 (2015)
Song, J., Patel, M., Ghaffari, M.: Fusing convolutional neural network and geometric constraint for image-based indoor localization. IEEE Robot. Autom. Lett. 7(2), 1674–1681 (2022)
Vidanapathirana, K., Ramezani, M., Moghadam, P., Sridharan, S., Fookes, C.: LoGG3D-Net: locally guided global descriptor learning for 3d place recognition. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 2215–2221. IEEE (2022)
Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696 (2022)
Yang, P.Y., Chang, T.H., Chang, Y.H., Wu, B.F.: Intelligent mobile robot controller design for hotel room service with deep learning arm-based elevator manipulator. In: 2018 International Conference on System Science and Engineering (ICSSE), pp. 1–6. IEEE (2018)
Zhou, X., Wang, D., Krähenbühl, P.: Objects as points. arXiv preprint arXiv:1904.07850 (2019)
Zhu, D., Li, T., Ho, D., Zhou, T., Meng, M.Q.: A novel OCR-RCNN for elevator button recognition. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3626–3631. IEEE (2018)
Zhu, D., Min, Z., Zhou, T., Li, T., Meng, M.Q.H.: An autonomous eye-in-hand robotic system for elevator button operation based on deep recognition network. IEEE Trans. Instrum. Meas. 70, 1–13 (2020)
Acknowledgement
This work was supported by “Research of Elevator Indicator Recognition Technology for Indoor Autonomous Navigation” project funded by ROBOTIS Co. Ltd. and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2019-0-00079, No. 2022-0-00871, No. 2022-0-00612, No. 2022-0-00480).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shin, S., Lee, J.H., Noh, J., Choi, S. (2023). Robust Detection for Autonomous Elevator Boarding Using a Mobile Manipulator. In: Lu, H., Blumenstein, M., Cho, SB., Liu, CL., Yagi, Y., Kamiya, T. (eds) Pattern Recognition. ACPR 2023. Lecture Notes in Computer Science, vol 14406. Springer, Cham. https://doi.org/10.1007/978-3-031-47634-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-47634-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47633-4
Online ISBN: 978-3-031-47634-1
eBook Packages: Computer ScienceComputer Science (R0)