Abstract
Mobile robots are an important participant in today’s modern life, and have huge commercial application prospects in the fields of unmanned security inspection, logistics, express delivery, cleaning and medical disinfection. Since LiDAR is not affected by ambient light and can operate in a dark environment, localization and navigation based on LiDAR point clouds have become one of the basic modules of mobile robots. However, compared with traditional binocular vision images, the sparse, disordered and noisy point cloud poses a challenge to efficient and stable feature extraction. This makes the LiDAR-based SLAM have more significant cumulative errors, and poor consistency of the final map, which affects tasks such as positioning based on the prior point cloud map. In order to alleviate the above problems and improve the positioning accuracy, a semantic SLAM with human-in-the-loop is proposed. First, the interactive SLAM is introduced to optimize the point cloud pose to obtain a highly consistent point cloud map; then the point cloud segmentation model is trained by artificial semantic annotation to obtain the semantic information of a single frame of point cloud; finally, the positioning accuracy is optimized based on the point cloud semantics. The proposed system is validated on the local platform in an underground garage, without involving GPS or expensive measuring equipment.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Luo, Y., Binbin, S., Zheng, X.: Trends and challenges for population and health during population aging-china, 2015–2050. China CDC Weekly 3(28), 593 (2021)
Yang, G., et al.: Homecare robotic systems for healthcare 4.0: visions and enabling technologies. IEEE J. Biomed. Health Inform. 24(9), 2535–2549 (2020)
Zhang, J., Singh, S.: Loam: lidar odometry and mapping in real-time. In: Robotics: Science and Systems, Berkeley, CA, vol. 2, no. 9, pp. 1–9 (2014)
Song, C.K., Uchanski, M., Karl Hedrick, J.: Vehicle speed estimation using accelerometer and wheel speed measurements. Soc. of Automotive Engineers (2002)
Barbieri, L., Brambilla, M., Trabattoni, A., Mervic, S., Nicoli, M.: UWB localization in a smart factory: augmentation methods and experimental assessment. IEEE Trans. Instrum. Meas. 70, 1–18 (2021)
Wang, L., et al.: Initial assessment of the LEO based navigation signal augmentation system from Luojia-1a satellite. Sensors 18(11), 3919 (2018)
Li, Y., He, L., Zhang, X., Zhu, L., Zhang, H., Guan, Y.: Multi-sensor fusion localization of indoor mobile robot. In: 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 481–486. IEEE (2019)
Campos, C., Elvira, R., Gómez Rodríguez, J.J., Montiel, J.M.M., Tardós, J.D.: Orb-slam3: an accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Trans. Robot. 37(6), 1874–1890 (2021)
Shan, T., Englot, B.: Lego-loam: lightweight and ground-optimized lidar odometry and mapping on variable terrain. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4758–4765. IEEE (2018)
Cui, J., Niu, J., Ouyang, Z., He, Y., Liu, D.: ACSC: automatic calibration for non-repetitive scanning solid-state lidar and camera systems. arXiv preprint arXiv:2011.08516 (2020)
Chen, X., Milioto, A., Palazzolo, E., Giguère, P., Behley, J., Stachniss, C.: Suma++: efficient lidar-based semantic slam. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4530–4537. IEEE (2019)
Koide, K., Miura, J., Yokozuka, M., Oishi, S., Banno, A.: Interactive 3d graph slam for map correction. IEEE Robot. Autom. Lett. 6(1), 40–47 (2020)
Behley, J., et al.: Semantickitti: a dataset for semantic scene understanding of lidar sequences. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9297–9307 (2019)
Zhang, Y., et al.: PolarNet: an improved grid representation for online lidar point clouds semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9601–9610 (2020)
Zhang, L., Wei, L., Shen, P., Wei, W., Zhu, G., Song, J.: Semantic slam based on object detection and improved octomap. IEEE Access 6, 75545–75559 (2018)
Mur-Artal, R., Tardós, J.D.: Orb-slam2: an open-source slam system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Wang, Z., Zhang, Q., Li, J., Zhang, S., Liu, J.: A computationally efficient semantic slam solution for dynamic scenes. Remote Sen. 11(11), 1363 (2019)
Kang, X., Yuan, S.: Robust data association for object-level semantic slam. arXiv preprint arXiv:1909.13493 (2019)
Long, X., Zhang, W., Zhao, B.: Pspnet-slam: a semantic slam detect dynamic object by pyramid scene parsing network. IEEE Access (2020)
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)
Zhao, Z., Mao, Y., Ding, Y., Ren, P., Zheng, N.: Visual-based semantic slam with landmarks for large-scale outdoor environment. In: 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI), pp. 149–154. IEEE (2019)
Rosinol, A., Abate, M., Chang, Y., Carlone, L.: Kimera: an open-source library for real-time metric-semantic localization and mapping. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 1689–1696. IEEE (2020)
Fan, Y., et al.: Semantic slam with more accurate point cloud map in dynamic environments. IEEE Access 8, 112237–112252 (2020)
Mahe, H., Marraud, D., Comport, A.I.: Real-time RGB-D semantic keyframe slam based on image segmentation learning from industrial cad models. In: 2019 19th International Conference on Advanced Robotics (ICAR), pp.s 147–154. IEEE (2019)
Nicholson, L., Milford, M., Sünderhauf, N.: Quadricslam: constrained dual quadrics from object detections as landmarks in semantic slam. IEEE Robot. Autom. Lett. (RA-L) (2018)
Li, R., Wang, S., Dongbing, G.: Ongoing evolution of visual slam from geometry to deep learning: challenges and opportunities. Cogn. Comput. 10(6), 875–889 (2018)
Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)
Lehtonen, M., Ostojic, D., Ilic, A., Michahelles, F.: Securing RFID systems by detecting tag cloning. In: Tokuda, H., Beigl, M., Friday, A., Brush, A.J.B., Tobe, Y. (eds.) Pervasive 2009. LNCS, vol. 5538, pp. 291–308. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-01516-8_20
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
Dong, X., Niu, J., Cui, J., Fu, Z., Ouyang, Z.: Fast segmentation-based object tracking model for autonomous vehicles. In: Qiu, M. (ed.) ICA3PP 2020. LNCS, vol. 12453, pp. 259–273. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60239-0_18
Graeter, J., Wilczynski, A., Lauer, M.: Limo: lidar-monocular visual odometry. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7872–7879. IEEE (2018)
Jian, R., et al.: A semantic segmentation based lidar SLAM system towards dynamic environments. In: Yu, H., Liu, J., Liu, L., Ju, Z., Liu, Y., Zhou, D. (eds.) ICIRA 2019. LNCS (LNAI), vol. 11742, pp. 582–590. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27535-8_52
Zhao, Z., Zhang, W., Jianfeng, G., Yang, J., Huang, K.: Lidar mapping optimization based on lightweight semantic segmentation. IEEE Trans. Intell. Veh. 4(3), 353–362 (2019)
Milioto, A., Vizzo, I., Behley, J., Stachniss, C.: Rangenet++: fast and accurate lidar semantic segmentation. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4213–4220. IEEE (2019)
Chen, X.: et al.: Overlapnet: loop closing for lidar-based SLAM. In: Proceedings of the Robotics: Science and Systems (RSS), Freiburg, Germany, pp. 12–16 (2020)
Sun, L., Yan, Z., Zaganidis, A., Zhao, C., Duckett, T.: Recurrent-octomap: learning state-based map refinement for long-term semantic mapping with 3-d-lidar data. IEEE Robot. Autom. Lett. 3(4), 3749–3756 (2018)
Pan, Y., Gao, B., Mei, J., Geng, S., Li, C., Zhao, H.: Semanticposs: a point cloud dataset with large quantity of dynamic instances. arXiv preprint arXiv:2002.09147 (2020)
Hu, Q., et al.: Randla-net: efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11108–11117 (2020)
Zhu, X., et al.: Cylindrical and asymmetrical 3D convolution networks for lidar segmentation. arXiv preprint arXiv:2011.10033 (2020)
Chen, X., Vizzo, I., Läbe, T., Behley, J., Stachniss, C.: Range image-based lidar localization for autonomous vehicles. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 5802–5808. IEEE (2021)
Razlaw, J., Droeschel, D., Holz, D., Behnke, S.: Evaluation of registration methods for sparse 3d laser scans. In: 2015 European Conference on Mobile Robots (ECMR), pp. 1–7. IEEE (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Ouyang, Z., Zhang, C., Cui, J. (2022). Semantic SLAM for Mobile Robot with Human-in-the-Loop. In: Gao, H., Wang, X., Wei, W., Dagiuklas, T. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 461. Springer, Cham. https://doi.org/10.1007/978-3-031-24386-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-24386-8_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-24385-1
Online ISBN: 978-3-031-24386-8
eBook Packages: Computer ScienceComputer Science (R0)