Abstract
In order to enable the unmanned motion platform to obtain real-time environmental semantic information and obstacle depth information, a real-time semantic segmentation and feature point matching based on binocular cameras are considered. This method firstly takes advantages of a real-time semantic segmentation network to obtain the road scene information and the region of obstacles on the road such as vehicles or pedestrians. Then, feature matching is performed on the region of interest (ROI) of left and right views. In the experiment part, firstly we conduct simulation verification on the KITTI dataset, and then we conduct binocular camera calibration, rectification, segmentation and stereo matching based on Oriented FAST and Rotated BRIEF (ORB) method on the actual system. The experiment results proves that the method is real-time and robust.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Chen, Z., Li, J., Wang, J., Wang, S., Zhao, J., Li, J.: Towards hybrid gait obstacle avoidance for a six wheel-legged robot with payload transportation. J. Intell. Robot. Syst. 102(3), 1–21 (2021)
Chen, Z., Li, J., Wang, S., Wang, J., Ma, L.: Flexible gait transition for six wheel-legged robot with unstructured terrains. Robot. Auton. Syst. 150, 103989 (2022)
Dai, Y., Li, J., Wang, J., Li, J.: Towards extreme learning machine framework for lane detection on unmanned mobile robot. Assem. Autom. 42(3), 361–371 (2022)
Dai, Y., Wang, J., Li, J., Li, J.: Mdrnet: a lightweight network for real-time semantic segmentation in street scenes. Assem. Autom. 41(6), 725–733 (2021)
Fetić, A., Jurić, D., Osmanković, D.: The procedure of a camera calibration using camera calibration toolbox for matlab. In: 2012 Proceedings of the 35th International Convention MIPRO, pp. 1752–1757 (2012)
Hirschmuller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 2, pp. 807–814. IEEE (2005)
Huang, H., Yang, C., Chen, C.L.P.: Optimal robot environment interaction under broad fuzzy neural adaptive control. IEEE Trans. Cybern. 51(7), 3824–3835 (2021)
Li, J., Li, R., Li, J., Wang, J., Wu, Q., Liu, X.: Dual-view 3D object recognition and detection via lidar point cloud and camera image. Robot. Auton. Syst. 150, 103999 (2022)
Li, J., Qin, H., Wang, J., Li, J.: Openstreetmap-based autonomous navigation for the four wheel-legged robot via 3D-lidar and CCD camera. IEEE Trans. Industr. Electron. 69(3), 2708–2717 (2022)
Li, J., Wang, J., Peng, H., Hu, Y., Su, H.: Fuzzy-torque approximation-enhanced sliding mode control for lateral stability of mobile robot. IEEE Trans. Syst. Man Cybern. Syst. 52(4), 2491–2500 (2022)
Li, J., Wang, J., Peng, H., Zhang, L., Hu, Y., Su, H.: Neural fuzzy approximation enhanced autonomous tracking control of the wheel-legged robot under uncertain physical interaction. Neurocomputing 410, 342–353 (2020)
Li, J., et al.: Parallel structure of six wheel-legged robot trajectory tracking control with heavy payload under uncertain physical interaction. Assem. Autom. 40(5), 675–687 (2020)
Li, J., Wang, J., Wang, S., Yang, C.: Human-robot skill transmission for mobile robot via learning by demonstration. Neural Comput. Appl. 1–11 (2021). https://doi.org/10.1007/s00521-021-06449-x
Li, J., Wang, S., Wang, J., Li, J., Zhao, J., Ma, L.: Iterative learning control for a distributed cloud robot with payload delivery. Assem. Autom. 41(3), 263–273 (2021)
Li, J., Zhang, X., Li, J., Liu, Y., Wang, J.: Building and optimization of 3d semantic map based on lidar and camera fusion. Neurocomputing 409, 394–407 (2020)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2015)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Menze, M., Heipke, C., Geiger, A.: Joint 3D estimation of vehicles and scene flow. ISPRS Ann. Photogram. Rem. Sens. Spat. Inf. Sci. 2, 427 (2015)
Paszke, A., Chaurasia, A., Kim, S., Culurciello, E.: Enet: a deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147 (2016)
Peng, G., Yang, C., He, W., Chen, C.P.: Force sensorless admittance control with neural learning for robots with actuator saturation. IEEE Trans. Ind. Electron. 67(4), 3138–3148 (2019)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient alternative to sift or surf. In :2011 International Conference on Computer Vision, pp. 2564–2571. IEEE (2011)
S. Wang, Z. Chen, J. Li, J. Wang, J. Li, and J. Zhao. Flexible motion framework of the six wheel-legged robot: experimental results. IEEE/ASME Transactions on Mechatronics, pages 1–9, 2021
Yang, C., Peng, G., Cheng, L., Na, J., Li, Z.: Force sensorless admittance control for teleoperation of uncertain robot manipulator using neural networks. IEEE Trans. Syst. Man Cybernet. Syst. 51(5), 3282–3292 (2021)
Yu, C., Gao, C., Wang, J., Yu, G., Shen, C., Sang, N.: Bisenet v2: bilateral network with guided aggregation for real-time semantic segmentation. Int. J. Comput. Vision 129(11), 3051–3068 (2021)
Z. Zhang. Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 666–673 (1999)
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6230–6239 (2017)
Acknowledgements
This work was supported by the National Key Research and Development Program of China under Grant 2019YFC1511401, and the National Natural Science Foundation of China under Grant 62173038.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, X., Wang, J., Li, J. (2022). Road Environment Perception for Unmanned Motion Platform Based on Binocular Vision. In: Liu, H., et al. Intelligent Robotics and Applications. ICIRA 2022. Lecture Notes in Computer Science(), vol 13455. Springer, Cham. https://doi.org/10.1007/978-3-031-13844-7_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-13844-7_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13843-0
Online ISBN: 978-3-031-13844-7
eBook Packages: Computer ScienceComputer Science (R0)