Abstract
Robust rock and pothole detection is essential for autonomous exploration in challenging desert environments. The barren environment requires accurate traversability estimation based on the detected rocks and potholes in the wild navigation. However, the diverse shapes, scales, and occlusions of rocks and potholes make the detection challenging. Current single-sensor approaches (e.g., Camera or LiDAR methods) struggle to obtain accurate locations and landmarks for traversability estimation. However, the light and ground conditions captured by Cameras and the sparsity of LiDAR point cloud pose challenges for singles-sensor pipelines. Additionally, the undulating ground of the desert often makes the sensor calibration with errors, which has a significant influence on the accurate detection. To this end, we present DyFTNet, a LiDAR-Camera fusion-based 3D rock and pothole detection method that specially combines features from both modalities by dynamically correcting point cloud projection errors induced by desert surface irregularities. Subsequently, we construct a multi-modality 3D detection dataset in the desert environment, named U2Ground dataset, consisting of 5534 rocks and 2926 potholes within 1463 pairs of LiDAR-Camera frames. DyFTNet is verified on U2Ground with a comparison to the state-of-the-art methods and superior performance is obtained. The code and dataset will be available on the website https://github.com/Yyb-XJTU/U2Ground-dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Verma, V., et al.: Autonomous robotics is driving perseverance rover’s progress on mars. Sci. Robot. 8(80), eadi3099 (2023)
Karunasekera, H., Zhang, H., Xi, T., Wang, H.: Stereo vision based negative obstacle detection. In: IEEE International Conference on Control and Automation, pp. 834–838 (2017)
Matthies, L., et al.: Lunar rover localization using craters as landmarks. In: IEEE Aerospace Conference, pp. 1–17 (2022)
Ishida, T., Takahashi, M., Fukuda, S.: Crater detection robust to illumination and shape changes using convolutional neural network. Trans. Japan Society Aeronaut. Space Sci. 64(4), 197–204 (2021)
Tewari, A., Verma, V., Srivastava, P., Jain, V., Khanna, N.: Automated crater detection from co-registered optical images, elevation maps and slope maps using deep learning. Planet. Space Sci. 218, 105500 (2022)
Lin, X., et al.: Lunar crater detection on digital elevation model: A complete workflow using deep learning and its application. Remote Sens. 14(3), 621 (2022)
Shang, E., An, X., Li, J., He, H.: A novel setup method of 3d lidar for negative obstacle detection in field environment. In: IEEE Conference on Intelligent Transportation Systems, pp. 1436–1441 (2014)
Zhou, Y., Li, X., Hua, B.: Crater identification simulation using LiDAR on Lunar rover. Measurement 210, 112550 (2023)
Zhong, Z., Wang, Z., Lin, L., Liang, H., Xu, F.. Robust negative obstacle detection in off-road environments using multiple lidars. In: IEEE International Conference on Control, Automation and Robotics, pp. 700–705 (2020)
Goodin, C., Carrillo, J., Monroe, J.G., Carruth, D.W., Hudson, C.R.: An analytic model for negative obstacle detection with lidar and numerical validation using physics-based simulation. Sensors 21(9), 3211 (2021)
Qian, Y., Wang, X., Zhuang, H., Wang, C., Yang, M.: 3d vehicle detection enhancement using tracking feedback in sparse point clouds environments. IEEE Open J. Intell. Transport. Syst. 4, 471–480 (2023)
Bavirisetti, D.P., Martinsen, H.R., Kiss, G.H., Lindseth, F.: A multi-task vision transformer for segmentation and monocular depth estimation for autonomous vehicles. IEEE Open J. Intell. Transport. Syst. 4, 909–928 (2023)
Valiente, R., et al.: Robust perception and visual understanding of traffic signs in the wild. IEEE Open J. Intell. Transport. Syst. 4, 611–625 (2023)
Liu, Z., et al.: Bevfusion: multi-task multi-sensor fusion with unified bird’s-eye view representation. In: IEEE International Conference on Robotics and Automation, pp. 2774–2781 (2023)
Chen, Y., Li, Y., Zhang, X., Sun, J., Jia, J.: Focal sparse convolutional networks for 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5428–5437 (2022)
Li, Y., et al.: Voxel field fusion for 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1120–1129 (2022)
Khalil, Y.H., Mouftah, H.T.: Licanet: further enhancement of joint perception and motion prediction based on multi-modal fusion. IEEE Open J. Intell. Transport. Syst. 3, 222–235 (2022)
Sindagi, V.A., Zhou, Y. and Tuzel, O.: Mvx-net: Multimodal voxelnet for 3d object detection. In: IEEE International Conference on Robotics and Automation, pp. 7276–7282 (2019)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems 30 (2017)
Liang, T., et al.: Bevfusion: A simple and robust lidar-camera fusion framework. Adv. Neural. Inf. Process. Syst. 35, 10421–10434 (2022)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Yan, Y., Mao, Y., Li, B.: Second: Sparsely embedded convolutional detection. Sensors 18(10), 3337 (2018)
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European Conference on Computer Vision, pp. 213–229 (2020)
Geiger, A., Lenz, P., Urtasun, R.: The kitti vision benchmark suite. http://www.cvlibs.net/datasets/kitti2(5) (2015)
Zhou, Y., Tuzel, O.: Voxelnet: End-to-end learning for point cloud based 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4490–4499 (2018)
Zhou, Y., et al.: End-to-end multi-view fusion for 3d object detection in lidar point clouds. In: Conference on Robot Learning, pp. 923–932 (2020)
Kingma, DP., Jimmy, B.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yuan, Y. et al. (2025). 3D Rock and Pothole Detection in Desert for the Wild Navigation. In: Lan, X., Mei, X., Jiang, C., Zhao, F., Tian, Z. (eds) Intelligent Robotics and Applications. ICIRA 2024. Lecture Notes in Computer Science(), vol 15209. Springer, Singapore. https://doi.org/10.1007/978-981-96-0789-1_32
Download citation
DOI: https://doi.org/10.1007/978-981-96-0789-1_32
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-96-0788-4
Online ISBN: 978-981-96-0789-1
eBook Packages: Computer ScienceComputer Science (R0)