Abstract
Currently it is hard to develop UAV in civil environments, being simulation the best option to develop complex UAV missions with AI. To carry out useful AI training in simulation for real-world use, it is best to do it over a similar environment as the one a real UAV will work, with realistic objects in the scene of interest (buildings, vehicles, structures, etc.). This work aims to detect, reconstruct, and extract metadata from those objects. A UAV mission was developed, which automatically detects all objects in a given area using both simulated camera and 2D LiDAR, and then performs a detailed scan of each object. Later, a reconstruct process will create a 3D model for each one of those objects, along with a geo-referenced information layer that contains the object information. If applied on reality, this mission ease bringing real content to a digital twin, thus improving, and extending the simulation capabilities. Results show great potential even with the current budget specification sensors. Additional post-processing steps could reduce the resulting artefacts in the export of 3D objects. Code, dataset, and details are available on the project page: https://danielamigo.github.io/projects/soco22/.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Grigoropoulos, N., Lalis, S.: Flexible deployment and enforcement of flight and privacy restrictions for drone applications. In: 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W). pp. 110–117. IEEE, Valencia, Spain (2020)
Hildmann, H., Kovacs, E.: Review: using unmanned aerial vehicles (UAVs) as mobile sensing platforms (MSPs) for disaster response. Civil Secur. Public Saf. Drones 3, 59 (2019). https://doi.org/10.3390/drones3030059
Kakaletsis, E., et al.: Computer vision for autonomous UAV flight safety: an overview and a vision-based safe landing pipeline example. ACM Comput. Surv. 54, 1–37 (2022). https://doi.org/10.1145/3472288
Shah, S., Dey, D., Lovett, C., Kapoor, A.: AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. ArXiv170505065 Cs (2017)
Meier, L., Honegger, D., Pollefeys, M.: PX4: A node-based multithreaded open source robotics framework for deeply embedded platforms. In: 2015 IEEE International Conference on Robotics and Automation (ICRA). pp. 6235–6240. IEEE, Seattle, WA, USA (2015)
Alvey, B., Anderson, D.T., Buck, A., Deardorff, M., Scott, G., Keller, J.M.: Simulated photorealistic deep learning framework and workflows to accelerate computer vision and unmanned aerial vehicle research. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). pp. 3882–3891. IEEE, Montreal, BC, Canada (2021)
Cinar, Z.M., Nuhu, A.A., Zeeshan, Q., Korhan, O.: Digital twins for industry 4.0: a review. In: Calisir, F., Korhan, O. (eds.) GJCIE 2019. LNMIE, pp. 193–203. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-42416-9_18
Amigo, D., Pedroche, D.S., García, J., Molina, J.M.: Automatic individual tree detection from combination of aerial imagery, LiDAR and environment context. In: Sanjurjo González, H., Pastor López, I., García Bringas, P., Quintián, H., Corchado, E. (eds.) SOCO 2021. AISC, vol. 1401, pp. 294–303. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-87869-6_28
Amigo, D., Pedroche, D.S., García, J., Molina, J.M.: Automatic context learning based on 360 imageries triangulation and 3D LiDAR validation. In: 2021 24th International Conference on Information Fusion (FUSION), p. 8 (2021)
Ghamisi, P., et al.: Multisource and multitemporal data fusion in remote sensing: a comprehensive review of the state of the art. IEEE Geosci. Remote Sens. Mag. 7, 6–39 (2019). https://doi.org/10.1109/MGRS.2018.2890023
Ribeiro, L.G.: 3D Reconstruction of Civil Infrastructures from UAV Lidar point clouds. 71
Siebert, S., Teizer, J.: Mobile 3D mapping for surveying earthwork projects using an unmanned aerial vehicle (UAV) system. Autom. Constr. 41, 1–14 (2014). https://doi.org/10.1016/j.autcon.2014.01.004
Mentasti, S., Pedersini, F.: Controlling the flight of a drone and its camera for 3D reconstruction of large objects. Sensors 19, 2333 (2019). https://doi.org/10.3390/s19102333
Luhmann, T., Chizhova, M., Gorkovchuk, D.: Fusion of UAV and terrestrial photogrammetry with laser scanning for 3D reconstruction of historic churches in Georgia. Drones. 4, 53 (2020). https://doi.org/10.3390/drones4030053
Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum PointNets for 3D Object Detection from RGB-D Data. ArXiv171108488 Cs (2018)
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. ArXiv170602413 Cs (2017)
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D ShapeNets: A Deep Representation for Volumetric Shapes. ArXiv14065670 Cs (2015)
Acknowledgement
This research was partially funded by public research projects of Spanish Ministry of Science and Innovation, references PID2020-118249RB-C22 and PDC2021-121567-C22 - AEI/10.13039/501100011033, and by the Madrid Government (Comunidad de Madrid-Spain) under the Multiannual Agreement with UC3M in the line of Excellence of University Professors, reference EPUC3M17.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Amigo, D., García, J., Molina, J.M., Lizcano, J. (2023). UAV Simulation for Object Detection and 3D Reconstruction Fusing 2D LiDAR and Camera. In: García Bringas, P., et al. 17th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2022). SOCO 2022. Lecture Notes in Networks and Systems, vol 531. Springer, Cham. https://doi.org/10.1007/978-3-031-18050-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-18050-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18049-1
Online ISBN: 978-3-031-18050-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)