Abstract
Autonomous nano-drones (\(\sim \)10 cm in diameter), thanks to their ultra-low power TinyML-based brains, are capable of coping with real-world environments. However, due to their simplified sensors and compute units, they are still far from the sense-and-act capabilities shown in their bigger counterparts. This system paper presents a novel deep learning-based pipeline that fuses multi-sensorial input (i.e., low-resolution images and \(8\times 8\) depth map) with the robot’s state information to tackle a human pose estimation task. Thanks to our design, the proposed system – trained in simulation and tested on a real-world dataset – improves a state-unaware State-of-the-Art baseline by increasing the \(R^2\) regression metric up to 0.10 on the distance’s prediction.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Clark, R., et al.: VINet: visual-inertial odometry as a sequence-to-sequence learning problem. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1 (2017)
Crupi, L., et al.: Sim-to-real vision-depth fusion CNNs for robust pose estimation aboard autonomous nano-quadcopters. In: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7711–7717 (2023)
Han, L., et al.: DeepVIO: self-supervised deep learning of monocular visual inertial odometry using 3D geometric constraints. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6906–6913 (2019)
Kaufmann, E., et al.: Beauty and the beast: optimal methods meet learning for drone racing. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 690–696. IEEE (2019)
Liu, C., et al.: Nano quadcopter obstacle avoidance with a lightweight monocular depth network. IFAC-PapersOnLine 56(2), 9312–9317 (2023). 22nd IFAC World Congress
Palossi, D., et al.: Self-sustainability in nano unmanned aerial vehicles: a blimp case study. In: Proceedings of the Computing Frontiers Conference, pp. 79–88 (2017)
Palossi, D., et al.: Ultra low-power visual odometry for nano-scale unmanned aerial vehicles. In: Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1647–1650 (2017)
Shakhatreh, H., et al.: Unmanned Aerial Vehicles (UAVs): a survey on civil applications and key research challenges. IEEE Access: Pract. Innov. Open Solutions 7, 48572–48634 (2019)
Zhang, N., et al.: End-to-end nano-drone obstacle avoidance for indoor exploration. Drones 8(2), 33 (2024)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Crupi, L., Cereda, E., Palossi, D. (2024). Fusing Multi-sensor Input with State Information on TinyML Brains for Autonomous Nano-drones. In: Secchi, C., Marconi, L. (eds) European Robotics Forum 2024. ERF 2024. Springer Proceedings in Advanced Robotics, vol 32. Springer, Cham. https://doi.org/10.1007/978-3-031-76424-0_53
Download citation
DOI: https://doi.org/10.1007/978-3-031-76424-0_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-76423-3
Online ISBN: 978-3-031-76424-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)