Skip to main content

GAP9Shield: A 150GOPS AI-Capable Ultra-low Power Module for Vision and Ranging Applications on Nano-drones

  • Conference paper
  • First Online:
European Robotics Forum 2024 (ERF 2024)

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 32))

Included in the following conference series:

  • 73 Accesses

Abstract

The evolution of AI and digital signal processing technologies, combined with affordable energy-efficient processors, has propelled the development of hardware and software for drone applications. Nano-drones, which fit into the palm of the hand, are suitable for indoor environments and safe for human interaction; however, they often fail to deliver the required performance for complex tasks due to the lack of hardware providing sufficient sensing and computing performance. Addressing this gap, we present the GAP9Shield, a nano-drone-compatible module powered by the GAP9, a 150GOPS-capable SoC. The system also includes a 5 MP camera for high-definition imaging, a Wi-Fi-BLE module, and a 5-directional laser-based ranging subsystem, enabling obstacle avoidance capabilities. Compared with similar state-of-the-art systems, GAP9Shield provides a 20% higher sample rate (RGB images) while offering 15% weight reduction. In this paper, we also highlight the energy efficiency and processing power capabilities of GAP9 for object detection using deep learning (YOLO), localization using a particle filter, and mapping, which can run within a power envelope of below 100 mW and at low latency (as 17 ms for object detection), highlighting the transformative potential of GAP9 for the new generation of nano-drone applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.bitcraze.io/products/crazyflie-2-1/.

  2. 2.

    https://greenwaves-technologies.com/gap9_processor/.

  3. 3.

    The AI and ranger decks together weigh about 7 g and occupy 6480 mm\(^3\).

References

  1. Palossi, D., Gomez, A., Draskovic, S., Keller, K., Benini, L., Thiele, L.: Self-sustainability in nano unmanned aerial vehicles: a blimp case study. In: Proceedings of the Computing Frontiers Conference (2017)

    Google Scholar 

  2. Birk, A., Wiggerich, B., Bülow, H., Pfingsthorn, M., Schwertfeger, S.: Safety, security, and rescue missions with an unmanned aerial vehicle (UAV). J. Intell. Robot. Syst. 64(1), 57–76 (2011)

    Article  MATH  Google Scholar 

  3. Palossi, D., et al.: Fully onboard AI-powered human-drone pose estimation on ultra-low power autonomous flying nano-UAVs. IEEE Internet Things J. 1 (2021)

    Google Scholar 

  4. Lamberti, L., Bompani, L., Kartsch, V.J., Rusci, M., Palossi, D., Benini, L.: Bio-inspired autonomous exploration policies with CNN-based object detection on nano-drones. In: Design, Automation & Test in Europe Conference & Exhibition (DATE), 1–6. IEEE (2023)

    Google Scholar 

  5. Zimmerman, N., Müller, H., Magno, M., Benini, L.: Fully onboard low-power localization with semantic sensor fusion on a nano-UAV using floor plans. In: 2024 IEEE International Conference on Robotics and Automation (ICRA), arXiv preprint arXiv:2310.12536 (2024)

  6. Moosmann, J., et al.: Ultra-efficient on-device object detection on AI-integrated smart glasses with tinyissimoyolo. arXiv preprint arXiv:2311.01057 (2023)

  7. Niculescu, V., Polonelli, T., Magno, M., Benini, L.: NanoSLAM: enabling fully onboard slam for tiny robots. IEEE Internet Things J. (2023)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Victor Kartsch .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Müller, H., Kartsch, V., Benini, L. (2024). GAP9Shield: A 150GOPS AI-Capable Ultra-low Power Module for Vision and Ranging Applications on Nano-drones. In: Secchi, C., Marconi, L. (eds) European Robotics Forum 2024. ERF 2024. Springer Proceedings in Advanced Robotics, vol 32. Springer, Cham. https://doi.org/10.1007/978-3-031-76424-0_52

Download citation

Publish with us

Policies and ethics