Skip to main content

Advertisement

SpringerLink
Log in

An integrated solution for an autonomous drone racing in indoor environments

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

The use of drones is becoming more present in modern daily life. One of the most challenging tasks associated with these vehicles is the development of perception and autonomous navigation systems. Competitions such as Artificial Intelligence Robotic Racing (AIRR) and Autonomous Drone Racing were launched to drive the advances in such strategies, requiring the development of integrated systems for autonomous navigation in a racing environment. In this context, this paper presents an improved integrated solution for autonomous drone racing, which focuses on simple, robust, and computationally efficient techniques to enable the application in embedded hardware. The strategy is divided into four modules: (i) A trajectory planner computes a path that passes through the sequence of desired gates; (ii) a perception system that obtains the global pose of the vehicle by using an onboard camera; (iii) a localization system which merges several sensed information to estimate the drone’s states; and, (iv) an artificial vector field-based controller in charge of following the plan by using the estimated states. To evaluate the performance of the entire integrated system, we extended the FlightGoggles simulator to use gates similar to those used in the AIRR competition. A computational cost analysis demonstrates the high performance of the proposed perception method running on limited hardware commonly embedded in aerial robots. In addition, we evaluate the localization system by comparing it with ground truth and when there is a disturbance in the location of the gates. Results in a representative race circuit based on the AIRR competition showed the proposed strategy’s competing performance, presenting itself as a feasible solution for drone racing systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Data availability

Algorithms and data sets can be made available upon acceptance and publication.

Notes

  1. https://lockheedmartin.com/en-us/index.html.

  2. https://thedroneracingleague.com/.

  3. https://herox.com/alphapilot.

  4. https://thedroneracingleague.com/airr/.

  5. https://www.herox.com/alphapilot/82-teams.

  6. http://www.xquadufmg.com/.

  7. https://github.com/AlexeyAB/darknet.

  8. https://github.com/raulmur/ORB_SLAM2

  9. http://wiki.ros.org/robot_pose_ekf

  10. https://github.com/xquadufmg/FlightGoggles.

  11. https://github.com/xquadufmg/xquad_autonomous_stack.

  12. https://youtu.be/B5_5yvwiW0E.

  13. https://pypi.org/project/cvxopt/.

  14. https://pypi.org/project/opencv-python/.

  15. https://github.com/appliedAI-Initiative/orb_slam_2_ros.

References

  1. Al Younes Y, Drak A, Noura H, Rabhi A, El Hajjaji A (2016) Robust model-free control applied to a quadrotor UAV. J Intell Robot Syst 84(1–4):37–52

    Article  Google Scholar 

  2. Araar O, Aouf N (2014) Visual servoing of a quadrotor uav for autonomous power lines inspection. In: 22nd mediterranean conference on control and automation. IEEE, pp 1418–1424

  3. Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:200410934

  4. Cimpoi M, Maji S, Kokkinos I, Mohamed S, Vedaldi A (2014) Describing textures in the wild. In: Proceedings of the IEEE conf. on computer vision and pattern recognition (CVPR)

  5. Cocoma-Ortega JA, Martinez-Carranza J (2019) A CNN based drone localisation approach for autonomous drone racing. In: Proceedings of the 11th international micro air vehicle competition and conference, Madrid, Spain, vol 30

  6. Cyba A, Szolc H, Kryjak T (2021) A simple vision-based navigation and control strategy for autonomous drone racing. arXiv preprint arXiv:210409815

  7. Deng H, Fu Q, Quan Q, Yang K, Cai KY (2019) Indoor multi-camera based testbed for 3D tracking and control of UAVs. IEEE Trans Instrum Meas 69(6):3139–3156

    Article  Google Scholar 

  8. Dolatabadi SH, Yazdanpanah MJ (2015) MIMO sliding mode and backstepping control for a quadrotor UAV. In: 2015 23rd Iranian conference on electrical engineering. IEEE, pp 994–999

  9. Durdevic P, Ortiz-Arroyo D, Li S, Yang Z (2019) Vision aided navigation of a quad-rotor for autonomous wind-farm inspection. IFAC-PapersOnLine 52(8):61–66

    Article  Google Scholar 

  10. Falanga D, Mueggler E, Faessler M, Scaramuzza D (2017) Aggressive quadrotor flight through narrow gaps with onboard sensing and computing using active vision. In: 2017 IEEE international conference on robotics and automation (ICRA), pp 5774–5781, https://doi.org/10.1109/ICRA.2017.7989679

  11. Falanga D, Foehn P, Lu P, Scaramuzza D (2018) Pampc: perception-aware model predictive control for quadrotors. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS). pp 1–8, https://doi.org/10.1109/IROS.2018.8593739

  12. Foehn P, Brescianini D, Kaufmann E, Cieslewski T, Gehrig M, Muglikar M, Scaramuzza D (2020) Alphapilot: autonomous drone racing. Robot Sci Syst

  13. Foehn P, Romero A, Scaramuzza D (2021) Time-optimal planning for quadrotor waypoint flight. Sci Robot. https://doi.org/10.1126/scirobotics.abh1221

    Article  Google Scholar 

  14. Gao XS, Hou XR, Tang J, Cheng HF (2003) Complete solution classification for the perspective-three-point problem. IEEE Trans Pattern Anal Mach Intell 25(8):930–943

    Article  Google Scholar 

  15. Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognit 47(6):2280–2292

    Article  Google Scholar 

  16. Geiger A, Moosmann F, Car Ö, Schuster B (2012) Automatic camera and range sensor calibration using a single shot. In: 2012 IEEE international conference on robotics and automation. IEEE, pp 3936–3943

  17. Gonçalves VM, Pimenta LC, Maia CA, Dutra BC, Pereira GA (2010) Vector fields for robot navigation along time-varying curves in \(n\)-dimensions. IEEE Trans Robot 26:647–659

    Article  Google Scholar 

  18. Guerra W, Tal E, Murali V, Ryou G, Karaman S (2019) Flightgoggles: photorealistic sensor simulation for perception-driven robotics using photogrammetry and virtual reality. In: 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). pp 6941–6948, https://doi.org/10.1109/IROS40897.2019.8968116

  19. Han Z, Wang Z, Xu C, Gao F (2021) Fast-racing: an open-source strong baseline for SE(3) planning in autonomous drone racing. arXiv preprint arXiv:210510276

  20. Hu F, Cheng J, Bao Y, He Y (2020) Accuracy enhancement for the front-end tracking algorithm of rgb-d slam. Intell Serv Robot 13(2):207–218

    Article  Google Scholar 

  21. Huang AS, Bachrach A, Henry P, Krainin M, Maturana D, Fox D, Roy N (2017) Visual odometry and mapping for autonomous flight using an rgb-d camera. Robotics research. Springer, New York, pp 235–252

    Chapter  Google Scholar 

  22. Jayasinghe J, Athauda A (2016) Smooth trajectory generation algorithm for an unmanned aerial vehicle (UAV) under dynamic constraints: using a quadratic bezier curve for collision avoidance. In: 2016 manufacturing & industrial engineering symposium (MIES). IEEE, pp 1–6

  23. Jing X, Cui J, He H, Zhang B, Ding D, Yang Y (2017) Attitude estimation for UAV using extended kalman filter. In: 2017 29th Chinese control and decision conference (CCDC). IEEE, pp 3307–3312

  24. Jung S, Hwang S, Shin H, Shim DH (2018) Perception, guidance, and navigation for indoor autonomous drone racing using deep learning. IEEE Robot Autom Lett 3(3):2539–2544

    Article  Google Scholar 

  25. Karaduman M, Çınar A, Eren H (2019) UAV traffic patrolling via road detection and tracking in anonymous aerial video frames. J Intell Robot Syst 95(2):675–690

    Article  Google Scholar 

  26. Karaman S, Frazzoli E (2011) Sampling-based algorithms for optimal motion planning. Int J Robot Res 30(7):846–894

    Article  Google Scholar 

  27. Kaufmann E, Gehrig M, Foehn P, Ranftl R, Dosovitskiy A, Koltun V, Scaramuzza D (2019) Beauty and the beast: optimal methods meet learning for drone racing. In: 2019 international conference on robotics and automation (ICRA). pp 690–696, https://doi.org/10.1109/ICRA.2019.8793631

  28. Kaufmann E, Loquercio A, Ranftl R, Müller M, Koltun V, Scaramuzza D (2020) Deep drone acrobatics. arXiv preprint arXiv:200605768

  29. Koch W, Mancuso R, West R, Bestavros A (2019) Reinforcement learning for UAV attitude control. ACM Trans Cyber-Phys Syst 3(2):1–21

    Article  Google Scholar 

  30. Kwon W, Park JH, Lee M, Her J, Kim SH, Seo JW (2019) Robust autonomous navigation of unmanned aerial vehicles (UAVs) for warehouses inventory application. IEEE Robot Autom Lett 5(1):243–249

    Article  Google Scholar 

  31. Lee KS, Ovinis M, Nagarajan T, Seulin R, Morel O (2015) Autonomous patrol and surveillance system using unmanned aerial vehicles. In: 2015 IEEE 15th international conference on environment and electrical engineering (EEEIC). IEEE, pp 1291–1297

  32. Lee T, Leok M, McClamroch NH (2010) Geometric tracking control of a quadrotor UAV on SE(3). In: 49th IEEE conference on decision and control (CDC). pp 5420–5425. https://doi.org/10.1109/CDC.2010.5717652

  33. Li S, van der Horst E, Duernay P, De Wagter C, de Croon G (2019) Visual model-predictive localization for computationally efficient autonomous racing of a 72-gram drone. arxiv 2019. arXiv preprint arXiv:190510110

  34. Li S, Ozo MM, De Wagter C, de Croon GC (2020) Autonomous drone race: a computationally efficient vision-based navigation and control strategy. Robot Auton Syst 133:103621. https://doi.org/10.1016/j.robot.2020.103621

    Article  Google Scholar 

  35. Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vis 128(2):261–318

    Article  Google Scholar 

  36. Loquercio A, Kaufmann E, Ranftl R, Dosovitskiy A, Koltun V, Scaramuzza D (2019) Deep drone racing: from simulation to reality with domain randomization. IEEE Trans Robot 36(1):1–14

    Article  Google Scholar 

  37. Mellinger D, Kumar V (2011) Minimum snap trajectory generation and control for quadrotors. In: 2011 IEEE international conference on robotics and automation. pp 2520–2525. https://doi.org/10.1109/ICRA.2011.5980409

  38. Moon H, Martinez-Carranza J, Cieslewski T, Faessler M, Falanga D, Simovic A, Scaramuzza D, Li S, Ozo M, De Wagter C et al (2019) Challenges and implemented technologies used in autonomous drone racing. Intell Serv Robot 12(2):137–148

    Article  Google Scholar 

  39. Morrell B, Rigter M, Merewether G, Reid R, Thakker R, Tzanetos T, Rajur V, Chamitoff G (2018) Differential flatness transformations for aggressive quadrotor flight. In: 2018 IEEE international conference on robotics and automation (ICRA). pp 5204–5210. https://doi.org/10.1109/ICRA.2018.8460838

  40. Mur-Artal R, Tardós JD (2017) ORB-SLAM2: an open-source SLAM system for monocular, stereo and RGB-D cameras. IEEE Trans Robot 33(5):1255–1262. https://doi.org/10.1109/TRO.2017.2705103

    Article  Google Scholar 

  41. Mur-Artal R, Montiel JM, Tardós JD (2015) ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans Robot 31(5):1147–1163. https://doi.org/10.1109/TRO.2015.2463671

    Article  Google Scholar 

  42. Mustafah YM, Azman AW, Akbar F (2012) Indoor UAV positioning using stereo vision sensor. Procedia Eng 41:575–579

    Article  Google Scholar 

  43. Pfeiffer C, Scaramuzza D (2021) Human-piloted drone racing: visual processing and control. IEEE Robot Autom Lett 6(2):3467–3474. https://doi.org/10.1109/LRA.2021.3064282

    Article  Google Scholar 

  44. Raffo GV, de Almeida MM (2016) Nonlinear robust control of a quadrotor uav for load transportation with swing improvement. In: 2016 American control conference (ACC). IEEE, pp 3156–3162

  45. Rezende AM, Miranda VR, Machado HN, Chiella AC, Gonçalves VM, Freitas GM (2019) Autonomous system for a racing quadcopter. In: 2019 19th International conference on advanced robotics (ICAR). IEEE

  46. Rezende AM, Gonçalves VM, Pimenta LC (2020) Robust quadcopter control with artificial vector fields. In: 2020 IEEE/RSJ international conference on robotics and automation (ICRA). IEEE, pp 6381–6387

  47. Richter C, Bry A, Roy N (2016) Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments. Robotics research. Springer, New York, pp 649–666

    Chapter  Google Scholar 

  48. Rojas-Perez LO, Martinez-Carranza J (2021) On-board processing for autonomous drone racing: an overview. Integration 80:46–59. https://doi.org/10.1016/j.vlsi.2021.04.007

    Article  Google Scholar 

  49. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: 2011 International conference on computer vision. IEEE, pp 2564–2571

  50. Saadeddin K, Abdel-Hafez MF, Jarrah MA (2014) Estimating vehicle state by GPS/IMU fusion with vehicle dynamics. J Intell Robot Syst 74(1–2):147–172

    Article  Google Scholar 

  51. Sayre-McCord T, Guerra W, Antonini A, Arneberg J, Brown A, Cavalheiro G, Fang Y, Gorodetsky A, McCoy D, Quilter S, et al. (2018) Visual-inertial navigation algorithm development using photorealistic camera simulation in the loop. In: 2018 IEEE international conference on robotics and automation (ICRA). IEEE, pp 2566–2573

  52. Schauwecker K, Zell A (2013) On-board dual-stereo-vision for autonomous quadrotor navigation. In: 2013 international conference on unmanned aircraft systems (ICUAS). IEEE, pp 333–342

  53. Song Y, Steinweg M, Kaufmann E, Scaramuzza D (2021) Autonomous drone racing with deep reinforcement learning. arXiv preprint arXiv:210308624

  54. Sung Y, Kwak J, Jeong YS, Park JH (2016) Beacon distance measurement method in indoor ubiquitous computing environment. Advances in parallel and distributed computing and ubiquitous services. Springer, New York, pp 125–130

    Chapter  Google Scholar 

  55. Thrun S, Burgard W, Fox D (2005) Probabilistic robotics. MIT press, Cambridge

    MATH  Google Scholar 

  56. Urzua S, Munguía R, Grau A (2019) Monocular slam system for MAVs aided with altitude and range measurements: a GPS-free approach. J Intell Robot Syst 94(1):203–217

    Article  Google Scholar 

  57. Vaidis M, Otis MJD (2020) Toward a robot swarm protecting a group of migrants. Intell Serv Robot 1–16

  58. Villa DK, Brandao AS, Sarcinelli-Filho M (2019) A survey on load transportation using multirotor uavs. J Intell Robot Syst 1–30

  59. Wang T, Chang DE (2021) Robust navigation for racing drones based on imitation learning and modularization. arXiv preprint arXiv:210512923

  60. Yang H, Lee Y, Jeon SY, Lee D (2017) Multi-rotor drone tutorial: systems, mechanics, control and state estimation. Intell Serv Robot 10(2):79–93

    Article  Google Scholar 

  61. Yousefi F, Monfared SB (2018) Cascade feedback linearization controller with actuators dynamic for trajectory tracking of flying robot. In: 2018 8th conference of AI & robotics and 10th RoboCup Iranopen international symposium (IRANOPEN). IEEE, pp 46–51

  62. Zhou D, Schwager M (2014) Vector field following for quadrotors using differential flatness. In: 2014 IEEE international conference on robotics and automation (ICRA). pp 6567–6572, https://doi.org/10.1109/ICRA.2014.6907828

Download references

Acknowledgements

This work was developed with the financial support of Coordenação de Aperfeiçamento de Pessoal de Nível Superiror (CAPES), Conselho Nacional de Desenvolvimento Cientíifico e Tecnológico (CNPq), the companies Vale S.A., Instituto Tecnológico Vale (ITV), and DTI Digital. From the Universidade Federal de Minas Gerais (UFMG), we are thankful for the support of Graduate Program in Electrical Engineering (PPGEE), Department of Electrical Engineering (DEE), Department of Computer Science (DCC), and the School of Engineering.

Funding

Universidade Federal de Minas Gerais (UFMG) Instituto Tecnológico Vale (ITV); Vale S.A.; Conselho Nacional de Desenvolvimento Cientíifico e Tecnológico (CNPq).

Author information

Authors and Affiliations

  1. Graduate Program in Electrical Engineering, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, MG, Brazil

    Adriano M. C. Rezende & Victor R. F. Miranda

  2. Department of Electrical Engineering, UFMG, Belo Horizonte, MG, Brazil

    Vinicius M. Gonçalves & Gustavo M. Freitas

  3. Computer Vision and Robotics Laboratory (VeRLab), Department of Computer Science, UFMG, Belo Horizonte, MG, Brazil

    Paulo A. F. Rezeck, Héctor Azpúrua, Elerson R. S. Santos & Douglas G. Macharet

  4. Instituto Tecnológico Vale (ITV), Ouro Preto, MG, Brazil

    Héctor Azpúrua

Authors
  1. Adriano M. C. Rezende
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Victor R. F. Miranda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paulo A. F. Rezeck
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Héctor Azpúrua
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elerson R. S. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vinicius M. Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Douglas G. Macharet
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Gustavo M. Freitas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adriano M. C. Rezende.

Ethics declarations

Conflict of interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rezende, A.M.C., Miranda, V.R.F., Rezeck, P.A.F. et al. An integrated solution for an autonomous drone racing in indoor environments. Intel Serv Robotics 14, 641–661 (2021). https://doi.org/10.1007/s11370-021-00385-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-021-00385-4

Keywords

Search

Navigation