Skip to main content

Advertisement

SpringerLink
Log in

A Vision-Based Approach for Unmanned Aerial Vehicle Landing

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

In this paper we present an on-board Computer Vision System for the pose estimation of an Unmanned Aerial Vehicle (UAV) with respect to a human-made landing target. The proposed methodology is based on a coarse-to-fine approach to search the target marks starting from the recognition of the characteristics visible from long distances, up to the inner details when short distances require high precisions for the final landing phase. A sequence of steps, based on a Point-to-Line Distance method, analyzes the contour information and allows the recognition of the target also in cluttered scenarios. The proposed approach enables to fully assist the UAV during its take-off and landing on the target, as it is able to detect anomalous situations, such as the loss of the target from the image field of view, and the precise evaluation of the drone attitude when only a part of the target remains visible in the image plane. Several indoor and outdoor experiments have been carried out to demonstrate the effectiveness, robustness and accuracy of developed algorithm. The outcomes have proven that our methodology outperforms the current state of art, providing high accuracies in estimating the position and the orientation of landing target with respect to the UAV.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Al-Kaff, A., Martn, D., Garca, F., de la Escalera, A., Armingol, J.M.: Survey of computer vision algorithms and applications for unmanned aerial vehicles. Expert Syst. Appl. 92(C), 447–463 (2017)

    Google Scholar 

  2. Balamurugan, G., Valarmathi, J., Naidu, V.P.S.: Survey on Uav Navigation in Gps Denied Environments. In: IEEE International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES), 198–204 (2016)

  3. Benini, A., Rutherford, M.J., Valavanis, K.P.: Real-Time, Gpu-Based Pose Estimation of a Uav for Autonomous Takeoff and Landing. In: IEEE International Conference on Robotics and Automation (ICRA), 3463–3470 (2016)

  4. Bonak, M., Matko, D.: Blaic: Quadrocopter hovering using position-estimation information from inertial sensors and a high-delay video system. J. Intell. Robot. Syst. 67(1), 43–60 (2012)

    Article  Google Scholar 

  5. Camera, V.: Available from: http://www.vicon.com/products/camera-systems/bonita/. Accessed: 2018-01-08

  6. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)

    Article  Google Scholar 

  7. Selvaraj, Chmaj G.: H.: Distributed processing applications for uav/drones: a survey. Progress in Systems Engineering, pp. 449–454 (2015)

  8. Devernay, F., Faugeras, O.: Straight lines have to be straight. Mach. Vis. Appl. 13(1), 14–24 (2001)

    Article  Google Scholar 

  9. Drone, D.F.G.: Available from: http://www.helipal.com/dji-flamewheel-f550-w-naza-v2-gps-drone-rtf.html. Accessed: 2018-01-08

  10. Garca, G.B., Suarez, O.D., Aranda, J.L.E., Tercero, J.S., Gracia, I.S., Enano, N.V.: Learning Image Processing with Opencv. In: Packt Publishing Ltd, Ref. Page 60 (2015)

  11. Gautam, A., Sujit, P.B., Saripalli, S.: A Survey of Autonomous Landing Techniques for Uavs. in Unmanned Aircraft Systems (Icuas). In: IEEE International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1210–1218 (2014)

  12. Hilbert, D., Cohn-Vossen, S.: Geometry and the Imagination. In: American Mathematical Soc, 87 (1999)

  13. James, A., Amrutha, V., Haridasan, P.: Vision based algorithm for automatic landing system of unmanned aerial vehicles: a review. Int. J. Innovat. Sci. Eng. Technol. (IJISET) 2(4), 1126–1129 (2015)

    Google Scholar 

  14. Kanatani, K., Sugaya, Y., Kanazawa, Y.: Ellipse Analysis and 3D Computation of Circles. In: Guide to 3D Vision Computation, pp 117–129. Springer, Cham (2016)

  15. Kanellakis, C., Nikolakopoulos, G.: Survey on computer vision for uavs: Current developments and trends. J. Intell. Robot. Syst. 87(1), 1–28 (2017)

    Article  Google Scholar 

  16. Kendoul, F.: Survey of advances in guidance, navigation, and control of unmanned rotorcraft systems. J. Field Rob. 29(2), 315–378 (2012)

    Article  Google Scholar 

  17. Kim, J., Lee, Y.S., Han, S.S., Kim, S.H., Lee, G.H., Ji, H.J., Choi, K.N.: Autonomous Flight System Using Marker Recognition on Drone. In: 21St Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV), IEEE, 1–4 (2015)

  18. Kong, W., Zhou, D., Zhang, D., Zhang, J.: Vision-Based Autonomous Landing System for Unmanned Aerial Vehicle: a Survey. In: IEEE International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI), 1–8 (2014)

  19. Lange, S., Sunderhauf, N., Protzel, P.: A Vision Based Onboard Approach for Landing and Position Control of an Autonomous Multirotor Uav in Gps-Denied Environments. In: IEEE International Conference on Advanced Robotics (ICAR), 1–6 (2009)

  20. Lee, S., Jang, J.W., Baek, K.R.: Implementation of Vision-Based Real Time Helipad Detection System. In: 12Th IEEE International Conference on Control, Automation and Systems (ICCAS), 191–194 (2012)

  21. Lensation: Available from: https://www.lensation.de/product/bt3620/. Accessed: 2018-01-08

  22. Lin, S., Garratt, M.A., Lambert, A.J.: Monocular vision-based real-time target recognition and tracking for autonomously landing an uav in a cluttered shipboard environment. Auton. Robot. 41(4), 881–901 (2017)

    Article  Google Scholar 

  23. Mellinger, D., Michael, N., Kumar, V.: Trajectory generation and control for precise aggressive maneuvers with quadrotors. Int. J. Robot. Res. 31(5), 664–674 (2012)

    Article  Google Scholar 

  24. Menold, P., Pearson, R., Allgower, F.: Online outlier detection and removal. In: Proceedings 7th Mediterranean Conference Control and Automation (MED), 1110–1133 (1999)

  25. Moriarty, P., Sheehy, R., Doody, P.: Neural Networks to Aid the Autonomous Landing of a Uav on a Ship. In: IEEE 28Th Irish Signals and Systems Conference (ISSC), 1–4 (2017)

  26. Mth, K., Busoniu, L.: Vision and control for uavs: a survey of general methods and of inexpensive platforms for infrastructure inspection. Sensors 15(7), 14887–14916 (2015)

    Article  Google Scholar 

  27. Mukadam, K., Sinh, A., Karani, R.: Detection of Landing Areas for Unmanned Aerial Vehicles. In: IEEE International Conference on Computing Communication Control and Automation (ICCUBEA), 1–5 (2016)

  28. ODROID-XU4: Available from: http://www.hardkernel.com/main/products/prdt_info.php. Accessed: 2018-01-08

  29. de Oliveira, C.S., Anvar, A.P., Anvar, A., Silva, Jr, M.C., Neto, A.A., Mozelli, L.A.: Comparison of Cascade Classifiers for Automatic Landing Pad Detection in Digital Images. In: Simposio Brasileiro De Automacao Ineligente (2015)

  30. OpenCV: Available from: https://opencv.org/. Accessed: 2018-01-08

  31. Patruno, C., Nitti, M., Stella, E., D’Orazio, T.: Helipad detection for accurate uav pose estimation by means of a visual sensor. Int. J. Adv. Robot. Syst. 14(5), 1–15 (2017)

    Article  Google Scholar 

  32. Prakash, R.O., Saravanan, C.: Autonomous Robust Helipad Detection Algorithm Using Computer Vision. In: IEEE International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT), 2599–2604 (2016)

  33. Rilling, S., Nielsen, M., Milella, A., Jestel, C., Frhlich, P., Reina, G.: A multisensor platform for comprehensive detection of crop status: results from two case studies. In: 14Th IEEE international conference on advanced video and signal based surveillance (AVSS), 1–6 (2017)

  34. Sanchez-Lopez, J.L., Pestana, J., Saripalli, S., Campoy, P.: An approach toward visual autonomous ship board landing of a vtol uav. J. Intell. Robot. Syst. 74(1-2), 113–127 (2014)

    Article  Google Scholar 

  35. Sanchez-Lopez, J.L., Saripalli, S., Campoy, P., Pestana, J., Fu, C.: Toward Visual Autonomous Ship Board Landing of a Vtol Uav. In: IEEE International Conference on Unmanned Aircraft Systems (ICUAS), 779–788 (2013)

  36. Saripalli, S., Montgomery, J.F., Sukhatme, G.S.: Visually guided landing of an unmanned aerial vehicle. IEEE Trans. Robot. Autom. 19(3), 371–380 (2003)

    Article  Google Scholar 

  37. Scherer, S., Chamberlain, L., Singh, S.: Autonomous landing at unprepared sites by a full-scale helicopter. Robot. Auton. Syst. 60(12), 1545–1562 (2012)

    Article  Google Scholar 

  38. Siciliano, B., Sciavicco, L., Villani, L., Oriolo, G.: Robotics Modelling, Planning and Control. In: Springer, 51–52 (2009)

  39. Siebert, S., Teizer, J.: Mobile 3d mapping for surveying earthwork projects using an unmanned aerial vehicle (uav) system. Autom. Constr. 41, 1–14 (2015)

    Article  Google Scholar 

  40. Silva, J., Mendona, R., Marques, F., Rodrigues, P., Santana, P., Barata, J.: Saliency-Based Cooperative Landing of a Multirotor Aerial Vehicle on an Autonomous Surface Vehicle. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), 1523–1530 (2014)

  41. System, R.O.: Available from: http://www.ros.org/. Accessed: 2018-03-07

  42. Team, O.D.: Opencv reference manual. chapter 14, pp. 77-79. available from: http://www.cs.unc.edu/stc/faqs/opencv/opencvreferencemanual.pdf. Accessed: 2018-05-21

  43. Trizzino, R., Caprioli, M., Mazzone, F., Scarano, M.: Applications of Uav Photogrammetric Surveys to Natural Hazard Detection and Cultural Heritage Documentation. In: 19Th EGU General Assembly, EGU2017 (2017)

  44. Umbaugh, S.E.: Digital Image Processing and Analysis: Human and Computer Vision Applications with Cviptools. In: CRC Press (2016)

  45. Vision, M.: Available from: https://www.matrix-vision.com/usb2.0-single-board-camera-mvbluefox-mlc.html. Accessed: 2018-01-08

  46. Vzquez-Arellano, M., Griepentrog, H.W., Reiser, D., Paraforos, D.S.: 3-d imaging systems for agricultural applications a review. Sensors 16(5) (2016)

  47. Waharte, S., Trigoni, N.: Supporting Search and Rescue Operations with Uavs. In: IEEE International Conference on Emerging Security Technologies (EST), 142–147. IEEE, Canterbury (2010). https://doi.org/10.1109/EST.2010.31

  48. Waldock, A., Greatwood, C., Salama, F., Richardson, T.: Learning to perform a perched landing on the ground using deep reinforcement learning. J. Intell. Robot. Syst., pp. 1–20 (2017)

  49. Wang, C., Chen, D., Li, M., Gong, J.: Direct solution for pose estimation of single circle with detected centre. Electron. Lett. 52(21), 1751–1753 (2016)

    Article  Google Scholar 

  50. Yang, S., Scherer, S.A., Schauwecker, K., Zell, A.: Autonomous landing of mavs on an arbitrarily textured landing site using onboard monocular vision. J. Intell. Robot. Syst. 74(1-2), 2743 (2014)

    Article  Google Scholar 

  51. Yang, S., Scherer, S.A., Zell, A.: An onboard monocular vision system for autonomous takeoff, hovering and landing of a micro aerial vehicle. J. Intell. Robot. Syst. 69(1-4), 499515 (2013)

    Google Scholar 

  52. Zhang, L., Xu, K.J., Zhao, R., Liang, L.P.: Improvement of position and orientation measurement algorithm of monocular vision based on circle features. Journal of Hefei University of Technology (Natural Science), p. 11 (2009)

  53. Zhou, D., Zhou, J., Zhang, M., Xiang, D., Zhong, Z.: Deep Learning for Unmanned Aerial Vehicles Landing Carrier in Different Conditions. In: IEEE 18Th International Conference on Advanced Robotics (ICAR), 469–475 (2017)

Download references

Acknowledgements

The authors would like to thank Dr. Roberto Colella for his valuable contribution in the setup of UAV and for his support during the experiments. In addition, the authors thank Mr Michele Attolico for his technical support.

Author information

Authors and Affiliations

  1. CNR STIIMA, via Amendola 122/D-O, Bari, Italy

    C. Patruno, M. Nitti, A. Petitti, E. Stella & T. D’Orazio

Authors
  1. C. Patruno
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Nitti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Petitti
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. Stella
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. T. D’Orazio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Patruno.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Patruno, C., Nitti, M., Petitti, A. et al. A Vision-Based Approach for Unmanned Aerial Vehicle Landing. J Intell Robot Syst 95, 645–664 (2019). https://doi.org/10.1007/s10846-018-0933-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-018-0933-2

Keywords

Search

Navigation