Skip to main content

Analysis of Computer Vision-Based Techniques for the Recognition of Landing Platforms for UAVs

  • Chapter
  • First Online:
Mobile Robot: Motion Control and Path Planning

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1090))

  • 476 Accesses

Abstract

Although advanced UAVs use self-guided landing with the support of GPS devices, issues such as signal loss, support for indoor environments, accuracy issues, etc., must be considered, which contribute greatly to the overall success of autonomous flight. Therefore, this chapter proposes two different methods that allow for the recognition of a landing platform using Computer Vision techniques in order to assist autonomous landing. The first method is based on an Expert System that allows for the recognition of a patented black and white platform by performing a geometric analysis of the regions based on thresholds that allow for a degree of plane distortion. The second method, based on Cognitive Computing, can be used to solve the limitations to plane distortion inherent to the first approach, and further uses a specific landing platform with six different colors in order to combine color segmentation techniques with pattern recognition algorithms, together with an intelligent geometric analysis based on a decision-making technique. As a result, recognition can be achieved at different ranges and inclination angles between the vision system and the platform. It is not affected by distortions to the image figures due to perspective projection, even making it possible to perform the recognition with only a partial view of the platform, something that has received scant attention in the literature to date. The novelty is therefore the robustness and precision in the recognition from a wide variety of perspectives, different lighting conditions, and even problems that result in only a partial view of the platform, such as those resulting from partial focus or blind spots due to overexposure.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. García-Pulido JA, Pajares G, Dormido S, de la Cruz JM (2017) Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques. Expert Syst Appl 76:152–165

    Article  Google Scholar 

  2. Cruz JM, Sánchez B, Pajares G (2012) System for guiding an unmanned vehicle towards a platform using visual analysis. Patent number 201001592, 2013

    Google Scholar 

  3. SALACOM (2013) Sistema Autónomo para la Localización y Actuación ante Contaminantes en el Mar, DPI2013–46665-C1

    Google Scholar 

  4. García-Pulido JA, Pajares G, Dormido S (2022) UAV landing platform recognition using cognitive computation combining geometric analysis and computer vision techniques. Cogn Comp

    Google Scholar 

  5. Peña JM, Torres-Sanchez J, de Castro AI, Kelly M, López-Granados F (2013) Weed mapping in early-season maize fields using object-based analysis of unmanned aerial vehicle (UAV) images. PLoS One 8(10):e77151

    Article  Google Scholar 

  6. Jiang T, Geller J, Ni D, Collura J (2016) Unmanned aircraft system traffic management: concept of operation and system architecture. Int J Transp Sci Technol 5(3):123–135

    Article  Google Scholar 

  7. Popescu D, Ichim L, Stoican F (2017) Unmanned aerial vehicle systems for remote estimation of flooded areas based on complex image processing. Sensors 17(3):446

    Article  Google Scholar 

  8. Pajares G (2015) Overview and current status of remote sensing applications based on unmanned aerial vehicles (UAVs). Photogramm Eng Remote Sens 81(4):281–329

    Article  Google Scholar 

  9. Menéndez O, Pérez M, Auat Cheein F (2019) Visual-based positioning of aerial maintenance platforms on overhead transmission lines. Appl Sci 9(1)

    Google Scholar 

  10. Amazon Prime Air Available online: https://www.amazon.com/Amazon-Prime-Air/b?ie=UTF8&node=8037720011 Accessed June 2022

  11. Kong W, Hu T, Zhang D, Shen L, Zhang J (2017) Localization framework for real-time UAV autonomous landing: an on-ground deployed visual approach. Sensors 2017(17):1437

    Article  Google Scholar 

  12. Nguyen PH, Kim KW, Lee YW, Park KR (2017) Remote marker-based tracking for UAV landing using visible-light camera sensor. Sensors 2017(17):1987

    Article  Google Scholar 

  13. Anitha G, Kumar RNG (2012) Vision based autonomous landing of an unmanned aerial vehicle. Procedia Eng 2012(38):2250–2256

    Article  Google Scholar 

  14. Xu G, Zhang Y, Ji S, Cheng Y, Tian Y (2009) Research on computer vision–based for UAV autonomous landing on a ship. Pattern Recognit Lett 2009(30):600–605

    Article  Google Scholar 

  15. Xu G, Qi X, Zeng Q, Tian Y, Guo R, Wang B (2013) Use of land’s cooperative object to estimate UAV’s pose for autonomous landing. Chin J Aeronaut 2013(26):1498–1505

    Article  Google Scholar 

  16. Gui Y, Guo P, Zhang H, Lei Z, Zhou X, Du J, Yu Q (2013) Airborne vision-based navigation method for UAV accuracy landing using infrared lamps. J Intell Robot Syst 72(2): 197

    Google Scholar 

  17. Yang T, Li G, Li J, Zhang Y, Zhang X, Zhang Z, Li Z (2016) A ground–based near infrared camera array system for UAV auto–landing in GPS–denied environment. Sensors 2016(16):1–20

    Google Scholar 

  18. Kong W, Zhang D, Wang X, Xian Z, Zhang J (2013) Autonomous landing of an UAV with a ground–based actuated infrared stereo vision system. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems. Tokyo, Japan, pp 2963–2970

    Google Scholar 

  19. Zhou D, Zhong Z, Zhang D, Shen L, Yan C (2015) Autonomous landing of a helicopter UAV with a ground-based multisensory fusion system. In: Seventh international conference on machine vision (ICMV 2014). International Society for Optics and Photonics, 2015, 94451R

    Google Scholar 

  20. Tang D, Hu T, Shen L, Zhang D, Kong W, Low KH (2016) Ground stereo vision-based navigation for autonomous take-off and landing of UAVs: a Chan-Vese model approach. Int J Adv Robot Syst 13(2):67

    Google Scholar 

  21. Martínez C, Campoy P, Mondragón I, Olivares–Méndez MA (2009) Trinocular ground system to control UAVs. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems. St. Louis, MO, USA, pp 3361–3367

    Google Scholar 

  22. Kong W, Zhou D, Zhang Y, Zhang D, Wang X, Zhao B, Yan C, Shen L, Zhang J (2014) A ground-based optical system for autonomous landing of a fixed wing UAV. In: Proceedingsof the IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS 2014) September 14–18. Chicago, IL, USA, pp 1–8

    Google Scholar 

  23. Al-Sharman MK, Emran BJM, Jaradat A, Najjaran H, Al-Husari R, Zweiri Y (2018) Precision landing using an adaptive fuzzy multi-sensor data fusion architecture. Appl Soft Comput 69:149–164

    Article  Google Scholar 

  24. Al-Sharman M, Al-Jarrah MA, Abdel-Hafez M (2018) Auto takeoff and precision terminal-phase landing using an experimental optical flow model for GPS/INS enhancement. ASCE-ASME J Risk Uncertainty Eng Syst Part B Mech Eng

    Google Scholar 

  25. Asadzadeh M, Palaiahnakote S, Idris MYI, Anisi MH, Lu T, Blumenstein M, Noor NM (2019) An automatic zone detection system for safe landing of UAVs. Expert Syst Appl 122:319–333

    Article  Google Scholar 

  26. Patterson T, McClean S, Morrow P, Parr G, Luo C (2014) Timely autonomous identification of UAV safe landing zones. Image Vis Comput 32(9):568–578

    Article  Google Scholar 

  27. Li X (2013) A software scheme for UAV’s safe landing area discovery. AASRI Procedia 2013(4):230–235

    Article  Google Scholar 

  28. Forster C, Faessler M, Fontana F, Werlberger M, Scaramuzza D (2015) Continuous on-board monocular-vision-based elevation mapping applied to autonomous landing of micro aerial vehicles. In: Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp 111–118

    Google Scholar 

  29. Johnson A, Montgomery J, Matthies L (2005) Vision guided landing of an autonomous helicopter in hazardous terrain. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation (ICRA)

    Google Scholar 

  30. Bosch S, Lacroix S, Caballero F (2006) Autonomous detection of safe landing areas for an UAV from monocular images. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

    Google Scholar 

  31. Desaraju V, Michael N, Humenberger M, Brockers R, Weiss S, Matthies L (2015) Vision-based landing site evaluation and trajectory generation toward rooftop landing. Auton Robots 39(3):445–463

    Google Scholar 

  32. Davide F, Alessio Z, Alessandro S, Jeffrey D, Scaramuzza D (2017) Vision-based autonomous quadrotor landing on a moving platform. J Intell Robot Syst 85(2):369–384

    Google Scholar 

  33. Lee D, Ryan T, Kim HJ (2012) Autonomous landing of a vtol uav on a moving platform using image-based visual servoing. In: 2012 IEEE international conference on robotics and automation. pp 971–976

    Google Scholar 

  34. Lee H, Jung S, Shim DH (2016) Vision–based UAV landing on the moving vehicle. In: Proceedings of the International Conference on Unmanned Aircraft System. Arlington, MA, USA, pp 1–7

    Google Scholar 

  35. Feng Y, Zhang C, Baek S, Rawashdeh S, Mohammadi A (2018) Autonomous landing of a UAV on a moving platform using model predictive control. Drones 2(34)

    Google Scholar 

  36. Line V (2018) Autonomous landing of a multirotor UAVon a platform in motion. Master Thesis, Norwegian University of Science and Technology, Noruega. https://brage.bibsys.no/xmlui/handle/11250/2558185 Accessed June 2022

  37. Rodríguez-Ramos A, Sampedro C, Bavle H, Milosevic Z, García-Vaquero A, Campoy P (2017) Towards fully autonomous landing on moving platforms for rotary unmanned aerial vehicle. In: 2017 International Conference on Unmanned Aircraft Systems (ICUAS). pp 170–178

    Google Scholar 

  38. Polvara R, Sharma S, Wan J, Manning A, Sutton R (2017) Towards autonomous landing on a moving vessel through fiducial markers. In: IEEE European conference on mobile robotics (ECMR). IEEE

    Google Scholar 

  39. Lin S, Garratt MA, Lambert AJ (2017) Monocular vision-based real-time target recognition and tracking for autonomously landing an uav in a cluttered shipboard environment. Auton Robots 41(4):881–901

    Google Scholar 

  40. Chen J, Miao X, Jiang H, Chen J, Liu X (2017) Identification of autonomous landing sign for unmanned aerial vehicle based on faster regions with convolutional neural network. In: IEEE international conference on chinese automation congress (CAC). pp 2019–2114

    Google Scholar 

  41. Polvara R, Patacchiola M, Sharma S, Wan J, Manning A, Sutton R, Cangelosi A (2017) Autonomous quadrotor landing using deep reinforcement learning. arXiv preprint arXiv:1709.03339

  42. Wang L, Yang Q, Guo X (2016) Recognition algorithm of the apron for unmanned aerial vehicle based on image corner points. Laser J 08:71–74

    Google Scholar 

  43. Nyein EE, Tun HM, Naing ZM, Moe WK (2015) Implementation of vision-based landing target detection for VTOL UAV using raspberry Pi. Int J Sci Technol Res 4(8):184–188

    Google Scholar 

  44. Sharp CS, Shakernia O, Sastry SS (2001) A vision system for landing an unmanned aerial vehicle. In: IEEE International Conference on Robotics and Automation. pp 1720–1727

    Google Scholar 

  45. Pajares G, Cruz JM (2007) Visión por computador: imágenes digitales y aplicaciones Fundamentos de Robótica. RA-MA., Segunda Edición, Madrid

    Google Scholar 

  46. Zhao YJ, Pei HL (2013) An improved vision–based algorithm for unmanned aerial vehicles autonomous landing. Appl Mech Mater 273:560–565

    Google Scholar 

  47. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) SURF: speeded up robust features. Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  48. Saavedra-Ruiz M, Pinto-Vargas AM, Romero-Cano V (2018) Detection and tracking of a landing platform for aerial robotics applications. In: IEEE 2nd Colombian Conference on Robotics and Automation (CCRA). Barranquilla, pp 1–6

    Google Scholar 

  49. Saripalli S, Montgomery JF, Sukhatme GS (2002) Vision-based autonomous landing of an unmanned aerial vehicle. IEEE Int Conf Robot Autom 11–15:2799–2804

    Google Scholar 

  50. Saripalli S, Sukhatme G (2006) Landing on a moving target using an autonomous helicopter. Field and service robotics. Springer Tracts Adv Robot 2006:277–286

    Article  Google Scholar 

  51. Cocchioni F, Mancini A, Longhi S (2014) Autonomous navigation, landing and recharge of a quadrotor using artificial vision. In: International conference on unmanned aircraft systems (ICUAS). pp 418–429

    Google Scholar 

  52. Li Y, Wang Y, Luo H, Chen Y, Jiang Y (2012) Landmark recognition for UAV autonomous landing based on vision. Appl Res Comp 07:2780–2783

    Google Scholar 

  53. Vega JA, Dormido-Canto S (2010) Máquinas de Vectores Soporte. Aprendizaje Automático (Pajares G. y de la Cruz J.M., Eds.). RA-MA, Madrid

    Google Scholar 

  54. Guili X, Yong Z, Shengyu J, Yuehua CH, Yupeng T (2009) Research on computer vision-based for UAV autonomous landing on a ship. Pattern Recogn Lett 30(6):600–605

    Article  Google Scholar 

  55. Lange S, Sünderhauf N, Protzel P (2008) Autonomous landing for a multirotor uav using vision. In: Workshop Proceeding of SIMPAR 2008 International conference on simulation, modeling, and programming for autonomous robots. pp 482–491

    Google Scholar 

  56. S Lange N Sunderhauf P Protzel 2009 A vision based onboard approach for landing and position control of an autonomous multirotor uav in gps-denied environments. In: 2009 International conference on advanced robotics. IEEE, pp 1–6

    Google Scholar 

  57. Olson E (2011) AprilTag: a robust and flexible visual fiducial system. In: IEEE international conference on robotics and automation (ICRA)

    Google Scholar 

  58. AprilTag. Available online: https://april.eecs.umich.edu/software/apriltag.html Accessed June 2022

  59. Ling K (2014) Precision landing of a quadrotor uav on a moving target using low–cost sensors. Master Thesis, University of Waterloo, Canada. UWSpace. https://uwspace.uwaterloo.ca/handle/10012/8803 Accessed June 2022

  60. Kyristsis S, Antonopoulos A, Chanialakis T, Stefanakis E, Linardos C, Tripolitsiotis A, Partsinevelos P (2016) Towards autonomous modular UAV missions: the detection, geolocation and landing paradigm. Sensors 16(11):1844

    Google Scholar 

  61. Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn 47(6):2280–2292

    Google Scholar 

  62. Sani F, Karimian G (2017) Automatic navigation and landing of an indoor AR. drone quadrotor using ArUco marker and inertial sensors. In: International Conference on Computer and Drone Applications (IConDA). pp 102–107

    Google Scholar 

  63. Detection of ArUco Markers. http://docs.opencv.org/trunk/d5/dae/tutorial_aruco_detection.html Accessed June 2022

  64. Chaves SM, Wolcott RW, Eustice RM (2015) NEEC research: toward GPS–denied landing of unmanned aerial vehicles on ships at sea. Nav Eng J 127:23–35

    Google Scholar 

  65. Araar O, Aouf N, Vitanov I (2017) Vision based autonomous landing of multirotor uav on moving platform. J Intell Robot Syst 85:369–384

    Google Scholar 

  66. Kalman RE (1960) A new approach to linear filtering and prediction problems. Transactions of the ASME. J Basic Eng 82:35–45

    Article  Google Scholar 

  67. Rabah M, Rohan A, Talha M, Nam K, Kim S (2018) Autonomous vision-based target detection and safe landing for UAV. Int J Control Autom Syst 16:3013–3025

    Article  Google Scholar 

  68. Lôbo-Medeiros FL, Faria-Gomes VC, Campos de Aquino MR, Geraldo D, Lopes-Honorato ME, Moreira-Dias LH (2015) A computer vision system for guidance of vtol uavs autonomous landing. In: Brazilian Conference on Intelligent Systems (BRACIS). pp 333–338

    Google Scholar 

  69. Nguyen PH, Arsalan M, Koo JH, Naqvi RA, Truong NQ, Park KR (2018) LightDenseYOLO: a fast and accurate marker tracker for autonomous UAV landing by visible light camera sensor on drone. Sensors 18(6):1703

    Google Scholar 

  70. Cesetti A, Frontoni E, Mancini A, Zingaretti P, Longhi S (2010) A vision-based guidance system for uav navigation. J Intell Robot Syst 57(1–4):233–257

    Google Scholar 

  71. Zongji C, Lei C, Rui Z, Weiqi L (2008) Unmanned aircraft landing navigation system based on vision. http://www.google.com/patents/CN101109640A Accessed June 2022. Patent CN 101109640

  72. Roy P, Yu J, Linden DS (2008) Methods, apparatus and systems for enhanced synthetic vision and multi-sensor data fusion to improve operational capabilities of unmanned aerial vehicles. http://www.google.sc/patents/US20080215204 Accessed June 2022. Patent US20080215204

  73. Blenkhorn KP, O'Hara SV (2009) Vision-based automated landing system for unmanned aerial vehicles. http://www.google.com/patents/US20090306840 Accessed June 2022. Patent US20090306840

  74. Grzywna J, Da Frota B, Meuse S (2007) System and method for onboard vision processing. http://www.google.nl/patents/US20070093945 Accessed June 2022. Patent US20070093945

  75. Shin-Je C, Jung-Ho M, Seung-Kie C, San K, Sung-Sik S (2008) Automatic recovery method of a UAV using image information to stably recover the UAV by obtaining real-time image information of a landing symbol on a recovery netting. http://patent.ipexl.com/KR/1020070058957.html Accessed June 2022. Patent 1020070058957

  76. Avitzour D (1996) Mobile robot location determination employing error-correcting distributed landmarks. http://www.google.com/patents/US5525883 Accessed June 2022. Patent US5525883

  77. Asaad A, Al-Salih M, Ahson SI (2009) Object detection and features extraction in video frames using direct thresholding. Multimedia, Signal Processing and Communication Technologies. IMPACT ‘09. International, pp 221–224

    Google Scholar 

  78. The MathWorks (2019) The Matlab. https://es.mathworks.com/products/matlab Accessed on June 2022

  79. Otsu N (1979) A threshold selection method from gray-level histogram. IEEE Trans Syst Man Cybern 9:62–66

    Article  Google Scholar 

  80. Kong TY, Rosenfeld A (1996) Topological algorithms for digital image processing. Elsevier Science, Inc.

    Google Scholar 

  81. Dougherty ER, Lotufo RA (2003) Hands-on morphological image processing. SPIE Tutorial Texts in Optical Engineering Vol. TT5. SPIE Publications

    Google Scholar 

  82. Dillencourt MB, Samet H, Tamminen M (1992) A general approach to connected-component labeling for arbitrary image representations. J Assoc Comp Mach 39(2):252–280

    Article  MathSciNet  MATH  Google Scholar 

  83. Sedgewick R (1998) Algorithms in C, 3rd Ed., Addison-Wesley, pp 11–20

    Google Scholar 

  84. Haralick RM, Shapiro GL (1992) Computer and robot vision, vol 1. Addison-Wesley, pp 28–48

    Google Scholar 

  85. Koschan A, Abido M (2008) Digital color image processing. John Wiley & Sons

    Book  Google Scholar 

  86. Hu MK (1962) Visual Problem recognition by Moment Invariant. IRE Trans Inform Theory, IT-8. pp 179–187

    Google Scholar 

  87. Bohanec M (2009) Decision making: a computer-science and information-technology viewpoint. Interdiscip Descr Complex Syst 7(2):22–37

    Google Scholar 

  88. Duda RO, Hart PE, Stork DG (2006) Pattern classification, 2nd edn. New York; New Delhi: Wiley

    Google Scholar 

  89. The MathWorks (2016) Color-Based Segmentation Using the L*a*b* Color Space. https://es.mathworks.com/help/images/examples/color-based-segmentation-using-the-l-a-b-color-space.html Accessed June 2022

  90. Campos Y, Sossa H, Pajares G (2016) Spatio-temporal analysis for obstacle detection in agricultural videos. Appl Soft Comput 45:86–97

    Article  Google Scholar 

  91. Mercimek M, Gulez K, Mumcu TV (2005) Real object recognition using moment invariants. IEEE Trans Pattern Anal 30:765–775

    MATH  Google Scholar 

  92. Viola P, Michael J (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, vol 1:511–518

    Google Scholar 

  93. Fan Z, Lu J, Gong M, Xie H, Goodman ED (2018) Automatic tobacco plant detection in UAV images via deep neural networks. IEEE J Select Top Appl Earth Observ Remote Sens 11(3):876–887

    Article  Google Scholar 

  94. Bicer Y, Moghadam M, Sahin C, Eroglu B, Üre NK (2019) Vision-based UAV guidance for autonomous landing with deep neural networks. In: American Institute of Aeronautics and Astronautics. AIAA Scitech 2019 Forum, AIAA SciTech Forum, (AIAA 2019–0140)

    Google Scholar 

Download references

Acknowledgements

The authors would especially thank Dr. Mark Watkins for his proofreading, proof-editing services and grammatical improvement. Without his selfless assistance and dedication this work would undoubtedly not have been possible.

Finally, we would also like to thank Mestrelab, especially Santi Dominguez and Carlos Cobas, for having created a company like this that provides the flexibility and conditions necessary to make research like this possible. Special mention to Agustín Barba from Mestrelab also and Rebeca Cuiñas, for their unconditional support in good and especially in bad moments.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to J. A. García-Pulido .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

García-Pulido, J.A., Pajares, G. (2023). Analysis of Computer Vision-Based Techniques for the Recognition of Landing Platforms for UAVs. In: Azar, A.T., Kasim Ibraheem, I., Jaleel Humaidi, A. (eds) Mobile Robot: Motion Control and Path Planning. Studies in Computational Intelligence, vol 1090. Springer, Cham. https://doi.org/10.1007/978-3-031-26564-8_3

Download citation

Publish with us

Policies and ethics