Skip to main content
SpringerLink
Log in

Visual-based Assistive Method for UAV Power Line Inspection and Landing

  • Short Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Over the years, many methods and technologies have been developed to improve power lines’ visual-based inspection, such as the recent advances in computer vision and machine learning techniques and the use of Unmanned Aerial Vehicles (UAVs). Although aerial inspection of transmission lines is a well-researched topic, there is still space for research papers and solutions that focus on shared control, landing, and experimental evaluation of the solutions. This kind of system has the capabilities to acquire and process information about its surrounding. By employing automatic UAVs embedded with artificial intelligence, industries, business and researchers can significantly improve their visual-based inspection routines, bringing safety to the user and processing reliable information. Therefore, this research work proposes a strategy to both detect and track power transmission lines and a method to allow assistive control during UAV landing. Both methods were evaluated in simulated and real-world scenarios. Regarding the detection and tracking strategies, the outcomes suggested that the proposed system is capable of correctly identifying power transmission lines and navigating above them, even in the presence of cluttered backgrounds. Futhermore, the results of the assisted landing strategy showed that the method has excellent performance and is technically viable for practical deployment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Code availability

The codes and information used will be made available in a github (https://github.com/lcfdiniz) in the future.

References

  1. Zormpas, A., Moirogiorgou, K., Kalaitzakis, K., Plokamakis, G.A., Partsinevelos, P., Giakos, G., Zervakis, M.: Power transmission lines inspection using properly equipped unmanned aerial vehicle (uav). In: 2018 IEEE International Conference on Imaging Systems and Techniques (IST), pp. 1–5. IEEE (2018)

  2. Laib dit Leksir, Y., Mansour, M., Moussaoui, A.: Localization of thermal anomalies in electrical equipment using infrared thermography and support vector machine. Infrared Physics & Technology 89, 120–128 (2018)

  3. Nguyen, V.N., Jenssen, R., Roverso, D.: Automatic autonomous vision-based power line inspection: A review of current status and the potential role of deep learning. Int. J. Electr. Power Energy Syst. 99, 107–120 (2018). https://doi.org/10.1016/j.ijepes.2017.12.016, https://www.sciencedirect.com/science/article/pii/S0142061517324444. Accessed May 2022

  4. Toth, J., Pouliot, N., Montambault, S.: Field experiences using linescout technology on large bc transmission crossings. In: 2010 1st International Conference on Applied Robotics for the Power Industry, pp. 1–6. IEEE (2010)

  5. Jenssen, R., Roverso, D., et al.: Intelligent monitoring and inspection of power line components powered by uavs and deep learning. IEEE Power and Energy Technology Systems Journal 6(1), 11–21 (2019)

    Article  Google Scholar 

  6. Austin, R.G., Earp, G.: Power line inspection by uav: A business case. In: Proc. 19th International Conference on Unmanned Air Vehicle Systems, pp. 14–1 (2004)

  7. Li, Z., Mu, S., Li, J., Wang, W., Liu, Y.: Transmission line intelligent inspection central control and mass data processing system and application based on uav. In 2016 4th International Conference on Applied Robotics for the Power Industry (CARPI), pp. 1–5. IEEE (2016)

  8. Zhang, X., Li, X., Yin, W.-r., Lu, C.-y., Wang, H.-y.: A binocular vision based auxiliary navigation system of the unmanned aerial vehicle for power line inspection. In: 2016 3rd International Conference on Systems and Informatics (ICSAI), pp. 291–296. IEEE (2016)

  9. Cui, J., Zhang, Y., Ma, S., Yi, Y., Xin, J., Liu, D.: Path planning algorithms for power transmission line inspection using unmanned aerial vehicles. In: 2017 29th Chinese Control And Decision Conference (CCDC), pp. 2304–2309. IEEE (2017)

  10. Hui, X., Bian, J., Zhao, X., Tan, M.: Vision-based autonomous navigation approach for unmanned aerial vehicle transmission-line inspection. Int. J. Adv. Robot. Syst. 15(1), 1729881417752821 (2018)

    Article  Google Scholar 

  11. da Silva, M.F., Honório, L.M., Marcato, A.L.M., Vidal, V.F., Santos, M.F.: Unmanned aerial vehicle for transmission line inspection using an extended kalman filter with colored electromagnetic interference. ISA Trans. 100, 322–333 (2020)

    Article  Google Scholar 

  12. Menéndez, O., Pérez, M., Cheein, F.A.: Visual-based positioning of aerial maintenance platforms on overhead transmission lines. Appl. Sci. 9(1), 165 (2019)

    Article  Google Scholar 

  13. Song, B., Li, X.: Power line detection from optical images. Neurocomputing 129, 350–361 (2014)

    Article  Google Scholar 

  14. Zhu, L., Cao, W., Han, J., Du, Y.: A double-side filter based power line recognition method for uav vision system. In: 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2655–2660. IEEE (2013)

  15. Hosseini, M.M., Umunnakwe, A., Parvania, M., Tasdizen, T.: Intelligent damage classification and estimation in power distribution poles using unmanned aerial vehicles and convolutional neural networks. IEEE Trans. Smart Grid 11(4), 3325–3333 (2020)

    Article  Google Scholar 

  16. Jenssen, R., Roverso, D., et al.: Automatic autonomous vision-based power line inspection: A review of current status and the potential role of deep learning. Int. J. Electr. Power Energy Syst. 99, 107–120 (2018)

    Article  Google Scholar 

  17. Melo, A.G., Pinto, M.F., Honorio, L.M., Dias, F.M., Masson, J.E.N.: 3d correspondence and point projection method for structures deformation analysis. IEEE Access 8, 177823–177836 (2020)

    Article  Google Scholar 

  18. de Sá Só, D.H.C., Viana, D.P., de Lima, A.A., Pinto, M.F., Tarrataca, L., e Silva, F.L., Gutiérrez, R.H.R., de Moura Prego, T., Monteiro, U.A.B.V., Haddad, D.B., et al.: Diagnostic and severity analysis of combined failures composed by imbalance and misalignment in rotating machines. The International Journal of Advanced Manufacturing Technology 114(9), 3077–3092 (2021)

  19. Ramos, G.S., Pinto, M.F., de Souza, E.S.S., Machado, G.B., de Castro, G.G.R.: Technical and economic feasibility study for implementing a novel mooring-assisting methodology in offloading operations using autonomous unmanned aerial vehicles. SPE Production & Operations pp. 1–16 (2021)

  20. Yetgin, ÖE., Benligiray, B., Gerek, ÖN.: Power line recognition from aerial images with deep learning. IEEE Trans. Aerosp. Electron. Syst. 55(5), 2241–2252 (2018)

    Article  Google Scholar 

  21. Ozcanli, A.K., Yaprakdal, F., Baysal, M.: Deep learning methods and applications for electrical power systems: A comprehensive review. Int. J. Energy Res. 44(9), 7136–7157 (2020)

    Article  Google Scholar 

  22. Yang, H., Liu, X., Zhang, D., Chen, T., Li, C., Huang, W.: Machine learning for power system protection and control. Electr. J. 34(1),(2021)

    Article  Google Scholar 

  23. Pernebayeva, D., James, A.P.: Deep-learning-based approach for outdoor electrical insulator inspection. In: Deep Learning Classifiers with Memristive Networks, pp. 81–88. Springer (2020)

  24. Silva, M.F., Honório, L.M., Santos, M.F., Vidal, V.F., Mercorelli, P.: Model and validation of the electromagnetic interference produced by power transmission lines in robotic systems. In: 2021 25th International Conference on System Theory, Control and Computing (ICSTCC), pp. 519–524 (2021). https://doi.org/10.1109/ICSTCC52150.2021.9607293

  25. Guan, H., Sun, X., Su, Y., Hu, T., Wang, H., Wang, H., Peng, C., Guo, Q.: Uav-lidar aids automatic intelligent powerline inspection. Int. J. Electr. Power Energy Syst. 130,(2021)

    Article  Google Scholar 

  26. Lopez, R.L., Sanchez, M.J.B., Jimenez, M.P., Arrue, B.C., Ollero, A.: Autonomous uav system for cleaning insulators in power line inspection and maintenance. Sensors 21(24), 8488 (2021)

    Article  Google Scholar 

  27. Biundini, I.Z., Pinto, M.F., Melo, A.G., Marcato, A.L.M., Honório, L.M., Aguiar, M.J.R.: A framework for coverage path planning optimization based on point cloud for structural inspection. Sensors 21(2), 570 (2021)

    Article  Google Scholar 

  28. Murray, C.C., Raj, R.: The multiple flying sidekicks traveling salesman problem: Parcel delivery with multiple drones. Transportation Research Part C: Emerging Technologies 110, 368–398 (2020)

    Article  Google Scholar 

  29. Pinto, M.F., Honorio, L.M., Melo, A., Marcato, A.L.M.: A robotic cognitive architecture for slope and dam inspections. Sensors 20(16), 4579 (2020)

    Article  Google Scholar 

  30. Pinto, M.F., Honório, L.M., Marcato, A.L.M., Dantas, M.A.R., Melo, A.G., Capretz, M., Urdiales, C.: Arcog: An aerial robotics cognitive architecture. Robotica 39(3), 483–502 (2021)

    Article  Google Scholar 

  31. Biundini, I.Z., Melo, A.G., Pinto, M.F., Marins, G.M., Marcato, A.L.M., Honorio, L.M.: Coverage path planning optimization for slopes and dams inspection. In: Silva, M.F., Lima, J.L., Reis, L.P., Sanfeliu, A., Tardioli, D. (eds.) Robot 2019: Fourth Iberian Robotics Conference, pp. 513–523. Springer International Publishing, Cham (2020)

    Chapter  Google Scholar 

  32. Toth, J., Gilpin-Jackson, A.: Smart view for a smart grid—unmanned aerial vehicles for transmission lines. In: 2010 1st International Conference on Applied Robotics for the Power Industry, pp. 1–6. IEEE (2010)

  33. Hui, Z., Fuzhen, H.: An intelligent fault diagnosis method for electrical equipment using infrared images. In: 2015 34th Chinese Control Conference (CCC), pp. 6372–6376. IEEE (2015)

  34. Baik, H., Valenzuela, J.: Unmanned aircraft system path planning for visually inspecting electric transmission towers. J. Intell. Robot. Syst. 95(3), 1097–1111 (2019)

    Article  Google Scholar 

  35. Mirallès, F., Hamelin, P., Lambert, G., Lavoie, S., Pouliot, N., Montfrond, M., Montambault, S.: Linedrone technology: Landing an unmanned aerial vehicle on a power line. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6545–6552. IEEE (2018)

  36. Hamelin, P., Mirallès, F., Lambert, G., Lavoie, S., Pouliot, N., Montfrond, M., Montambault, S.: Discrete-time control of linedrone: An assisted tracking and landing uav for live power line inspection and maintenance. In: 2019 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 292–298. IEEE (2019)

  37. Azevedo, F., Dias, A., Almeida, J., Oliveira, A., Ferreira, A., Santos, T., Martins, A., Silva, E.: Real-time lidar-based power lines detection for unmanned aerial vehicles. In: 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 1–8. IEEE (2019)

  38. Hui, X., Bian, J., Zhao, X., Tan, M.: A monocular-based navigation approach for unmanned aerial vehicle safe and autonomous transmission-line inspection. Int. J. Adv. Robot. Syst. 16(1), 1729881419829941 (2019)

    Article  Google Scholar 

  39. Zhou, G., Yuan, J., Yen, I.-L., Bastani, F.: Robust real-time uav based power line detection and tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 744–748. IEEE (2016)

  40. Araar, O., Aouf, N.: Visual servoing of a quadrotor uav for autonomous power lines inspection. In: 22nd Mediterranean Conference on Control and Automation, pp. 1418–1424. IEEE (2014)

  41. Loureiro, G., Dias, A., Martins, A., Almeida, J.: Emergency landing spot detection algorithm for unmanned aerial vehicles. Remote Sensing 13(10), 1930 (2021)

    Article  Google Scholar 

  42. Diniz, L., Maria, T.S.: Guilherme Augusto Pussente. Power transmission line dataset. In: IEEE Dataport (2021). https://doi.org/10.21227/t9qk-cn48

  43. Song, Y. Naji, S., Kaufmann, E., Loquercio, A., Scaramuzza, D.: Flightmare: A flexible quadrotor simulator. (2020) CoRR, arXiv:2009.00563

  44. Wong, S.C., Gatt, A., Stamatescu, V., McDonnell, M.D.: Understanding data augmentation for classification: when to warp? (2016) CoRR, arXiv:1609.08764

  45. Choi, J., Chun, D., Kim, H., Lee, H.-J.: Gaussian yolov3: An accurate and fast object detector using localization uncertainty for autonomous driving. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 502–511 (2019)

  46. Bochkovskiy, A.., Wang, C.-Y., Liao, H.-Y.M.: Yolov4: Optimal speed and accuracy of object detection. (2020) arXiv:2004.10934

  47. Padilla, R., Netto, S.L., da Silva, E.A.B.: A survey on performance metrics for object-detection algorithms. In: 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 237–242 (2020)

  48. Santos, M.F., Pereira, V.S., Ribeiro, A.C., Silva, M.F., do Carmo, M.J., Vidal, V.F., Honório, L.M., Cerqueira, A.S., Oliveira, E.J.: Simulation and comparison between a linear and nonlinear technique applied to altitude control in quadcopters. In: 2017 18th International Carpathian Control Conference (ICCC), pp. 234–239. IEEE (2017)

  49. Pinto, M.F., Marcato, A.L.M., Melo, A.G., Honório, L.M., Urdiales, C.: A framework for analyzing fog-cloud computing cooperation applied to information processing of uavs. Wireless Communications and Mobile Computing, 2019 (2019)

Download references

Acknowledgements

The authors would like to thank the following Brazilian Federal Agencies CEFET-RJ, FAPERJ, CAPES, CNPq, and UFJF for supporting this research.

Funding

The authors would like to thank the following Brazilian Federal Agencies UFJF, CEFET-RJ, CAPES, CNPq, FAPERJ, INCT–INERGE, ANEEL P&D Program, and TBE (PD-02651-0016/2019) for supporting this research.

Author information

Authors and Affiliations

  1. Federal University of Juiz de Fora (UFJF), Juiz de Fora, Brazil

    Lucas F. Diniz, Aurelio G. Melo & Leonardo M. Honório

  2. Federal Center for Technological Education of Rio de Janeiro, Rio de Janeiro, Brazil

    Milena F. Pinto

Authors
  1. Lucas F. Diniz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Milena F. Pinto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aurelio G. Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Leonardo M. Honório
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Lucas F. Diniz, Aurélio G. Melo, and Milena F. Pinto. The supervision and resources were performed by Leonardo M. Honório; The first draft of the manuscript was written by Lucas F. Diniz, Aurélio G. Melo and Milena F. Pinto, all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Aurelio G. Melo.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethics approval

Not applicable

Consent to participate

Not applicable

Consent for publication

Not applicable

Informed consent

Not applicable

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Diniz, L.F., Pinto, M.F., Melo, A.G. et al. Visual-based Assistive Method for UAV Power Line Inspection and Landing. J Intell Robot Syst 106, 41 (2022). https://doi.org/10.1007/s10846-022-01725-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01725-x

Keywords

Search

Navigation