Skip to main content
SpringerLink
Log in

Real-time drogue detection and template tracking strategy for autonomous aerial refueling

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Autonomous aerial refueling technology is an effective solution to extend flight duration of unmanned aerial vehicles, and also a great challenge due to its high risk. A novel real-time drogue detection and tracking strategy of monocular vision system for autonomous aerial refueling is proposed. It uses a direct image registration-based tracking method to ensure reliable and real-time tracking, and an ROI detection based on edge features to solve the tracking drift problem. A multiply patches fusion structure is adopted in the tracking method to improve the tracking accuracy and slow the divergence speed. Finally, various experiments are conducted to validate the proposed image processing strategy. These results show that the proposed strategy obtains a high accuracy as well as the real-time performance, and achieves a better performance than state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Baker, S., Matthews, I.: Equivalence and efficiency of image alignment algorithms. In: IEEE Conference on Computer Vision and Pattern Recognition, USA (2001)

  2. Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. Lecture Notes in Computer Science 588, 237–252 (1992)

    Article  Google Scholar 

  3. Brannan, D., Esplen, M., Gray, J.: Geometry. Cambridge University Press, Cambridge, UK (1999)

    Book  Google Scholar 

  4. Chen, C., Koseluk, R., Buchanan, C., Duerner, A., Jeppesen, B., Laux, H.: Autonomous aerial refueling ground test demonstation—a sensor-in-the-loop, non-tracking method. Sensors 15, 10948–10972 (2015)

  5. Christian, J.A., Lightsey, G.E.: Onboard image-processing algorithm for a spacecraft optical navigation sensor system. Journal of Spacecraft and Rockets 49, 337–352 (2012)

    Article  Google Scholar 

  6. Dellaquila, R.V., Campa, G., Napolitano, M.R., Mammarella, M.: Real-time machine-vision-based position sensing system for uav aerial refueling. J. Real-Time Image Process. 1, 213–224 (2007)

    Article  Google Scholar 

  7. Fravolini, M., Mammarella, M., Campa, G., Napolitano, M.R., Perhinschi, M.: Machine vision algorithms for autonomous aerial refueling for uavs using the usaf refueling boom method. Studi. Comput. Intell. 304, 95–138 (2010)

    MathSciNet  Google Scholar 

  8. Gander, W., Golub, G.H., Strebel, R.: Least squares fitting of circles and ellipses. BIT Numer. Math. 34, 558–578 (1994)

    Article  MathSciNet  Google Scholar 

  9. Gao, S.B., Cheng, Y.M., Song, C.H.: Drogue detection for vision-based autonomous aerial refueling via low rank and sparse decomposition with multiple features. Infrared Phys. Technol. 60, 266–274 (2013)

    Article  Google Scholar 

  10. Giampiero, C., Marcello, R.N., Mario, L.F.: Simulation environment for machine vision based aerial refueling for UAVs. IEEE Trans. Aerosp. Electron. Syst. 45, 138–151 (2009)

    Article  Google Scholar 

  11. Hansen, J.L., Murray, J.E., Campos, N.V.: The NASA dryden aar project: a flight test approach to an aerial refueling system. In: Proceeding of the AIAA Atmospheric Flight Mechanics Conference and Exhibit, Providence, RI, USA (2004)

  12. Hare, S., Saffari, A., Torr, P.H.S.: Struck: structured output tracking with kernels. Int. Conf. Comput. Vis. 2011, 263–270 (2011)

    Google Scholar 

  13. Hsia, C.H., Liou, Y.J., Chiang, J.S.: Directional prediction camshift algorithm based on adaptive search pattern for moving object tracking. J. Real-Time Image Process. 12(1), 183–195 (2016). https://doi.org/10.1007/s11554-013-0382-x

  14. Huang, B., Sun, Y.R., Sun, X.D., Liu, J.Y.: Circular drogue pose estimation for vision-based navigation in autonomous aerial refueling. In: Proceedings of the 2016 IEEE Chinese Guidance, Navigation and Control Conference (CGNCC), Nanjing, China (2016)

  15. Irani, M., Anandan, P.: About direct methods. Lect. Notes Comput. Sci. 1883, 267–277 (2000)

    Article  Google Scholar 

  16. Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1409–1422 (2012)

    Article  Google Scholar 

  17. Kimmett, J., Valasek, J., Junkins, J.L.: Autonomous aerial refueling utilizing a vision based navigation system. In: Proceedings of AIAA Guidance, Navigation, and Control Conference and Exhibit, Monterey, California, USA (2002a)

  18. Kimmett, J., Valasek, J., Junkins, J.L.: Vision based controller for autonomous aerial refueling. In: Proceedings of the 2002 IEEE International Conference on Control Applications, Glasgow, Scotland (2002b)

  19. Li, S., Lu, R.K., Zhang, L., Peng, Y.M.: Image processing algorithms for deep-space autonomous optical navigation. J. Navig. 66, 605–623 (2013)

    Article  Google Scholar 

  20. Luington, B., Johnson, E.N., Vachtsevanos, G.J.: Vision based navigation and target tracking for unmanned aerial vehicles. Intell. Syst. Control Autom. Sci. Eng. 33, 245–266 (2007)

    Article  Google Scholar 

  21. Mao, W., Eke, F.O.: A survey of the dynamics and control of aircraft during aerial refueling. Nonlinear Dyn. Syst. Theory 8, 375–388 (2008)

    MathSciNet  MATH  Google Scholar 

  22. Martinez, C., Richardson, T., Thomas, P., du Bois, J.L., Campoy, P.A.: Vision-based strategy for autonomous aerial refueling tasks. Robot. Auton. Syst. 61, 876–895 (2013)

    Article  Google Scholar 

  23. Mrwald, T., Prankl, J., Zillich, M., Vincze, M.: Advances in real-time object tracking. J. Real-Time Image Process. 10(4), 683–697 (2015). https://doi.org/10.1007/s11554-013-0388-4

  24. Nelepka, J.P., Hinchman, J.L.: Automated aerial refueling: extending the effectiveness of unmanned air vehicles. In: Proceedings of the AIAA Modeling and Simulation Technologies Conference, San Francisco, CA, USA (2005)

  25. Nieto, M., Corts, A., Otaegui, O., Arrspide, J., Salgado, L.: Real-time lane tracking using rao-blackwellized particle filter. J. Real-Time Image Process. 11(1), 179–191 (2016). https://doi.org/10.1007/s11554-012-0315-0

  26. Pollini, L., Mati, R., Innocenti, M.: Experimental evaluation of vision algorithms for formation flight and aerial refueling. In: Proceedings of the AIAA Modeling and Simulation Technologies Conference and Exhibit, Providence, Rhode Island (2004)

  27. Pollini, L., Innocenti, M., Mati, R.: Vision algorithms for formation flight and aerial refueling with optimal marker labeling. In: Proceedings of AIAA Modeling and Simulation Technologies Conference and Exhibit, San Francisco, California, USA (2005)

  28. Ro, K., Kamman, J.W.: Modeling and simulation of hose-paradrogue aerial refueling systems. J. Guid. Control Dyn. 33, 53–63 (2010)

    Article  Google Scholar 

  29. Tandale, M.D., Bowers, R., Valasek, J.: Robust trajectory tracking controller for vision based probe and drogue autonomous aerial refueling. In: Proceedings of AIAA Guidance, Navigation, and Control Conference and Exhibit, San Francisco, CA, USA (2005)

  30. Tomas, P., Bhandari, U., Bullock, S., Richardson, T.S., du Bois, J.L.: Advanced in air to air refueling. Prog. Aerosp. Sci. 71, 14–35 (2014)

    Article  Google Scholar 

  31. Valasek, J., Gunman, K., Kimmett, J., Junkins, J.L., Hughes, D., Tandale, M.D.: Vision-based sensor and navigation system for autonomous air refueling. In: Proceedings of the 1st AIAA Unmanned Aerospace Vehicles, Systems, Technologies, and Operations Conference and Exhibit, Vancouver, Canada (2002)

  32. Varfolomieiev, A., Lysenko, O.: An improved algorithm of median flow for visual object tracking and its implementation on arm platform. J. Real-Time Image Process. 11(3), 527–534 (2016). https://doi.org/10.1007/s11554-013-0354-1

  33. Wang, X.F., Kong, X.W., Zhi, J.H.: Real-time drogue recognition and 3d locating for UAV autonomous aerial refueling based on monocular machine vision. Chin. J. Aeronaut. 28, 1667–1675 (2015)

    Article  Google Scholar 

  34. Williamson, W.R., Glenn, G.J., Dang, V.T., Speyer, J.L.: Sensors fusion applied to autonomous aerial refueling. J. Guid. Control Dyn. 32, 262–275 (2009)

    Article  Google Scholar 

  35. Wu Y, Lim J, Yang MH (2013) Online object tracking: a benchmark. In: Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, Oregon, USA, pp 2411–2418

  36. Yin, Y., Xu, D., Wang, X.G., Bai, M.R.: Detection and tracking strategies for autonomous aerial refueling tasks based on monocular vision. Int. J. Adv. Robot. Syst. 11, 1–12 (2014)

    Article  Google Scholar 

  37. Zhang, K.H., Zhang, L., Yang, M.H.: Real-time compressive tracking. Comput. Vis. ECCV 2012, 864–877 (2012)

    Google Scholar 

  38. Zhu, Y.F., Sun, Y.R., Huang, B., Li, R.B., Wu, L.: Research on relative navigation algorithm for uav autonomous aerial refueling rendezvous phase. In: Proceedings of the 2016 IEEE Chinese Guidance, Navigation and Control Conference (CGNCC), Nanjing, China (2016)

Download references

Author information

Authors and Affiliations

  1. Navigation Research Center, College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Bin Huang, Yongrong Sun & Qinghua Zeng

Authors
  1. Bin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongrong Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qinghua Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Huang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, B., Sun, Y. & Zeng, Q. Real-time drogue detection and template tracking strategy for autonomous aerial refueling. J Real-Time Image Proc 17, 437–446 (2020). https://doi.org/10.1007/s11554-018-0787-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-018-0787-7

Search

Navigation