Skip to main content
SpringerLink
Log in

Vision Based Neuro-Fuzzy Controller for a Two Axes Gimbal System with Small UAV

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents the development of a vision-based neuro-fuzzy controller for a two axes gimbal system mounted on a small Unmanned Aerial Vehicle (UAV). The controller uses vision-based object detection as input and generates pan and tilt motion and velocity commands for the gimbal in order to keep the interest object at the center of the image frame. A readial basis function based neuro-fuzzy system and a learning algorithm is developed for the controller to address the dynamic and non-linear characteristics of the gimbal movement. The controller uses two separate, but identical radial basis function networks, one for pan and one for tilt motion of the gimbal. Each system is initialized with a fixed number of neurons that act as rules basis for the fuzzy inference system. The membership functions and rule strengths are then adjusted with the feedback from the visual tracking system. The controller is trained off-line until a desired error level is achieved. Training is then continued on-line to allow the system to accommodate air speed changes. The algorithm learns from the error computed from the detected position of the object in image frame and generates position and velocity commands for the gimbal movement. Several tests including lab tests and actual flight tests of the UAV have been carried out to demonstrate the effectiveness of the controller. Test results show that the controller is able to converge effectively and generate accurate position and velocity commands to keep the object at the center of the image frame.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kumar, R., Samarasekera, S., Hsu, S., Hanna, K.: Registration of highly-oblique and zoomed in aerial video to reference imagery. In: Proceedings of IEEE Computer Society Computer Vision and Pattern Recognition Conference. Barcelona, Spain (2000)

  2. Kumar, R., Sawhney, H.S., Asmuth, J.C., Pope, A., Hsu, S.: Registration of video to geo-referenced imagery. In: Proceedings of the 14th International Conference on Pattern Recognition, vol. 2, pp. 1393–1400. Brisbane, Australia (1998)

  3. Stolle, S., Rysdyk, R.: Flight path following guidance for unmanned air vehicles with pan-tilt camera for target observation. In: 22nd Digital Avionics Systems Conference, Indianapolis (2003)

  4. Dobrokhodov, V.N., Kaminer, I.I., Jones, K.D., Ghabcheloo, R.: Vision-based tracking and motion estimation for moving target using small UAVs. In: Proceedings of 2006 American Control Conference, Minneapolis, 14–16 June 2006

  5. Barber, D.B., Redding, J.D., McLain, T.W., Beard, R.W., Taylor, C.N.: Vision-based target geo-location using a fixed-wing miniature air vehicle. J. Intell. Robot. Syst. 47, 361–382 (2006)

    Article  Google Scholar 

  6. Redding, J.D.: Vision based target localization from a small fixed-wing unmanned air vehicle. Master’s thesis, Brigham Young University, Provo, Utah 84602 (2005)

  7. Qadir, A., Neubert, J., Semke, W.: On-board visual tracking with small unmanned aircraft systems. In: AIAA Infotech@Aerospace conference, St. Louis, MO, 28–31 March 2011

  8. Jang, J.S.R.: ANFIS: adaptive-network-based fuzzy inference system. IEEE Trans. Syst. Man Cybern. 23, 665–684 (1993)

    Article  Google Scholar 

  9. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern. 15, 116–132 (1985)

    Article  MATH  Google Scholar 

  10. Leng, G., McGinnity, T.M., Prasad, G.: An approach for on-line extraction of fuzzy rules using a self-organizing fuzzy neural network. Fuzzy Sets and Systems 150, 211–243 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  11. Li, W., Hori, Y.: An algorithm for extracting fuzzy rules based on RBF neural network. IEEE Trans. Ind. Electron. 53(4), 1269–1276 (2006)

    Article  Google Scholar 

  12. Jang, J.S.R., Sun, C.T.: Functional equivalence between radial basis function networks and fuzzy inference systems. IEEE Trans. Neural Netw. 4, 156–159 (1993)

    Article  Google Scholar 

  13. Cho, K.B., Wang, B.H.: Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction. Fuzzy Sets and Systems 83, 325–339 (1996)

    Article  MathSciNet  Google Scholar 

  14. Diao, Y., Passino, K.M.: Adaptive neural/fuzzy control for interpolated nonlinear systems. IEEE Trans. Fuzzy Syst. 10, 583–595 (2002)

    Article  Google Scholar 

  15. Chen, B.S., Lee, C.H., Chang, Y.C.: Tracking design of uncertain nonlinear siso systems: adaptive fuzzy approach. IEEE Trans. Fuzzy Syst. 4, 32–43 (1996)

    Article  Google Scholar 

  16. Spooner, J.T., Passino, K.M.: Stable adaptive control using fuzzy systems and neural networks. IEEE Trans. Fuzzy Syst. 4, 339–359 (1996)

    Article  Google Scholar 

  17. Lee, C., Teng, C.: Identification and control of dynamic systems using recurrent fuzzy neural networks. IEEE Trans. Fuzzy Syst. 8, 349–366 (2000)

    Article  Google Scholar 

  18. Narendra, K.S., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Netw. 1, 4–27 (1990)

    Article  Google Scholar 

  19. Polycarpou, M.M., Mears, M.J.: Stable adaptive tracking of uncertain systems using nonlinearly parameterized online approximators. Int. J. Control 70(3), 363–384 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  20. Hsiao, F.-H., Xu, S.-D., Lin, C.-Y., Tsai, Z.-R.: Robustness design of fuzzy control for nonlinear multiple time-delay large-scale systems via neuralnetwork-based approach. IEEE Trans. Syst. Man Cybern. B Cybern. 38(1), 244–251 (2008)

    Article  Google Scholar 

  21. Lewis, F.L., Jagannathan, S., Yeildirek, A.: Neural Network Control of Robot Manipulators and Nonlinear Systems. Taylor and Francis, London (1999)

    Google Scholar 

  22. Golea, N., Golea, A., Benmahammed, K.: Stable indirect fuzzy adaptive control. Fuzzy Sets and Systems 13(7), 353–366 (2003)

    Article  MathSciNet  Google Scholar 

  23. Wang, T., Lina, C., Liub, H.: Observer-based indirect adaptive fuzzy-neural tracking control for nonlinear SISO systems using VSS and H∞ approaches. Fuzzy Sets and Systems 14(3), 211–232 (2004)

    Google Scholar 

  24. Labiod, S., Boucherit, M.S., Guerra, T.M.: Adaptive fuzzy control of a class of MIMO nonlinear systems. Fuzzy Sets and Systems 15(1), 59–77 (2005)

    Article  MathSciNet  Google Scholar 

  25. Rovithakis, G.A., Christodoulou, M.A.: Adaptive control of unknown plants using dynamical neural networks. IEEE Trans. Syst. Man Cybern. 24, 400–412 (1994)

    Article  MathSciNet  Google Scholar 

  26. Chen, F.C., Liu, C.C.: Adaptively controlling nonlinear continuous-time systems using multilayer neural netoworks. IEEE Trans. Automat. Control 39, 1306–1310 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  27. Rovithakis, G.A., Christodoulou, M.A.: Direct adaptive regulation of unknown nonlinear dynamical systems via dynamic neural networks. IEEE Trans. Syst. Man Cybern. 25, 1578–1594 (1995)

    Article  Google Scholar 

  28. Wang, C., Liu, H., Lin, T.: Direct adaptive fuzzyneural control with state observer and supervisory controller for unknown nonlinear dynamical systems. IEEE Trans. Fuzzy Syst. 10, 39–49 (2002)

    Article  Google Scholar 

  29. Leu, Y., Wang, W., Lee, T.: Observer-based direct adaptive fuzzy-neural control for nonaffine nonlinear systems. IEEE Trans. Neural Netw. 16, 853–861 (2005)

    Article  Google Scholar 

  30. Phan, P., Gale, T.J.: Direct adaptive fuzzy control with a self-structuring algorithm. Fuzzy Sets and Systems 159, 871–899 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  31. Christodoulou, M.A., Theodoridis, D.C., Boutalis, Y.S.: Building optimal fuzzy dynamical systems description based on recurrent neural network approximation. In: Proc. Int. Conf. of Networked Distributed Systems for Intelligent Sensing and Control. Kalamata, Greece (2007)

  32. Boutalis, Y.S., Theodoridis, D.C., Christodoulou, M.A.: A new neuro FDS definition for indirect adaptive control of unknown nonlinear systems using a method of parameter hopping. IEEE Trans. Neural Netw. 20(4), 609–625 (2009)

    Article  Google Scholar 

  33. Theodoridis, D.C, Boutalis, Y.S., Christodoulou, M.A.: Indirect adaptive control of unknown multi variable nonlinear systems with parametric and dynamic uncertainties using a new neuro-fuzzy system description. Int. J. Neural. Syst. 20(2), 129–148 (2010)

    Article  Google Scholar 

  34. Cong, S., Liang, Y.: PID-like neural network nonlinear adaptive control for uncertain multivariable motion control system. IEEE Trans. Ind. Electron. 56(10), 3872–3879 (2009)

    Article  Google Scholar 

  35. Mbede, J.B., Huang, X., Wang, M.: Robust neural-fuzzy sensor based motion control among dynamic obstacles for robot manipulators. IEEE Trans. Fuzzy Syst. 11, 249–260 (2003)

    Article  Google Scholar 

  36. Patino, H.D., Carelli, R., Kuchen, B.R.: Neural networks for advanced control of robot manipulators. IEEE Trans. Neural Netw. 13, 343–354 (2002)

    Article  Google Scholar 

  37. Kelly, W., et al.: Neuro-fuzzy control of a robotic arm. In: Proceedings of the Artificial Neural Networks in Engineering Conference, pp. 837–842. St. Louis, MO, 10–13 Nov 1996

  38. Beom, H.R., Cho, H.S.: A sensor-based navigation for a mobile robot using fuzzy-logic and reinforcement learning. IEEE Trans. Syst. Man Cybern. 25(3), 464–477 (1995)

    Article  Google Scholar 

  39. Lee, C.-H., Chiu, M.-H.: Recurret neuro fuzzy control design for tracking of mobile robots via hybrid algorithm. Expert. Syst. Appl. 36, 8993–8999 (2009)

    Article  Google Scholar 

  40. Rusu, P., Petriu, E.M., Whalen, T.E., Crnell, A., Spoelder, H.J.W.: Behavior-based neuro-fuzzy controller for mobile robot navigation. IEEE Trans. Instrum. Meas. 52(4), 1335–1340 (2003)

    Article  Google Scholar 

  41. Martins, F.N., Celeste, W.C., Carelli, R., Sarcinelli-Finho, M., Bastos-Filho, T.F.: An adaptive dynamic controller for autonomous mobile robot trajectory tracking. Control Eng. Pract. 16, 1354–1363 (2008)

    Article  Google Scholar 

  42. Wang, J.S., Lee, C.S.G.: Self-adaptive recurrent neuro-fuzzy control of an autonomous underwater vehicle. IEEE Trans. Rob. Autom. 19(2), 283–295 (2003)

    Article  MATH  Google Scholar 

  43. Xu, B., Pandian, S.R., Sakagami, N., Petry, F.: Neuro-fuzzy control of underwater vehicle-manipulator systems. J. Franklin Inst. 349, 1125–1138 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  44. Suresh, S., Kannan, N., Sundararajan, N., Saratchandran, P.: Neural adaptive control for vibration suppression in composite fin-tip of aircraft. Int. J. Neural. Syst. 18(3), 219–231 (2008)

    Article  Google Scholar 

  45. Collani, Y.V., Zhang, J., Knoll, A.: A neuo-fuzzy solution for fine-motion control based on vision and force sensors. Technical Report, Department of Technology, University of Bielefeld (1997)

  46. Lin, F.-J., Chou, P.-H., Shieh, P.-H., Chen, S.-Y.: Robust control of an LUSM-based X – Y – θ motion control stage using an adaptive interval type-2 fuzzy neural network. IEEE Trans. Fuzzy Syst. 17(1), 24–38 (2009)

    Article  Google Scholar 

  47. Wu, C.S., Gao, J.Q.: Vision-based neuro-fuzzy control of weld penetration in gas tungsten arc welding of thin sheets. Int. J. Model. Ident. Control 1(2) (2006)

  48. Lewis, J.P.: Fast normalized cross-correlation. In: Proceedings of Vision Interface, pp. 120–123 (1995)

  49. Welch, G., Bishop, G.: An introduction to the Kalman Filter. In: Dept. Comp. Sci., Univ. North Carolina, Chapel Hill, TR95-041 (2000)

  50. Ranganathan, J., Semke, W.: Three-axis gimbal surveillance algorithms for use in small UAS. In: Proceedings of the ASME International Mechanical Engineering Conference and Exposition, IMECE2008-67667 (2008)

  51. Semke W., Ranganathan, J., Buisker, M.: Active gimbal control for surveillance using small unmanned aircraft systems. In: Proceedings of the International Model analysis Conference (IMAC) XXVI: A Conference and Exposition on Structural Dynamics (2008)

Download references

Author information

Authors and Affiliations

  1. Mechanical Engineering Department, University of North Dakota, Grand Forks, ND, USA

    Ashraf Qadir, William Semke & Jeremiah Neubert

Authors
  1. Ashraf Qadir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William Semke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeremiah Neubert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashraf Qadir.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Qadir, A., Semke, W. & Neubert, J. Vision Based Neuro-Fuzzy Controller for a Two Axes Gimbal System with Small UAV. J Intell Robot Syst 74, 1029–1047 (2014). https://doi.org/10.1007/s10846-013-9865-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-013-9865-z

Keywords

Search

Navigation