Skip to main content

Advertisement

Springer Nature Link
Log in

Fault-tolerant visual servo control for a robotic arm with actuator faults

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The study targets uncertain coupling faults in robotic arm actuators and proposes a new fault-tolerant visual servo control strategy. Specifically, it considers both multiplicative and additive actuator faults within the dynamic of the robotic arm, treating the coupling faults and time-varying disturbances as an aggregate of concentrated uncertainties. A radial basis function neural network-based state observer is introduced to online approximate these concentrated uncertainties, which include fault information, eliminating the need for prior knowledge of faults. Furthermore, a fault-tolerant controller based on a non-singular fast terminal sliding mode is proposed, which separately decouples the nominal quantities and concentrated uncertainties and develops individual adaptive control laws for each. This effectively reduces the detrimental impact of coupled faults and disturbances on the system’s performance, facilitating image feature trajectory tracking control with minimal jitter, high precision, and strong transient response capabilities. The stability of the state observer and the fault-tolerant controller has been substantiated through Lyapunov’s theory. Lastly, numerical simulations validate the efficacy and robustness of the proposed fault-tolerant visual servo control approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availibility

The datasets during the current study are available from the corresponding author on reasonable request.

References

  1. Singh A, Narula R, Rashwan HA, Abdel-Nasser M, Puig D, Nandi GC (2022) Efficient deep learning-based semantic mapping approach using monocular vision for resource-limited mobile robots. Neural Comput Appl 34(18):15617–15631. https://doi.org/10.1007/s00521-022-07273-7

    Article  Google Scholar 

  2. Munoz-Molina LJ, Cazorla-Pinar I, Dominguez-Morales JP, Lafuente L, Perez-Pena F (2022) Real-time detection of uncalibrated sensors using neural networks. Neural Comput Appl 34(10):8227–8239. https://doi.org/10.1007/s00521-021-06865-z

    Article  Google Scholar 

  3. MahmoudZadeh S, Yazdani A, Elmi A, Abbasi A, Ghanooni P (2021) Exploiting a fleet of UAVs for monitoring and data acquisition of a distributed sensor network. Neural Comput Appl 34(7):5041–5054. https://doi.org/10.1007/s00521-021-05906-x

    Article  Google Scholar 

  4. Chaumetter F, Hutchinson S (2007) Visual servo control - Part II: advanced approaches. IEEE Robot Autom Mag 14(1):109–118. https://doi.org/10.1109/MRA.2007.339609

    Article  Google Scholar 

  5. Chaumetter F, Hutchinson S (2006) Visual servo control - Part I: basic approaches. IEEE Robot Autom Mag 13(4):82–90. https://doi.org/10.1109/MRA.2006.250573

    Article  Google Scholar 

  6. Lin J, Wang Y, Miao Z, Zhong H, Fierro R (2022) Low-complexity control for vision-based landing of quadrotor UAV on unknown moving platform. IEEE Trans Industr Inf 18(8):5348–5358. https://doi.org/10.1109/TII.2021.3129486

    Article  Google Scholar 

  7. Zhou S, Shen C, Pang F, Chen Z, Gu J, Zhu S (2022) Position-based visual servoing control for multi-joint hydraulic manipulator. J Intell Robot Syst 105(2):33. https://doi.org/10.1007/s10846-022-01628-x

    Article  Google Scholar 

  8. Chen H, Liu K, Xing G, Dong Y, Sun H, Lin W (2014) A robust visual servo control system for narrow seam double head welding robot. Int J Adv Manuf Technol 71(9–12):1849–1860. https://doi.org/10.1007/s00170-013-5593-6

    Article  Google Scholar 

  9. Wang N, He H (2020) Adaptive homography-based visual servo for micro unmanned surface vehicles. Int J Adv Manuf Technol 105(12):4875–4882. https://doi.org/10.1007/s00170-019-03994-7

    Article  Google Scholar 

  10. Gans NR, Hutchinson SA (2007) Stable visual servoing through hybrid switched-system control. IEEE Trans Robot 23(3):530–540. https://doi.org/10.1109/TRO.2007.895067

    Article  Google Scholar 

  11. Wang Y, Langm H, De Silva CW (2010) A hybrid visual servo controller for robust grasping by wheeled mobile robots. IEEE-ASME Trans Mechatron 15(5):757–769. https://doi.org/10.1109/TMECH.2009.2034740

    Article  Google Scholar 

  12. Gong Z, Tao B, Yang H, Yin Z, Ding H (2018) An uncalibrated visual servo method based on projective homography. IEEE Trans Autom Sci Eng 15(2):806–817. https://doi.org/10.1109/TASE.2017.2702195

    Article  Google Scholar 

  13. Zhong XG, Zhong XY, Peng XF (2015) Robots visual servo control with features constraint employing Kalman-neural-network filtering scheme. Neurocomputing 151:268–277. https://doi.org/10.1016/j.neucom.2014.09.043

    Article  Google Scholar 

  14. Shi HB, Xu M, Hwang KS (2020) A fuzzy adaptive approach to decoupled visual servoing for a wheeled mobile robot. IEEE Trans Fuzzy Syst 28(12):3229–3243. https://doi.org/10.1109/TFUZZ.2019.2931219

    Article  Google Scholar 

  15. Li SP, Li D, Zhang CH, Wang J (2022) An intelligence image processing method of visual servo system in complex environment. IEEE Sens J 22(18):17370–17377. https://doi.org/10.1109/JSEN.2020.3040288

    Article  Google Scholar 

  16. Wang N, He HK (2021) Extreme learning-based monocular visual servo of an unmanned surface vessel. IEEE Trans Ind Inf 17(8):5152–5163. https://doi.org/10.1109/TII.2020.3033794

    Article  Google Scholar 

  17. Brahmi B, Saad M, Lam JTAT, Luna CO, Archambault PS, Rahman MH (2018) Adaptive control of a 7-DOF exoskeleton robot with uncertainties on kinematics and dynamics. Eur J Control 42:77–87. https://doi.org/10.1016/j.ejcon.2018.03.002

    Article  MathSciNet  Google Scholar 

  18. Shao XL, Liu N, Wang ZQ, Zhang WD, Yang W (2020) Neuroadaptive integral robust control of visual quadrotor for tracking a moving object. Mech Syst Signal Process 136:106513. https://doi.org/10.1016/j.ymssp.2019.106513

    Article  Google Scholar 

  19. Wu JH, Jin ZH, Liu AD, Yu L (2021) Vision-based neural predictive tracking control for multi-manipulator systems with parametric uncertainty. ISA Trans 110:247–257. https://doi.org/10.1016/j.isatra.2020.10.057

    Article  Google Scholar 

  20. Guo C, Li F, Tian Z, Guo W, Tan S (2019) Intelligent active fault-tolerant system for multi-source integrated navigation system based on deep neural network. Neural Comput Appl 32(22):16857–16874. https://doi.org/10.1007/s00521-018-03975-z

    Article  Google Scholar 

  21. Sun SX, Dai X, Xi RP, Zhang J (2021) Fault-tolerant control for uncertain switched random systems with multiple interval time-varying delays and intermittent faults. Neural Comput Appl 33(24):17471–17487. https://doi.org/10.1007/s00521-021-06338-3

    Article  Google Scholar 

  22. Tang P, Lin DF, Zheng D, Fan SP, Ye JC (2020) Observer based finite-time fault tolerant quadrotor attitude control with actuator faults. Aerosp Sci Technol 104:105968. https://doi.org/10.1016/j.ast.2020.105968

    Article  Google Scholar 

  23. Guo SC, Yang MH, Xing ZR (2012) Actuator fault detection and isolation for robot manipulators with the adaptive observer. Adv Mater Res 48:529–532. https://doi.org/10.4028/www.scientific.net/AMR.482-484.529

    Article  Google Scholar 

  24. Moreno JA, Osorio M (2008) A Lyapunov approach to second-order sliding mode controllers and observers. In: 47th IEEE conference on decision and control, Cancun, Mexico, 9-11 December, pp 2856-2861

  25. Yu XY, Wang T, Qiu JB, Gao HJ (2021) Barrier Lyapunov function-based adaptive fault-tolerant control for a class of strict-feedback stochastic nonlinear systems. IEEE Trans Cybern 51(2):938–946. https://doi.org/10.1109/TCYB.2019.2941367

    Article  Google Scholar 

  26. Wang J, Wei Q, Sun LL, Li ZG (2022) Fault-tolerant adaptive PID switched control of robot manipulator based on average dwell time. Trans Inst Meas Control 44(16):3165–3175. https://doi.org/10.1177/01423312221099714

    Article  Google Scholar 

  27. Karras GC, Fourlas GK (2020) Model predictive fault tolerant control for Omni-directional mobile robots. J Intell Robot Syst 97(3–7):635–655. https://doi.org/10.1007/s10846-019-01029-7

    Article  Google Scholar 

  28. Qiu JB, Ma M, Wang T (2022) Event-triggered adaptive fuzzy fault-tolerant control for stochastic nonlinear systems via command filtering. IEEE Trans Syst Man Cybern Syst 52(2):1145–1155. https://doi.org/10.1109/TSMC.2020.3013744

    Article  Google Scholar 

  29. Zhang HY, Zhao XD, Zhang L, Niu B, Zong GD, Xu N (2022) Observer-based adaptive fuzzy hierarchical sliding mode control of uncertain under-actuated switched nonlinear systems with input quantization. Int J Robust Nonlinear Control 32(14):8163–8185. https://doi.org/10.1002/rnc.6269

    Article  MathSciNet  Google Scholar 

  30. Tang P, Zhang FB, Ye JC, Lin DF (2021) An integral TSMC-based adaptive fault-tolerant control for quadrotor with external disturbances and parametric uncertainties. Aerosp Sci Technol 109:106415. https://doi.org/10.1016/j.ast.2020.106415

    Article  Google Scholar 

  31. Shabbir W, Li AJ, Cui TW (2023) Neural network-based sensor fault estimation and active fault-tolerant control for uncertain nonlinear systems. J Franklin Inst 360(4):2678–2701. https://doi.org/10.1016/j.jfranklin.2022.12.044

    Article  MathSciNet  Google Scholar 

  32. Zhao H, Wang HQ, Niu B, Zhao XD, Alharbi KH (2023) Event-triggered fault-tolerant control for input-constrained nonlinear systems with mismatched disturbances via adaptive dynamic programming. Neural Netw 164:508–520. https://doi.org/10.1016/j.neunet.2023.05.001

    Article  Google Scholar 

  33. Xu SS, Chen CC, Wu ZL (2015) Study of nonsingular fast terminal sliding-mode fault-tolerant control. IEEE Trans Ind Electron 62(6):3906–3913. https://doi.org/10.1109/TIE.2015.2399397

    Article  Google Scholar 

  34. Van M, Mavrovouniotis M, Ge SS (2019) An adaptive backstepping nonsingular fast terminal sliding mode control for robust fault tolerant control of robot manipulators. IEEE Trans Syst Man Cybern Syst 49(7):1448–1458. https://doi.org/10.1109/TSMC.2017.2782246

    Article  Google Scholar 

  35. Huang HF, He W, Li JS, Xu B, Yang CG, Zhang WC (2022) Disturbance observer-based fault-tolerant control for robotic systems with guaranteed prescribed performance. IEEE Trans Cybern 52(2):772–783. https://doi.org/10.1109/TCYB.2019.2921254

    Article  Google Scholar 

  36. Ruan ZW, Yang QM, Ge SS, Sun YX (2020) Performance-guaranteed fault-tolerant control for uncertain nonlinear systems via learning-based switching scheme. IEEE Trans Neural Netw Learn Syst 32(9):4138–4150. https://doi.org/10.1109/TNNLS.2020.3016954

    Article  MathSciNet  Google Scholar 

  37. Feng Y, Yu XH, Man ZH (2002) Non-singular terminal sliding mode control of rigid manipulators. Automatica 38(12):2159–2167. https://doi.org/10.1016/S0005-1098(02)00147-4

    Article  MathSciNet  Google Scholar 

  38. Man ZH, Paplinski AP, Wu HR (1994) A robust MIMO terminal sliding mode control scheme for rigid robotic manipulators. IEEE Trans Autom Control 39(12):2464–2469. https://doi.org/10.1109/9.362847

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We would like to extend our sincere gratitude to the editors and reviewers for their insightful comments and suggestions, which have significantly improved the quality of this manuscript.

Funding

This research was funded by the Natural Science Foundation of Heilongjiang Province grant number KY10400210217 and the Fundamental Strengthening Program Technical Field Fund grant number 2021-JCJQ-JJ-0026.

Author information

Authors and Affiliations

  1. College of Intelligent Systems Science and Engineering, Harbin Engineering University, Nantong, Harbin, 150001, Heilongjiang, China

    Jiashuai Li, Xiuyan Peng, Bing Li & Jiawei Wu

  2. Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, Stirling Highway, Perth, WA, 6009, Australia

    Victor Sreeram

Authors
  1. Jiashuai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiuyan Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Victor Sreeram
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiawei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jiashuai Li contributed to the methodology, investigation, formal analysis, and writing—original draft. Xiuyan Peng was involved in the supervision and writing—review and editing. Bing Li assisted in the supervision and writing—review and editing. Victor Sreeram was involved in the supervision and writing—review and editing. Jiawei Wu contributed to the methodology, investigation, formal analysis, and writing—original draft.

Corresponding author

Correspondence to Bing Li.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Peng, X., Li, B. et al. Fault-tolerant visual servo control for a robotic arm with actuator faults. Neural Comput & Applic 36, 15815–15828 (2024). https://doi.org/10.1007/s00521-024-09714-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09714-x

Keywords