Skip to main content

Advertisement

Springer Nature Link
Log in

Value Iteration-Based Adaptive Fuzzy Backstepping Optimal Control of Modular Robot Manipulators via Integral Reinforcement Learning

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

An adaptive fuzzy backstepping optimal control method is developed for modular robot manipulators (MRMs) via value iteration (VI). This paper adopts joint torque feedback (JTF) technique to construct subsystem dynamics model, and the state space description is deduced. According to fusion function which contains the error in joint angular velocity and position, the cost function is established. The integral reinforcement learning (IRL) is integrated into the VI algorithm, which solves the optimal tracking control issue without system drift dynamics. For purpose of improving the control effect, the optimal tracking control issue of manipulator can be reconsidered as the optimal compensation issue which adopting the local dynamics information. Then the uncertainty in the model can be compensated by an adaptive fuzzy backstepping compensation controller which is constructed by fuzzy logic system (FLS) and backstepping control method. The optimal compensation control strategy is adopted to deal with the interconnected dynamic coupling (IDC), which contains global information about each joint. Based on the VI algorithm and adaptive dynamic programming (ADP) method, an effective solution of Hamiltonian-Jacobi-Bellman (HJB) equation is presented. According to Lyapunov theorem, the trajectory tracking error is uniformly ultimately bounded (UUB) by using the adaptive fuzzy backstepping optimal control method. Finally, the effectiveness of the proposed method is verified by experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Data will be made available upon reasonable request.

References

  1. Hoernicke, M., Stark, K., Schoch, N.: Modular engineering of conventional plants: using MTP for world-scale industry plants. ATP Mag. 63(4), 62–68 (2022)

    Google Scholar 

  2. An, T., Wang, Y., Liu, G., Li, Y., Dong, B.: Cooperative game-based approximate optimal control of modular robot manipulators for human-robot collaboration. IEEE Trans. Cybernetics 53(7), 4691–4703 (2023)

    Google Scholar 

  3. Hauser, S., Mutlu, M., Léziart, P.: Roombots extended: challenges in the next generation of self-reconfigurable modular robots and their application in adaptive and assistive furniture. Robot. Auton. Syst. 127, 103467 (2020)

    Google Scholar 

  4. Baca, J., Hossain, S.G.M., Dasgupta, P.: Modred: Hardware design and reconfiguration planning for a high dexterity modular self-reconfigurable robot for extra-terrestrial exploration. Robot. Auton. Syst. 62(7), 1002–1015 (2014)

    Google Scholar 

  5. Li, Y., Hao, X., She, Y.: Constrained motion planning of free-float dual-arm space manipulator via deep reinforcement learning. Aerosp. Sci. Technol. 109, 106446 (2021)

    Google Scholar 

  6. Ginting, M.F., Otsu, K., Edlund, J.A.: CHORD: distributed data-sharing via hybrid ROS 1 and 2 for multi-robot exploration of large-scale complex environments. IEEE Robot. Autom. Lett. 6(3), 5064–5071 (2021)

    Google Scholar 

  7. Aggravi, M., Elsherif, A.A.S., Giordano, P.R.: Haptic-enabled decentralized control of a heterogeneous human-robot team for search and rescue in partially-known environments. IEEE Robot. Autom. Lett. 6(3), 4843–4850 (2021)

    Google Scholar 

  8. Bellman, R.: Dynamic programming. Science 153(3731), 34–37 (1966)

    Google Scholar 

  9. Ciarlet, P.G., Raviart, P.A.: Maximum principle and uniform convergence for the finite element method. Comput. Methods Appl. Mech. Eng. 2(1), 17–31 (1973)

    MathSciNet  Google Scholar 

  10. Werbos, P.: Advanced forecasting methods for global crisis warning and models of intelligence. Gen. Syst. Yearb. 22(6), 25–38 (1977)

    Google Scholar 

  11. Li, C., Ding, J., Lewis, F.L.: A novel adaptive dynamic programming based on tracking error for nonlinear discrete-time systems. Automatica 129, 109687 (2021)

    MathSciNet  Google Scholar 

  12. Xue, S., Luo, B., Liu, D.: Adaptive dynamic programming based event-triggered control for unknown continuous-time nonlinear systems with input constraints. Neurocomputing 396, 191–200 (2020)

    Google Scholar 

  13. Zhang, H., Cui, L., Zhang, X.: Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method. IEEE Trans. Neural Netw. 22(12), 2226–2236 (2011)

    Google Scholar 

  14. Safaei, A., Mahyuddin, M.N.: Optimal model-free control for a generic MIMO nonlinear system with application to autonomous mobile robots. Int. J. Adapt. Control Signal Process. 32(6), 792–815 (2018)

    MathSciNet  Google Scholar 

  15. Kong, L., He, W., Yang, C.: Robust neurooptimal control for a robot via adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 32(6), 2584–2594 (2020)

    MathSciNet  Google Scholar 

  16. Xia, H., Guo, P.: Sliding mode-based online fault compensation control for modular reconfigurable robots through adaptive dynamic programming. Complex Intell. Syst. 8(3), 1963–1973 (2022)

    Google Scholar 

  17. Li, Y., Wei, C., An, T.: Event-triggered-based cooperative game optimal tracking control for modular robot manipulator with constrained input. Nonlinear Dyn. 109(4), 2759–2779 (2022)

    MathSciNet  Google Scholar 

  18. Cao, L., Cheng, Z., Liu, Y., Li, H.: Event-based adaptive NN fixed-time cooperative formation for multiagent systems. IEEE Trans. Neural Netw. Learn. Syst. (2022). https://doi.org/10.1109/TNNLS.2022.3210269

    Article  Google Scholar 

  19. Yu, D., Long, J., Chen, C.L.P., Wang, Z.: Adaptive swarm control within saturated input based on nonlinear coupling degree. IEEE Trans. Syst. Man Cybernetics Syst. 52(8), 4900–4911 (2022)

    Google Scholar 

  20. Cao, L., Pan, Y., Liang, H., Huang, T.: Observer-based dynamic event-triggered control for multiagent systems with time-varying delay. IEEE Trans. Cybernetics 53(5), 3376–3387 (2023)

    Google Scholar 

  21. Wang, X., Xu, B., Guo, Y.: Fuzzy logic system-based robust adaptive control of auv with target tracking. Int. J. Fuzzy Syst. 25(1), 338–346 (2023)

    Google Scholar 

  22. Ma, L., Huo, X., Zhao, X., et al.: Adaptive fuzzy tracking control for a class of uncertain switched nonlinear systems with multiple constraints: a small-gain approach. Int. J. Fuzzy Syst. 21, 2609–2624 (2019)

    MathSciNet  Google Scholar 

  23. Liu, H., Li, S., Li, G., et al.: Adaptive controller design for a class of uncertain fractional-order nonlinear systems: an adaptive fuzzy approach. Int. J. Fuzzy Syst. 20, 366–379 (2018)

    MathSciNet  Google Scholar 

  24. Yu, D., Yang, M., Liu, Y.J., Wang, Z., Chen, C.L.P.: Adaptive fuzzy tracking control for uncertain nonlinear systems with multiple actuators and sensors faults. IEEE Trans. Fuzzy Syst. 31(1), 104–116 (2023)

    Google Scholar 

  25. Li, Y., Min, X., Tong, S.: Adaptive fuzzy inverse optimal control for uncertain strict-feedback nonlinear systems. IEEE Trans. Fuzzy Syst. 28(10), 2363–2374 (2019)

    Google Scholar 

  26. Bu, X., Qi, Q.: Fuzzy optimal tracking control of hypersonic flight vehicles via single-network adaptive critic design. IEEE Trans. Fuzzy Syst. 30(1), 270–278 (2020)

    Google Scholar 

  27. Long, J., Yu, D., Wen, G., Li, L., Wang, Z., Chen, C.L.P.: Game-based backstepping design for strict-feedback nonlinear multi-agent systems based on reinforcement learning. IEEE Trans. Neural Netw. Learn. Syst. (2022). https://doi.org/10.1109/TNNLS.2022.3177461

    Article  Google Scholar 

  28. Chang, C.W., Hsu, C.F., Lee, T.T.: Backstepping-based finite-time adaptive fuzzy control of unknown nonlinear systems. Int. J. Fuzzy Syst. 20, 2545–2555 (2018)

    MathSciNet  Google Scholar 

  29. Sun, J., Liu, C.: Distributed fuzzy adaptive backstepping optimal control for nonlinear multimissile guidance systems with input saturation. IEEE Trans. Fuzzy Syst. 27(3), 447–461 (2018)

    Google Scholar 

  30. Li, Y., Sun, K., Tong, S.: Observer-based adaptive fuzzy fault-tolerant optimal control for SISO nonlinear systems. IEEE Trans. Cybernetics 49(2), 649–661 (2018)

    Google Scholar 

  31. Dong, B., Zhou, F., Liu, K.: Torque sensorless decentralized neuro-optimal control for modular and reconfigurable robots with uncertain environments. Neurocomputing 282, 60–73 (2018)

    Google Scholar 

  32. He, S., Fang, H., Zhang, M.: Adaptive optimal control for a class of nonlinear systems: the online policy iteration approach. IEEE Trans. Neural Netw. Learn. Syst. 31(2), 549–558 (2019)

    MathSciNet  Google Scholar 

  33. Pang, B., Jiang, Z.P.: Adaptive optimal control of linear periodic systems: an off-policy value iteration approach. IEEE Trans. Autom. Control 66(2), 888–894 (2020)

    MathSciNet  Google Scholar 

  34. Heydari, A.: Stability analysis of optimal adaptive control using value iteration with approximation errors. IEEE Trans. Autom. Control 63(9), 3119–3126 (2018)

    MathSciNet  Google Scholar 

  35. Ha, M., Wang, D., Liu, D.: A novel value iteration scheme with adjustable convergence rate. IEEE Trans. Neural Netw. Learn. Syst. (2022). https://doi.org/10.1109/TNNLS.2022.3143527

    Article  Google Scholar 

  36. Heydari, A.: Revisiting approximate dynamic programming and its convergence. IEEE Trans. Cybernetics 44(12), 2733–2743 (2014)

    Google Scholar 

  37. Mu, C., Wang, K., Qiu, T.: Dynamic event-triggering neural learning control for partially unknown nonlinear systems. IEEE Trans. Cybern. 52(4), 2200–2213 (2022)

    Google Scholar 

  38. Vamvoudakis, K.G., Vrabie, D., Lewis, F.L.: Online adaptive algorithm for optimal control with integral reinforcement learning. Int. J. Robust Nonlinear Control 24(17), 2686–2710 (2014)

    MathSciNet  Google Scholar 

  39. Zhang, Y., Zhao, B., Liu, D.: Event-triggered adaptive dynamic programming for multi-player zero-sum games with unknown dynamics. Soft. Comput. 25(3), 2237–2251 (2021)

    Google Scholar 

  40. Dong, B., Chen, J., Pan, Q.: Event-trigger-based approximate optimal control of modular robot manipulators using zero-sum game. Opt. Control Appl. Methods (2022). https://doi.org/10.1002/oca.2893

    Article  MathSciNet  Google Scholar 

  41. Dong, B., Liu, K., Li, Y.: Decentralized control of harmonic drive based modular robot manipulator using only position measurements: theory and experimental verification. J. Intell. Robotic Syst. 88(1), 3–18 (2017)

    Google Scholar 

  42. Imura, J., Yokokohji, Y., Yoshikawa, T.: Robust control of robot manipulators based on joint torque sensor information. Int. J. Robot. Res. 13(5), 434–442 (1994)

    Google Scholar 

  43. Armstrong-Hélouvry, B., Dupont, P., De Wit, C.C.: A survey of models, analysis tools and compensation methods for the control of machines with friction. Automatica 30(7), 1083–1138 (1994)

    Google Scholar 

  44. Liu, G., Goldenberg, A.A., Zhang, Y.: Precise slow motion control of a direct-drive robot arm with velocity estimation and friction compensation. Mechatronics 14(7), 821–834 (2004)

    Google Scholar 

  45. Liu, G.: Decomposition-based friction compensation of mechanical systems. Mechatronics 12(5), 755–769 (2002)

    Google Scholar 

  46. Sun, K., Sui, S., Tong, S.: Fuzzy adaptive decentralized optimal control for strict feedback nonlinear large-scale systems. IEEE Trans. Cybernetics 48(4), 1326–1339 (2017)

    Google Scholar 

  47. Zhou, Q., Wang, L., Wu, C.: Adaptive fuzzy control for nonstrict-feedback systems with input saturation and output constraint. IEEE Trans. Syst. Man Cybernetics Syst. 47(1), 1–12 (2016)

    Google Scholar 

  48. Wu, X., Zhao, Y., Xu, K.: Nonlinear disturbance observer based sliding mode control for a benchmark system with uncertain disturbances. ISA Trans. 110, 63–70 (2021)

    Google Scholar 

  49. Xu, N., Zhao, X., Zong, G.: Adaptive control design for uncertain switched nonstrict-feedback nonlinear systems to achieve asymptotic tracking performance. Appl. Math. Comput. 408, 126344 (2021)

    MathSciNet  Google Scholar 

Download references

Acknowledgements

The work is supported by the National Natural Science Foundation of China (62173047), the Scientific Technological Development Plan Project in Jilin Province of China (20220201038GX), Key Laboratory of Advanced Structural Materials (Changchun University of Technology), Ministry of Education, China (ASM-202202).

Author information

Authors and Affiliations

  1. Department of Control Science and Engineering, Changchun University of Technology, Changchun, 130012, China

    Bo Dong, Hucheng Jiang, Yiming Cui, Xinye Zhu & Tianjiao An

Authors
  1. Bo Dong
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Hucheng Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yiming Cui
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xinye Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Tianjiao An
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Tianjiao An.

Ethics declarations

Conflict of interest

The authors declare no potential conflict of interests.

Supplementary Information

Below is the link to the electronic supplementary material.

(PPT 166,973 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, B., Jiang, H., Cui, Y. et al. Value Iteration-Based Adaptive Fuzzy Backstepping Optimal Control of Modular Robot Manipulators via Integral Reinforcement Learning. Int. J. Fuzzy Syst. 26, 1347–1363 (2024). https://doi.org/10.1007/s40815-023-01670-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-023-01670-3

Keywords