Skip to main content

Advertisement

SpringerLink
Log in

Reinforcement Fuzzy Q-Learning Incorporated with Genetic Kinematics Analysis for Self-organizing Holonomic Motion Control of Six-Link Stewart Platforms

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

This paper contributes to the development of reinforcement fuzzy Q-learning incorporated with genetic kinematics analysis for self-organizing holonomic motion control of six-link Stewart platforms. The fuzzy theory is integrated with the Q-learning algorithm to develop a hybrid reinforcement learning strategy. Both the actions and the Q-functions in the classical Q-learning algorithm are inferred by means of fuzzy rules. The proposed hybrid reinforcement learning is then applied to self-organizing holonomic motion control of parallel Stewart platforms for industrial applications. After the kinematics analysis using a geometrical approach and genetic algorithm (GA)-based Newton–Raphson formulation, this study develops a practical control scheme that combines fuzzy Q-learning and proportional–integral–differential (PID) control scheme, called FQPID. In the proposed FQPID holonomic motion control method, the PID control gains are online-tuned via the fuzzy Q-learning computation process. An experimental Stewart platform consisting of six servo motors, adjustable links, and two platforms is constructed to illustrate the effectiveness of the presented methods. The experimental results are compared with some existing control methods using the same evaluation criterion. Through the experimental results and comparative works, the superiority of the proposed FQPID control scheme is presented and discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data Availability

All data generated or analysed during this study are included in this published article.

References

  1. He, Q., Zeng, C., Gao, Z., Wu, Z.: Analysis and design of the Stewart platform-based parallel support bumper for inertially stabilized platforms. IEEE Trans. Ind. Electron. 67(5), 4203–4215 (2020)

    Article  Google Scholar 

  2. Stewart, D.: A platform with six degree of freedom. Proc. Inst. Mech. Eng. 180(15), 371–386 (1965)

    Article  Google Scholar 

  3. Datta, S., Das, A., Gayen, R.K.: Kinematic analysis of Stewart platform using MATLAB. In: International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech), pp. 1–4 (2021)

  4. Cai, Y., Zheng, S., Liu, W., Qu, Z., Han, J.: Model analysis and modified control method of ship-mounted Stewart platforms for wave compensation. IEEE Access. 9, 4505–4517 (2021)

    Article  Google Scholar 

  5. Ding, B., Cazzolato, B.S., Stanley, R.M., Grainger, S., Costi, J.J.: Stiffness analysis and control of a Stewart platform-based manipulator with decoupled sensor–actuator locations for ultrahigh accuracy positioning under large external loads. J. Dyn. Syst. Meas. Contr. 136(6), 1–12 (2014)

    Article  Google Scholar 

  6. Wang, Y.: A direct numerical solution to forward kinematics of general Stewart–Gough platforms. Robotica 25(1), 121–128 (2007)

    Article  Google Scholar 

  7. Kumar, P.R., Behera, A.K., Bandyopadhyay, B.: Robust finite-time tracking of Stewart platform: a super-twisting like observer-based forward kinematics solution. IEEE Trans. Industr. Electron. 64(5), 3776–3785 (2017)

    Article  Google Scholar 

  8. Chen, S.H., Fu, L.C.: Output feedback sliding mode control for a Stewart platform with a nonlinear observer-based forward kinematics solution. IEEE Trans. Control Syst. Technol. 21(1), 176–185 (2013)

    Article  Google Scholar 

  9. Ji, P., Wu, H.: A closed-form forward kinematics solution for 6–6p Stewart platform. IEEE Trans. Robot. Autom. 17(4), 522–526 (2001)

    Article  Google Scholar 

  10. Phu, N.D., Ahmadian, N.N.: A new fuzzy PID control system based on fuzzy PID controller and fuzzy control process. Int. J. Fuzzy Syst. 22, 2163–2187 (2020)

    Article  Google Scholar 

  11. Zhou, H., Chen, R., Zhou, S., Liu, Z.: Design and analysis of a drive system for a series manipulator based on orthogonal-fuzzy PID control. Electronics 8(1051), 1–15 (2019)

    Google Scholar 

  12. Li, R., Deng, S., Hu, Y.: Autonomous vehicle modeling and velocity control based on decomposed fuzzy PID. Int. J. Fuzzy Syst. 10, 1–9 (2022)

    MathSciNet  Google Scholar 

  13. Hasanien, H.M.: Design optimization of PID controller in automatic voltage regulator system using Taguchi combined genetic algorithm method. IEEE Syst. J. 7(4), 825–831 (2013)

    Article  Google Scholar 

  14. Sabahi, K., Hajizadeh, A., Tavan, A.: Adaptive type-2 fuzzy PID LFC for an interconnected power system considering input time-delay. Int. J. Fuzzy Syst. 23, 1042–1054 (2021)

    Article  Google Scholar 

  15. Liu, G., Wang, Z., Zhao, G., Yan, R., Xu, Y., Wang, J., Pu, Y., Jiang, W.: Automatic hot test of gyrotron-traveling wave tubes by adaptive PID feedback control. IEEE Trans. Electron Devices 64(3), 1302–1306 (2017)

    Article  Google Scholar 

  16. Rothmann, M., Porrmann, M.: A survey of domain-specific architectures for reinforcement learning. IEEE Access 10, 13753–13767 (2022)

    Article  Google Scholar 

  17. Ma, Y., Chang, T., Fan, W.: A single-task and multi-decision evolutionary game model based on multi-agent reinforcement learning. J. Syst. Eng. Electron. 32(3), 642–657 (2021)

    Article  Google Scholar 

  18. Jiandong, Z., Qiming, Y., Guoqing, S., Yi, L., Yong, W.: UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning. J. Syst. Eng. Electron. 32(6), 1421–1438 (2021)

    Article  Google Scholar 

  19. Watkins, C.J.: Learning from delayed rewards. Ph.D. dissertation, King’s College, Cambridge (1989)

  20. Er, M.J., Deng, C.: Online tuning of fuzzy inference systems using dynamic fuzzy Q-learning. IEEE Trans. Syst. Man Cybern. B (Cybernetics) 34(3), 1478–1489 (2004)

    Article  Google Scholar 

  21. Gheisarnejad, M., Sharifzadeh, M., Khooban, M., Al-Haddad, K.: Adaptive fuzzy Q-learning control design and application to grid-tied nine-level packed e-cell (PEC9) inverter. IEEE Trans. Ind. Electron. (2022). https://doi.org/10.1109/TIE.2022.3153803

    Article  Google Scholar 

  22. Shi, H., Xu, M., Hwang, K.S.: A fuzzy adaptive approach to decoupled visual servoing for a wheeled mobile robot. IEEE Trans. Fuzzy Syst. 28(12), 3229–3243 (2020)

    Article  Google Scholar 

  23. Wu, B.F., Chen, P.Y., Lin, C.H.: A new criterion of human comfort assessment for wheelchair robots by Q-learning based accompanist tracking fuzzy controller. Int. J. Fuzzy Syst. 18, 1039–1053 (2016)

    Article  Google Scholar 

  24. Zhang, C.: Classification rule mining algorithm combining intuitionistic fuzzy rough sets and genetic algorithm. Int. J. Fuzzy Syst. 22, 1694–1715 (2020)

    Article  Google Scholar 

  25. Engelbrecht, A.P.: Computational Intelligence: An Introduction, 2nd edn. Wiley, Hoboken (2007)

    Book  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the anonymous reviewers for their constructive comments to improve the quality of this paper. This work is financially supported in part by the Ministry of Science and Technology, Taiwan, under the Grants MOST 109-2221-E-011-074, MOST 110-2221-E-011-121, MOST 111-2221-E-011-146-MY2, and MOST 111-2221-E-197-025.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Ilan University, Yilan, 260007, Taiwan

    Hsu-Chih Huang & Yu-Xiang Chen

  2. Graduate Institute of Automation and Control, National Taiwan University of Science and Technology, Taipei, 106335, Taiwan

    Sendren Sheng-Dong Xu & Chien-Ming Chen

  3. Advanced Manufacturing Research Center, National Taiwan University of Science and Technology, Taipei, 106335, Taiwan

    Sendren Sheng-Dong Xu

Authors
  1. Hsu-Chih Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sendren Sheng-Dong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu-Xiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chien-Ming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hsu-Chih Huang.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, HC., Xu, S.SD., Chen, YX. et al. Reinforcement Fuzzy Q-Learning Incorporated with Genetic Kinematics Analysis for Self-organizing Holonomic Motion Control of Six-Link Stewart Platforms. Int. J. Fuzzy Syst. 25, 1239–1255 (2023). https://doi.org/10.1007/s40815-022-01439-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-022-01439-0

Keywords

Search

Navigation