Abstract
This paper investigates a privacy-preserving consensus tracking problem for a class of nonstrict-feedback discrete-time multi-agent systems (MASs). An improved Liu cryptosystem is developed to alleviate the errors between encryption and decryption on the plaintext, which ensures satisfactory recovery of the plaintext information. A reinforcement learning (RL) technique is then employed to compensate for unknown dynamics and errors between true signals and decrypted ones. Based on the backstepping and graph theory, an RL-based privacy-preserving consensus tracking control strategy is further designed. By virtue of graph theory and Lyapunov stability theory, it is shown that the consensus tracking errors and all signals in the MAS are ultimately bounded. Finally, simulation examples are presented for verification of the effectiveness of the control strategy.
摘要
本文研究了一类非严格反馈离散时间多智能体系统的隐私保护一致性跟踪问题。为减轻明文加密和解密之间的误差影响,开发一种改进的Liu加密系统,以确保明文信息恢复良好。采用强化学习技术补偿未知动态和真实信号与解密信号之间的误差。采用反步法和图论知识,设计基于强化学习的隐私保护一致性跟踪控制策略。借助李雅普诺夫稳定性理论,证明多智能体系统的一致跟踪误差和所有信号最终有界。最后,通过仿真实例验证设计控制策略的有效性。
Article PDF
Avoid common mistakes on your manuscript.
Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Bai WW, Zhang B, Zhou Q, et al., 2020a. Multigradient recursive reinforcement learning NN control for affine nonlinear systems with unmodeled dynamics. Int J Robust Nonl Contr, 30(4):1643–1663. https://doi.org/10.1002/rnc.4843
Bai WW, Li TS, Tong SC, 2020b. NN reinforcement learning adaptive control for a class of nonstrict-feedback discrete-time systems. IEEE Trans Cybern, 50(11):4573–4584. https://doi.org/10.1109/TCYB.2020.2963849
Chen W, Liu L, Liu GP, 2023. Privacy-preserving distributed economic dispatch of microgrids: a dynamic quantization-based consensus scheme with homomorphic encryption. IEEE Trans Smart Grid, 14(1):701–713. https://doi.org/10.1109/TSG.2022.3189665
Ding L, Li S, Gao HB, et al., 2020. Adaptive partial reinforcement learning neural network-based tracking control for wheeled mobile robotic systems. IEEE Trans Syst Man Cybern Syst, 50(7):2512–2523. https://doi.org/10.1109/TSMC.2018.2819191
Ding L, Li S, Gao HB, et al., 2021. Adaptive neural network-based finite-time online optimal tracking control of the nonlinear system with dead zone. IEEE Trans Cybern, 51(1):382–392. https://doi.org/10.1109/TCYB.2019.2939424
Elgamal T, 1985. A public key cryptosystem and a signature scheme based on discrete logarithms. IEEE Trans Inform Theory, 31(4):469–472. https://doi.org/10.1109/TIT.1985.1057074
Fang WT, Zamani M, Chen ZY, 2021. Secure and privacy preserving consensus for second-order systems based on Paillier encryption. Syst Contr Lett, 148:104869. https://doi.org/10.1016/j.sysconle.2020.104869
Gao C, Wang ZD, He X, et al., 2021. Encryption–decryption-based consensus control for multi-agent systems: handling actuator faults. Automatica, 134:109908. https://doi.org/10.1016/j.automatica.2021.109908
Gao L, Deng SJ, Ren W, 2019. Differentially private consensus with an event-triggered mechanism. IEEE Trans Contr Netw Syst, 6(1):60–71. https://doi.org/10.1109/TCNS.2018.2795703
Ge SS, Li GY, Lee TH, 2003. Adaptive NN control for a class of strict-feedback discrete-time nonlinear systems. Automatica, 39(5):807–819. https://doi.org/10.1016/S0005-1098(03)00032-3
Ge XH, Xiao SY, Han QL, et al., 2022. Dynamic event-triggered scheduling and platooning control co-design for automated vehicles over vehicular ad-hoc networks. IEEE/CAA J Autom Sin, 9(1):31–46. https://doi.org/10.1109/JAS.2021.1004060
Ge XH, Han QL, Wu Q, et al., 2023. Resilient and safe platooning control of connected automated vehicles against intermittent denial-of-service attacks. IEEE/CAA J Autom Sin, 10(5):1234–1251. https://doi.org/10.1109/JAS.2022.105845
Ge XH, Han QL, Zhang XM, et al., 2024. Communication resource-efficient vehicle platooning control with various spacing policies. IEEE/CAA J Autom Sin, 11(2):362–376. https://doi.org/10.1109/JAS.2023.123507
He WL, Xu WY, Ge XH, et al., 2022. Secure control of multiagent systems against malicious attacks: a brief survey. IEEE Trans Ind Inform, 18(6):3595–3608. https://doi.org/10.1109/TII.2021.3126644
Ju YM, Ding DR, He X, et al., 2022. Consensus control of multi-agent systems using fault-estimation-in-the-loop: dynamic event-triggered case. IEEE/CAA J Autom Sin, 9(8):1440–1451. https://doi.org/10.1109/JAS.2021.1004386
Kishida M, 2019. Encrypted control system with quantiser. IET Contr Theory Appl, 13(1):146–151. https://doi.org/10.1049/iet-cta.2018.5764
Li HY, Wu Y, Chen M, 2021. Adaptive fault-tolerant tracking control for discrete-time multiagent systems via reinforcement learning algorithm. IEEE Trans Cybern, 51(3):1163–1174. https://doi.org/10.1109/TCYB.2020.2982168
Li JN, Yuan L, Chai TY, et al., 2023. Consensus of nonlinear multiagent systems with uncertainties using reinforcement learning based sliding mode control. IEEE Trans Circ Syst I Regular Papers, 70(1):424–434. https://doi.org/10.1109/TCSI.2022.3206102
Li P, Hu JP, Qiu L, et al., 2022. A distributed economic dispatch strategy for power-water networks. IEEE Trans Contr Netw Syst, 9(1):356–366. https://doi.org/10.1109/TCNS.2021.3104103
Li YM, Min X, Tong SC, 2020a. Adaptive fuzzy inverse optimal control for uncertain strict-feedback nonlinear systems. IEEE Trans Fuzzy Syst, 28(10):2363–2374. https://doi.org/10.1109/TFUZZ.2019.2935693
Li YM, Shao XF, Tong SC, 2020b. Adaptive fuzzy prescribed performance control of nontriangular structure nonlinear systems. IEEE Trans Fuzzy Syst, 28(10):2416–2426. https://doi.org/10.1109/TFUZZ.2019.2937046
Li YM, Liu YJ, Tong SC, 2022. Observer-based neuroadaptive optimized control of strict-feedback nonlinear systems with state constraints. IEEE Trans Neur Netw Learn Syst, 33(7):3131–3145. https://doi.org/10.1109/TNNLS.2021.3051030
Liang CD, Ge MF, Xu JZ, et al., 2022. Secure and privacy-preserving formation control for networked marine surface vehicles with sampled-data interactions. IEEE Trans Veh Technol, 71(2):1307–1318. https://doi.org/10.1109/TVT.2021.3133902
Lin XZ, Chen CC, Li SH, 2022. Finite-time output feedback stabilization for a class of output-constrained planar switched systems. IEEE Trans Circ Syst II Express Briefs, 69(1):164–168. https://doi.org/10.1109/TCSII.2021.3077603
Liu DX, 2013. Homomorphic Encryption for Database Querying. US Patent 20150295716.
Liu L, Ding SH, Yu XH, 2021. Second-order sliding mode control design subject to an asymmetric output constraint. IEEE Trans Circ Syst II Express Briefs, 68(4):1278–1282. https://doi.org/10.1109/TCSII.2020.3021715
Liu L, Cui YJ, Liu YJ, et al., 2022. Adaptive event-triggered output feedback control for nonlinear switched systems based on full state constraints. IEEE Trans Circ Syst II Express Briefs, 69(9):3779–3783. https://doi.org/10.1109/TCSII.2022.3173679
Ning BD, Han QL, Zuo ZY, et al., 2023. Fixed-time and prescribed-time consensus control of multiagent systems and its applications: a survey of recent trends and methodologies. IEEE Trans Ind Inform, 19(2):1121–1135. https://doi.org/10.1109/TII.2022.3201589
Nozari E, Tallapragada P, Cortés J, 2017. Differentially private average consensus: obstructions, trade-offs, and optimal algorithm design. Automatica, 81:221–231. https://doi.org/10.1016/j.automatica.2017.03.016
Peng ZH, Wang J, Wang D, et al., 2021. An overview of recent advances in coordinated control of multiple autonomous surface vehicles. IEEE Trans Ind Inform, 17(2):732–745. https://doi.org/10.1109/TII.2020.3004343
Qi XJ, Liu WH, Lu Y, 2023. Event-triggered-based fuzzy adaptive tracking control for nonstrict-feedback asymmetric state constrained systems. Fuzzy Sets Syst, 470:108642. https://doi.org/10.1016/j.fss.2023.108642
Reddy SS, Sinha S, Zhang W, 2023. Design and analysis of RSA and Paillier homomorphic cryptosystems using PSO-based evolutionary computation. IEEE Trans Comput, 72(7):1886–1900. https://doi.org/10.1109/TC.2023.3234213
Rivest RL, Shamir A, Adleman L, 1978. A method for obtaining digital signatures and public-key cryptosystems. Commun ACM, 21(2):120–126. https://doi.org/10.1145/359340.359342
Ruan MH, Gao H, Wang YQ, 2019. Secure and privacy-preserving consensus. IEEE Trans Autom Contr, 64(10):4035–4049. https://doi.org/10.1109/TAC.2019.2890887
Sakthivel R, Sakthivel R, Kaviarasan B, et al., 2019. Finite-time leaderless consensus of uncertain multi-agent systems against time-varying actuator faults. Neurocomputing, 325:159–171. https://doi.org/10.1016/j.neucom.2018.10.020
Shahvali M, Naghibi-Sistani MB, Askari J, 2018. Adaptive output-feedback bipartite consensus for nonstrict-feedback nonlinear multi-agent systems: a finite-time approach. Neurocomputing, 318:7–17. https://doi.org/10.1016/j.neucom.2018.07.039
Sun JL, Yi JQ, Pu ZQ, 2022. Fixed-time adaptive fuzzy control for uncertain nonstrict-feedback systems with time-varying constraints and input saturations. IEEE Trans Fuzzy Syst, 30(4):1114–1128. https://doi.org/10.1109/TFUZZ.2021.3052610
Sun KK, Qiu JB, Karimi HR, et al., 2021. A novel finite-time control for nonstrict feedback saturated nonlinear systems with tracking error constraint. IEEE Trans Syst Man Cybern Syst, 51(6):3968–3979. https://doi.org/10.1109/TSMC.2019.2958072
Tong SC, Li YM, Sui S, 2016. Adaptive fuzzy tracking control design for SISO uncertain nonstrict feedback non-linear systems. IEEE Trans Fuzzy Syst, 24(6):1441–1454. https://doi.org/10.1109/TFUZZ.2016.2540058
Tong SC, Sun KK, Sui S, 2018. Observer-based adaptive fuzzy decentralized optimal control design for strict-feedback nonlinear large-scale systems. IEEE Trans Fuzzy Syst, 26(2):569–584. https://doi.org/10.1109/TFUZZ.2017.2686373
Wang AJ, Liu WP, Dong T, et al., 2022. DisEHPPC: enabling heterogeneous privacy-preserving consensus-based scheme for economic dispatch in smart grids. IEEE Trans Cybern, 52(6):5124–5135. https://doi.org/10.1109/TCYB.2020.3027572
Wang H, Li M, 2022. Model-free reinforcement learning for fully cooperative consensus problem of nonlinear multiagent systems. IEEE Trans Neur Netw Learn Syst, 33(4):1482–1491. https://doi.org/10.1109/TNNLS.2020.3042508
Wang SB, 2022. Asymptotic tracking control for nonaffine systems with disturbances. IEEE Trans Circ Syst II Express Briefs, 69(2):479–483. https://doi.org/10.1109/TCSII.2021.3080524
Wang YQ, 2019. Privacy-preserving average consensus via state decomposition. IEEE Trans Autom Contr, 64(11):4711–4716. https://doi.org/10.1109/TAC.2019.2902731
Wang YQ, Lu JQ, Zheng WX, et al., 2021. Privacy-preserving consensus for multi-agent systems via node decomposition strategy. IEEE Trans Circ Syst I Regular Papers, 68(8):3474–3484. https://doi.org/10.1109/TCSI.2021.3081372
Wen GX, Li B, 2022. Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems. IEEE Trans Syst Man Cybern Syst, 52(9):5546–5555. https://doi.org/10.1109/TSMC.2021.3130070
Xie ML, Ding DR, Ge XH, et al., 2022. Distributed platooning control of automated vehicles subject to replay attacks based on proportional integral observers. IEEE/CAA J Autom Sin, 11(9):1954–1966. https://doi.org/10.1109/JAS.2022.105941
Xu HY, Ni YH, Liu ZX, et al., 2021. Privacy-preserving leader-following consensus via node-augment mechanism. IEEE Trans Circ Syst II Express Briefs, 68(6):2117–2121. https://doi.org/10.1109/TCSII.2020.3047850
Yan YM, Chen ZY, Varadharajan V, et al., 2021. Distributed consensus-based economic dispatch in power grids using the Paillier cryptosystem. IEEE Trans Smart Grid, 12(4):3493–3502. https://doi.org/10.1109/TSG.2021.3063712
Yang XD, Zhang H, Wang ZP, 2022. Data-based optimal consensus control for multi-agent systems with policy gradient reinforcement learning. IEEE Trans Neur Netw Learn Syst, 33(8):3872–3883. https://doi.org/10.1109/TNNLS.2021.3054685
Yang ZW, Yu LY, Liu YR, et al., 2022. Event-triggered privacy-preserving bipartite consensus for multi-agent systems based on encryption. Neurocomputing, 503:162–172. https://doi.org/10.1016/j.neucom.2022.06.074
Yin TJ, Lv YZ, Yu WW, 2020. Accurate privacy preserving average consensus. IEEE Trans Circ Syst II Express Briefs, 67(4):690–694. https://doi.org/10.1109/TCSII.2019.2918709
Yu T, Ma L, Zhang HW, 2019. Prescribed performance for bipartite tracking control of nonlinear multi-agent systems with hysteresis input uncertainties. IEEE Trans Cybern, 49(4):1327–1338. https://doi.org/10.1109/TCYB.2018.2800297
Zhang HG, Jiang H, Luo YH, et al., 2017. Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method. IEEE Trans Ind Electron, 64(5):4091–4100. https://doi.org/10.1109/TIE.2016.2542134
Zhang JX, Li KW, Li YM, 2021. Output-feedback based simplified optimized backstepping control for strict-feedback systems with input and state constraints. IEEE/CAA J Autom Sin, 8(6):1119–1132. https://doi.org/10.1109/JAS.2021.1004018
Zhang P, Huang T, Sun XQ, et al., 2023. Privacy-preserving and outsourced multi-party K-means clustering based on multi-key fully homomorphic encryption. IEEE Trans Dependab Secure Comput, 20(3):2348–2359. https://doi.org/10.1109/TDSC.2022.3181667
Zhang XM, Han QL, Ge XH, et al., 2023. Sampled-data control systems with non-uniform sampling: a survey of methods and trends. Annu Rev Contr, 55:70–91. https://doi.org/10.1016/j.arcontrol.2023.03.004
Zhou Q, Li HY, Wang LJ, et al., 2018. Prescribed performance observer-based adaptive fuzzy control for nonstrict-feedback stochastic nonlinear systems. IEEE Trans Syst Man Cybern Syst, 48(10):1747–1758. https://doi.org/10.1109/TSMC.2017.2738155
Zuo XJ, Li LX, Peng HP, et al., 2021. Privacy-preserving multidimensional data aggregation scheme without trusted authority in smart grid. IEEE Syst J, 15(1):395–406. https://doi.org/10.1109/JSYST.2020.2994363
Author information
Authors and Affiliations
Contributions
Yang YANG, Fanming HUANG, and Dong YUE designed the research. Yang YANG and Fanming HUANG processed the data and drafted the paper. Dong YUE helped organize the paper. Yang YANG and Fanming HUANG revised and finalized the paper.
Corresponding author
Ethics declarations
All the authors declare that they have no conflict of interest.
Additional information
List of supplementary materials
1 Example 2
Fig. S1 Output consensus tracking performance of \(\breve{x}_{i},\ \breve{y}_{i}\), and \(\breve\psi_{i}\) in Example 2
Fig. S2 Two-dimensional output consensus tracking in Example 2
Fig. S3 Comparison of follower 2’s decryption errors with different methods in Example 2
Project supported by the National Natural Science Foundation of China (Nos. 62473204 and 61873130), the “Chunhui Program” Collaborative Scientific Research Project, China (No. 202202004), the Natural Science Foundation of Nanjing University of Posts and Telecommunications, China (Nos. NY221082, NY222144, and NY223075), and the Huali Program for Excellent Talents in Nanjing University of Posts and Telecommunications, China
Supplementary materials
11714_2025_532_MOESM1_ESM.pdf
Reinforcement learning based privacy-preserving consensus tracking control of nonstrict-feedback discrete-time multi-agent systems
Rights and permissions
About this article
Cite this article
Yang, Y., Huang, F. & Yue, D. Reinforcement learning based privacy-preserving consensus tracking control of nonstrict-feedback discrete-time multi-agent systems. Front Inform Technol Electron Eng 26, 456–471 (2025). https://doi.org/10.1631/FITEE.2300532
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2300532