Abstract
Data privacy is an important issue in control systems, especially when datasets contain sensitive information about individuals. In this paper, the authors are concerned with the differentially private distributed parameter estimation problem, that is, we estimate an unknown parameter while protecting the sensitive information of each agent. First, the authors propose a distributed stochastic approximation estimation algorithm in the form of the differentially private consensus+innovations (DP-CI), and establish the privacy and convergence property of the proposed algorithm. Specifically, it is shown that the proposed algorithm asymptotically unbiased converges in mean-square to the unknown parameter while differential privacy-preserving holds for finite number of iterations. Then, the exponentially damping step-size and privacy noise for DP-CI algorithm is given. The estimate approximately converges to the unknown parameter with an error proportional to the step-size parameter while differential privacy-preserving holds for all iterations. The tradeoff between accuracy and privacy of the algorithm is effectively shown. Finally, a simulation example is provided to verify the effectiveness of the proposed algorithm.
Similar content being viewed by others
References
Wang Y, Zhao Y L, and Zhang J F, Distributed recursive projection identification with binary-valued observations, Journal of Systems Science and Complexity, 2021, 34(5): 2048–2068.
Lourenço I, Mattila R, Rojas C R, et al., Hidden Markov models: Inverse filtering, belief estimation and privacy protections, Journal of Systems Science and Complexity, 2021, 34(5): 2048–2068.
Ma X, Yi P, and Chen J, Distributed gradient tracking methods with finite data rates, Journal of Systems Science and Complexity, 2021, 34(5): 1927–1952.
Kar S and Moura J M F, Convergence rate analysis of distributed gossip (linear parameter) estimation: Fundamental limits and tradeoffs, IEEE Journal of Selected Topics in Signal Processing, 2021, 5(4): 674–690.
Zhang Q and Zhang J F, Distributed parameter estimation over unreliable networks with Markovian switching topologies, IEEE Transactions on Automatic Control, 2012, 57(10): 2545–2560.
Kar S, Moura J M F, and Ramanan K, Distributed parameter estimation in sensor networks: Nonlinear observation models and imperfect communication, IEEE Transactions on Information Theory, 2012, 58(6): 3575–3605.
Kar S, Moura J M F, and Poor H V, Distributed linear parameter estimation: Asymptotically efficient adaptive strategies, SIAM Journal on Control and Optimization, 2013, 51(3): 2200–2229.
You K, Xie L, and Song S, Asymptotically optimal parameter estimation with scheduled measurements, IEEE Transactions on Signal Processing, 2013, 61(14): 3521–3531.
Lin P and Qi H, Distributed gradient-based sampling algorithm for least-squares in switching multi-agent networks, Science China Information Sciences, 2020, 63(9): 199203:1–199203:3.
Chen Y, Kar S, and Moura J M F, Resilient distributed estimation: Exponential convergence under sensor attacks, IEEE Conference on Decision and Control, 2018, 7275–7282
Chen Y, Kar S, and Moura J M F, Resilient distributed estimation through adversary detection, IEEE Transactions on Signal Processing, 2018, 66(9): 2455–2469.
Xiao H C, Ding D R, Dong H L, et al., Adaptive event-triggered state estimation for large-scale systems subject to deception attacks, Science China Information Sciences, 2022, 65: 122207:1–122207:16.
Zhang J F, Tan J W, and Wang J M, Privacy security in control systems, Science China Information Sciences, 2021, 64: 176201:1–176201:3.
Farokhi F, Shames I, and Batterham N, Secure and private control using semi-homomorphic encryption, Control Engineering Practice, 2017, 67: 13–20.
Lu Y and Zhu M H, Privacy preserving distributed optimization using homomorphic encryption, Automatica, 2018, 96: 314–325.
Mo Y L and Murray R M, Privacy preserving average consensus, IEEE Transactions on Automatic Control, 2017, 62(2): 753–765.
Liu X K, Zhang J F, and Wang J M, Differentially private consensus algorithm for continuous-time heterogeneous multi-agent systems, Automatica, 2020, 122: Article 109283.
Dwork C, Differential privacy, Proceedings of the 33rd International Colloquium on Automata, Languages and Programming, 2006, 1–12.
Dwork C, Rothblum G N, and Vadhan S, Boosting and differential privacy, Proceedings of the 51st Annual IEEE Symposium on Foundations of Computer Science, 2010, 51–60.
Liang W J, Chen H, Zhang J, et al., An effective scheme for top-k frequent itemset mining under differential privacy conditions, Science China Information Sciences, 2020, 63: 159101:1–159101:3.
Xue Q, Zhu Y W, Wang J, et al., Locally differentially private distributed algorithms for set intersection and union, Science China Information Sciences, 2021, 64: 219101:1–219101:3
Ding J, Gong Y, Zhang C, et al., Optimal differentially private ADMM for distributed machine learning, arXiv preprint arXiv: 1901.02094v2, 2019.
Li C, Zhou P, Xiong L, et al., Differentially private distributed online learning, IEEE Transactions on Knowledge and Data Engineering, 2018, 30(8): 1440–1453.
Zhu J L, Xu C Q, Guan J F, et al., Differentially private distributed online algorithms over time-varying directed networks, IEEE Transactions on Signal and Information Processing over Networks, 2018, 4(1): 4–17.
Huang Z, Mitra S, and Vaidya N, Differentially private distributed optimization, Proceedings of the 16th International Conference on Distributed Computing and Networking, 2015, 4:1–4:10.
Han S, Topcu U, and Pappas G J, Differentially private distributed constrained optimization, IEEE Transactions on Automatic Control, 2017, 62(1): 50–64.
Showkatbakhsh M, Karakus C, and Diggavi S, Differentially private consensus-based distributed optimization, arXiv preprint arXiv: 1903.07792v1, 2019.
Wang Y, Huang Z, Mitra S, et al., Differential privacy in linear distributed control systems: Entropy minimizing mechanisms and performance tradeoffs, IEEE Transactions on Control of Network Systems, 2017, 4(1): 118–130.
Huang Z, Mitra S, and Dullerud G E, Differentially private iterative synchronous consensus, Proceeding of CCS Workshop on Privacy in the Electronic Society, USA, 2012.
Gao L, Deng S, and Ren W, Differentially private consensus with event-triggered mechanism, IEEE Transactions on Control of Network Systems, 2019, 6(1): 60–71.
Wang A, Liao X F, and He H B, Event-triggered differentially private average consensus for multi-agent network, IEEE/CAA Journal of Automatica Sinica, 2019, 6(1): 75–83.
Nozari E, Tallapragada P, and Cortes J, Differentially private average consensus: Obstructions, trade-offs, and optimal algorithm design, Automatica, 2017, 81: 221–231.
Fiore D and Russo G, Resilient consensus for multi-agent systems subject to differential privacy requirements, Automatica, 2019, 106: 18–26.
Ny J L and Pappas G J, Differentially private filtering, IEEE Transactions on Automatic Control, 2014, 59(2): 341–354.
Katewa V, Chakrabortty A, and Gupta V, Differential privacy for network identification, IEEE Transactions on Control of Network Systems, 2020, 7(1): 266–277.
Song S, Chaudhuri K, and Sarwate A D, Stochastic gradient descent with differentially private updates, Proceedings of the Global Conference on Signal and Information Processing, 2013, 245–248.
Liu Y, Liu J, and Başar T, Differentially private gossip gradient descent, IEEE Conference on Decision and Control, 2018, 2777–2782.
Liu Y, Zhang X, Qin S, et al., Differentially private linear regression over fully decentralized datasets, 33rd Conference on Neural Information Processing Systems (NeurIPS), 2019.
Hale M T and Egerstedt M, Cloud-enabled differentially private multiagent optimization with constraints, IEEE Transactions on Control of Network Systems, 2018, 5(4): 1693–1706.
Goodwin G and Sin K, Adaptive Filtering, Prediction and Control, Englewood Cliffs, N.J.: Prentice-Hall, 1984.
Mehran M and Magnus E, Graph Theoretic Methods in Multiagent Network, Princeton: Princeton University Press, 2010.
Author information
Authors and Affiliations
Corresponding author
Additional information
The work is supported by the National Key R&D Program of China under Grant No. 2018YFA0703800, the National Natural Science Foundation of China under Grant No. 61877057, and China Post-Doctoral Science Foundation under Grant No. 2018M641506.
This paper was recommended for publication by Editor WU Zhengguang.
Rights and permissions
About this article
Cite this article
Wang, J., Tan, J. & Zhang, JF. Differentially Private Distributed Parameter Estimation. J Syst Sci Complex 36, 187–204 (2023). https://doi.org/10.1007/s11424-022-2012-9
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-022-2012-9