Skip to main content
SpringerLink
Log in

Differentially Private Distributed Parameter Estimation

  • Published:
Journal of Systems Science and Complexity Aims and scope Submit manuscript

Abstract

Data privacy is an important issue in control systems, especially when datasets contain sensitive information about individuals. In this paper, the authors are concerned with the differentially private distributed parameter estimation problem, that is, we estimate an unknown parameter while protecting the sensitive information of each agent. First, the authors propose a distributed stochastic approximation estimation algorithm in the form of the differentially private consensus+innovations (DP-CI), and establish the privacy and convergence property of the proposed algorithm. Specifically, it is shown that the proposed algorithm asymptotically unbiased converges in mean-square to the unknown parameter while differential privacy-preserving holds for finite number of iterations. Then, the exponentially damping step-size and privacy noise for DP-CI algorithm is given. The estimate approximately converges to the unknown parameter with an error proportional to the step-size parameter while differential privacy-preserving holds for all iterations. The tradeoff between accuracy and privacy of the algorithm is effectively shown. Finally, a simulation example is provided to verify the effectiveness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wang Y, Zhao Y L, and Zhang J F, Distributed recursive projection identification with binary-valued observations, Journal of Systems Science and Complexity, 2021, 34(5): 2048–2068.

    Article  MathSciNet  MATH  Google Scholar 

  2. Lourenço I, Mattila R, Rojas C R, et al., Hidden Markov models: Inverse filtering, belief estimation and privacy protections, Journal of Systems Science and Complexity, 2021, 34(5): 2048–2068.

    Article  MathSciNet  MATH  Google Scholar 

  3. Ma X, Yi P, and Chen J, Distributed gradient tracking methods with finite data rates, Journal of Systems Science and Complexity, 2021, 34(5): 1927–1952.

    Article  MathSciNet  MATH  Google Scholar 

  4. Kar S and Moura J M F, Convergence rate analysis of distributed gossip (linear parameter) estimation: Fundamental limits and tradeoffs, IEEE Journal of Selected Topics in Signal Processing, 2021, 5(4): 674–690.

    Article  Google Scholar 

  5. Zhang Q and Zhang J F, Distributed parameter estimation over unreliable networks with Markovian switching topologies, IEEE Transactions on Automatic Control, 2012, 57(10): 2545–2560.

    Article  MathSciNet  MATH  Google Scholar 

  6. Kar S, Moura J M F, and Ramanan K, Distributed parameter estimation in sensor networks: Nonlinear observation models and imperfect communication, IEEE Transactions on Information Theory, 2012, 58(6): 3575–3605.

    Article  MathSciNet  MATH  Google Scholar 

  7. Kar S, Moura J M F, and Poor H V, Distributed linear parameter estimation: Asymptotically efficient adaptive strategies, SIAM Journal on Control and Optimization, 2013, 51(3): 2200–2229.

    Article  MathSciNet  MATH  Google Scholar 

  8. You K, Xie L, and Song S, Asymptotically optimal parameter estimation with scheduled measurements, IEEE Transactions on Signal Processing, 2013, 61(14): 3521–3531.

    Article  MathSciNet  MATH  Google Scholar 

  9. Lin P and Qi H, Distributed gradient-based sampling algorithm for least-squares in switching multi-agent networks, Science China Information Sciences, 2020, 63(9): 199203:1–199203:3.

    Article  Google Scholar 

  10. Chen Y, Kar S, and Moura J M F, Resilient distributed estimation: Exponential convergence under sensor attacks, IEEE Conference on Decision and Control, 2018, 7275–7282

    Google Scholar 

  11. Chen Y, Kar S, and Moura J M F, Resilient distributed estimation through adversary detection, IEEE Transactions on Signal Processing, 2018, 66(9): 2455–2469.

    Article  MathSciNet  MATH  Google Scholar 

  12. Xiao H C, Ding D R, Dong H L, et al., Adaptive event-triggered state estimation for large-scale systems subject to deception attacks, Science China Information Sciences, 2022, 65: 122207:1–122207:16.

    Article  MathSciNet  Google Scholar 

  13. Zhang J F, Tan J W, and Wang J M, Privacy security in control systems, Science China Information Sciences, 2021, 64: 176201:1–176201:3.

    Article  Google Scholar 

  14. Farokhi F, Shames I, and Batterham N, Secure and private control using semi-homomorphic encryption, Control Engineering Practice, 2017, 67: 13–20.

    Article  Google Scholar 

  15. Lu Y and Zhu M H, Privacy preserving distributed optimization using homomorphic encryption, Automatica, 2018, 96: 314–325.

    Article  MathSciNet  MATH  Google Scholar 

  16. Mo Y L and Murray R M, Privacy preserving average consensus, IEEE Transactions on Automatic Control, 2017, 62(2): 753–765.

    Article  MathSciNet  MATH  Google Scholar 

  17. Liu X K, Zhang J F, and Wang J M, Differentially private consensus algorithm for continuous-time heterogeneous multi-agent systems, Automatica, 2020, 122: Article 109283.

  18. Dwork C, Differential privacy, Proceedings of the 33rd International Colloquium on Automata, Languages and Programming, 2006, 1–12.

    Google Scholar 

  19. Dwork C, Rothblum G N, and Vadhan S, Boosting and differential privacy, Proceedings of the 51st Annual IEEE Symposium on Foundations of Computer Science, 2010, 51–60.

    Google Scholar 

  20. Liang W J, Chen H, Zhang J, et al., An effective scheme for top-k frequent itemset mining under differential privacy conditions, Science China Information Sciences, 2020, 63: 159101:1–159101:3.

    Article  Google Scholar 

  21. Xue Q, Zhu Y W, Wang J, et al., Locally differentially private distributed algorithms for set intersection and union, Science China Information Sciences, 2021, 64: 219101:1–219101:3

    Article  MathSciNet  Google Scholar 

  22. Ding J, Gong Y, Zhang C, et al., Optimal differentially private ADMM for distributed machine learning, arXiv preprint arXiv: 1901.02094v2, 2019.

    Google Scholar 

  23. Li C, Zhou P, Xiong L, et al., Differentially private distributed online learning, IEEE Transactions on Knowledge and Data Engineering, 2018, 30(8): 1440–1453.

    Article  Google Scholar 

  24. Zhu J L, Xu C Q, Guan J F, et al., Differentially private distributed online algorithms over time-varying directed networks, IEEE Transactions on Signal and Information Processing over Networks, 2018, 4(1): 4–17.

    Article  MathSciNet  Google Scholar 

  25. Huang Z, Mitra S, and Vaidya N, Differentially private distributed optimization, Proceedings of the 16th International Conference on Distributed Computing and Networking, 2015, 4:1–4:10.

    Google Scholar 

  26. Han S, Topcu U, and Pappas G J, Differentially private distributed constrained optimization, IEEE Transactions on Automatic Control, 2017, 62(1): 50–64.

    Article  MathSciNet  MATH  Google Scholar 

  27. Showkatbakhsh M, Karakus C, and Diggavi S, Differentially private consensus-based distributed optimization, arXiv preprint arXiv: 1903.07792v1, 2019.

    Google Scholar 

  28. Wang Y, Huang Z, Mitra S, et al., Differential privacy in linear distributed control systems: Entropy minimizing mechanisms and performance tradeoffs, IEEE Transactions on Control of Network Systems, 2017, 4(1): 118–130.

    Article  MathSciNet  MATH  Google Scholar 

  29. Huang Z, Mitra S, and Dullerud G E, Differentially private iterative synchronous consensus, Proceeding of CCS Workshop on Privacy in the Electronic Society, USA, 2012.

    Google Scholar 

  30. Gao L, Deng S, and Ren W, Differentially private consensus with event-triggered mechanism, IEEE Transactions on Control of Network Systems, 2019, 6(1): 60–71.

    Article  MathSciNet  MATH  Google Scholar 

  31. Wang A, Liao X F, and He H B, Event-triggered differentially private average consensus for multi-agent network, IEEE/CAA Journal of Automatica Sinica, 2019, 6(1): 75–83.

    Article  MathSciNet  Google Scholar 

  32. Nozari E, Tallapragada P, and Cortes J, Differentially private average consensus: Obstructions, trade-offs, and optimal algorithm design, Automatica, 2017, 81: 221–231.

    Article  MathSciNet  MATH  Google Scholar 

  33. Fiore D and Russo G, Resilient consensus for multi-agent systems subject to differential privacy requirements, Automatica, 2019, 106: 18–26.

    Article  MathSciNet  MATH  Google Scholar 

  34. Ny J L and Pappas G J, Differentially private filtering, IEEE Transactions on Automatic Control, 2014, 59(2): 341–354.

    Article  MathSciNet  MATH  Google Scholar 

  35. Katewa V, Chakrabortty A, and Gupta V, Differential privacy for network identification, IEEE Transactions on Control of Network Systems, 2020, 7(1): 266–277.

    Article  MathSciNet  MATH  Google Scholar 

  36. Song S, Chaudhuri K, and Sarwate A D, Stochastic gradient descent with differentially private updates, Proceedings of the Global Conference on Signal and Information Processing, 2013, 245–248.

    Google Scholar 

  37. Liu Y, Liu J, and Başar T, Differentially private gossip gradient descent, IEEE Conference on Decision and Control, 2018, 2777–2782.

    Google Scholar 

  38. Liu Y, Zhang X, Qin S, et al., Differentially private linear regression over fully decentralized datasets, 33rd Conference on Neural Information Processing Systems (NeurIPS), 2019.

    Google Scholar 

  39. Hale M T and Egerstedt M, Cloud-enabled differentially private multiagent optimization with constraints, IEEE Transactions on Control of Network Systems, 2018, 5(4): 1693–1706.

    Article  MathSciNet  MATH  Google Scholar 

  40. Goodwin G and Sin K, Adaptive Filtering, Prediction and Control, Englewood Cliffs, N.J.: Prentice-Hall, 1984.

    MATH  Google Scholar 

  41. Mehran M and Magnus E, Graph Theoretic Methods in Multiagent Network, Princeton: Princeton University Press, 2010.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, 100083, China

    Jimin Wang

  2. Key Laboratory of Systems and Control, Institute of Systems Science, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China

    Jianwei Tan & Ji-Feng Zhang

  3. School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China

    Jianwei Tan & Ji-Feng Zhang

Authors
  1. Jimin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianwei Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ji-Feng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ji-Feng Zhang.

Additional information

The work is supported by the National Key R&D Program of China under Grant No. 2018YFA0703800, the National Natural Science Foundation of China under Grant No. 61877057, and China Post-Doctoral Science Foundation under Grant No. 2018M641506.

This paper was recommended for publication by Editor WU Zhengguang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Tan, J. & Zhang, JF. Differentially Private Distributed Parameter Estimation. J Syst Sci Complex 36, 187–204 (2023). https://doi.org/10.1007/s11424-022-2012-9

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11424-022-2012-9

Keywords

Search

Navigation