Skip to main content
SpringerLink
Log in

Privacy-Preserving Dual Stochastic Push-Sum Algorithm for Distributed Constrained Optimization

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

This paper investigates a private distributed optimization problem over a multi-agent network, where the goal is to cooperatively minimize the sum of all locally convex cost functions subject to coupled equality constraints over time-varying unbalanced directed networks while considering privacy concerns. To solve this problem, we integrate push-sum protocols with dual subgradient methods to design a private distributed dual stochastic push-sum algorithm. Under the assumption of convexity, we first establish the convergence of the algorithm in terms of dual variables, primal variables and constraint violations. Then we show that the algorithm has a sub-linear growth with order of \(O(\ln t/\sqrt{t})\). The result reveals that there is a tradeoff between the privacy level and the accuracy of the algorithm. Finally, the efficiency of the algorithm is verified numerically over two applications to the economic dispatch problems and electric vehicles charging control problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Aybat, N.S., Fallah, A., Gürbüzbalaban, M., A. Ozdaglar.: Robust accelerated gradient methods for smooth strongly convex functions. SIAM J. Optim. 30(1), 717-751 (2020)

  2. Aybat, N.S., Hamedani, E.Y.: A distributed ADMM-like method for resource sharing over time-varying networks. SIAM J. Optim. 29(4), 3036–3068 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  3. Beck, A., Nedić, A., Ozdaglar, A., Teboulle, M.: An \( O (1/k) \) gradient method for network resource allocation problems. IEEE Trans. Control of Netw. Syst. 1(1), 64–73 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bekkerman, R., Bilenko, M., Langford, J.: Scaling up machine learning: parallel and distributed approaches. Cambridge University Press, Cambridge (2011)

    Book  Google Scholar 

  5. Blondin, M.J., Hale, M.: A decentralized multi-objective optimization algorithm. J. Optim. Theory Appl. 189(2), 458–485 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  6. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  7. Chang, T.H., Nedić, A., Scaglione, A.: Distributed constrained optimization by consensus-based primal-dual perturbation method. IEEE Trans. Autom. Control. 59(6), 1524–1538 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dong, T., Bu, X., Hu, W.: Distributed differentially private average consensus for multi-agent networks by additive functional Laplace noise. J. Franklin Inst. 357(6), 3565–3584 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  9. Du, B., Zhou, J., Sun, D.: Improving the convergence of distributed gradient descent via inexact average consensus. J. Optim. Theory Appl. 185(2), 504–521 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  10. Duchi, J.C., Agarwal, A., Wainwright, M.J.: Dual averaging for distributed optimization: convergence analysis and network scaling. IEEE Trans. Autom. Control. 57(3), 592–606 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Theory of Cryptography Conference, pp. 265-284. Springer, Berlin, Heidelberg (2006)

  12. Falsone, A., Margellos, K., Garatti, S., Prandini, M.: Dual decomposition for multi-agent distributed optimization with coupling constraints. Automatica 84, 149–158 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gu, C., Li, J., Wu, Z.: An adaptive online learning algorithm for distributed convex optimization with coupled constraints over unbalanced directed graphs. J. Franklin Inst. 356(13), 7548–7570 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  14. Huang, Z., Mitra, S., Vaidya, N.: Differentially private distributed optimization. In: 2015 16th International Conference on Distributed Computing and Networking, pp. 1-10 (2015)

  15. Jain, P., Kothari, P., Thakurta, A.: Differentially private online learning. In: Conference on Learning Theory. JMLR Workshop and Conference Proceedings, pp. 24-1 (2012)

  16. Li, J., Wu, Z., Wu, C., Long, Q., Wang, X.: An inexact dual fast gradient-projection method for separable convex optimization with linear coupled constraints. J. Optim. Theory Appl. 168(1), 153–171 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Li, C., Zhou, P., Xiong, L., Wang, Q., Wang, T.: Differentially private distributed online learning. IEEE Trans. Knowl. Data En. 30(8), 1440–1453 (2018)

    Article  Google Scholar 

  18. Li, H., Zhang, H., Wang, Z., Zhu, Y., Han, Q.: Distributed consensus-based multi-agent convex optimization via gradient tracking technique. J. Franklin Inst. 356(6), 3733–3761 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  19. Li, X., Yi, X., Xie, L.: Distributed online optimization for multi-agent networks with coupled inequality constraints. IEEE Trans. Autom. Control. 66(8), 3575–3591 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  20. Lou, Y., Yu, L., Wang, S., Yi, P.: Privacy preservation in distributed subgradient optimization algorithms. IEEE Trans. Cybern. 48(7), 2154–2165 (2017)

    Article  Google Scholar 

  21. Lü, Q., Liao, X., Xiang, T., Li, H., Huang, T.: Privacy masking stochastic subgradient-push algorithm for distributed online optimization. IEEE Trans. Cybern. 51(6), 3224–3237 (2020)

    Article  Google Scholar 

  22. Mao, S., Tang, Y., Dong, Z., Meng, K., Dong, Z.Y., Qian, F.: A privacy preserving distributed optimization algorithm for economic dispatch over time-varying directed networks. IEEE Trans. Ind. Inform. 17(3), 1689–1701 (2020)

    Article  Google Scholar 

  23. McSherry, F.D.: Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In: 2009 35th ACM SIGMOD International Conference on Management of Data, pp. 19-30 (2009)

  24. Nedić, A., Olshevsky, A.: Distributed optimization over time-varying directed graphs. IEEE Trans. Autom. Control. 60(3), 601–615 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  25. Nedić, A., Olshevsky, A.: Stochastic gradient-push for strongly convex functions on time-varying directed graphs. IEEE Trans. Autom. Control 12(61), 3936–3947 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  26. Nedić, A., Ozdaglar, A.: Distributed sub-gradient methods for multi-agent optimization. IEEE Trans. Autom. Control. 54(1), 48–61 (2009)

    Article  MATH  Google Scholar 

  27. Nozari, E., Tallapragada, P., Cortés, J.: Differentially private distributed convex optimization via functional perturbation. IEEE Trans. Control Netw. Syst. 5(1), 395–408 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  28. Ram, S.S., Nedić, A., Veeravalli, V.V.: Distributed stochastic subgradient projection algorithms for convex optimization. J. Optim. Theory Appl. 147(3), 516–545 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  29. Shi, Z., Zhou, C.: An improved distributed gradient-push algorithm for bandwidth resource allocation over wireless local area network. J. Optim. Theory Appl. 183(3), 1153–1176 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  30. Simonetto, A., Jamali-Rad, H.: Primal recovery from consensus-based dual decomposition for distributed convex optimization. J. Optim. Theory Appl. 168(1), 172–197 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  31. Wang, Y., Hale, M., Egerstedt, M., Dullerud, G.: Differentially private objective functions in distributed cloud-based optimization. In: 2016 55th IEEE Conference on Decision and Control, pp. 3688-3694. IEEE (2016)

  32. Wang, S., Li, C.: Distributed stochastic algorithm for global optimization in networked system. J. Optim. Theory Appl. 179(3), 1001–1007 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  33. Wang, A., Liu, W., Li, T., Huang, T.: Privacy-preserving weighted average consensus and optimal attacking strategy for multi-agent networks. J. Franklin Inst. 358(6), 3033–3050 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  34. Xia, X., Elaiw, A.M.: Optimal dynamic economic dispatch of generation: a review. Electr. Power Syst. Res. 80(8), 975–986 (2010)

    Article  Google Scholar 

  35. Yuan, D., Xu, S., Zhao, H.: Distributed primal-dual subgradient method for multiagent optimization via consensus algorithms. IEEE Trans. Syst. Man. Cybern. B. Cybern. 41(6), 1715-1724 (2011)

  36. Zhang, T., Zhu, Q.: Dynamic differential privacy for ADMM-based distributed classification learning. IEEE Trans. Inf. Forensics Security 12(1), 172–187 (2016)

    Article  Google Scholar 

  37. Zhang, B., Gu, C., Li, J.: Distributed convex optimization with coupling constraints over time-varying directed graphs. J. Ind. Manag. Optim. 17(4), 2119–2138 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  38. Zhu, J., Xu, C., Guan, J., Wu, D.O.: Differentially private distributed online algorithms over time-varying directed networks. IEEE Trans. Signal Process 4(1), 4–17 (2018)

    MathSciNet  Google Scholar 

Download references

Acknowledgements

The work was supported in part by the National Natural Science Foundation of China under grants 11991024 and 11971083, and in part by the Natural Science Foundation Projection of Chongqing under grant cstc2020jcyj-msxmX0287 and in part by the Project for Creative Research Groups at Institutions of Higher Education in Chongqing under grant CXQT20014.

Author information

Authors and Affiliations

  1. School of Management, Guangzhou University, Guangzhou, 510006, China

    Chuanye Gu

  2. Department of Mathematics and Statistics, Curtin University, Perth, WA, 6845, Australia

    Lin Jiang

  3. School of Mathematical Sciences, Chongqing Normal University, Chongqing, 401331, China

    Jueyou Li & Zhiyou Wu

Authors
  1. Chuanye Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jueyou Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiyou Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jueyou Li.

Additional information

Communicated by Kyriakos G. Vamvoudakis.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gu, C., Jiang, L., Li, J. et al. Privacy-Preserving Dual Stochastic Push-Sum Algorithm for Distributed Constrained Optimization. J Optim Theory Appl 197, 22–50 (2023). https://doi.org/10.1007/s10957-023-02173-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-023-02173-9

Keywords

Mathematics Subject Classification

Search

Navigation