Abstract
This paper investigates the distributed convex optimization problem over a multi-agent system with Markovian switching communication networks. The objective function is the sum of each agent’s local nonsmooth objective function, which cannot be known by other agents. The communication network is assumed to switch over a set of weight-balanced directed graphs with a Markovian property. The authors propose a consensus sub-gradient algorithm with two time-scale step-sizes to handle the Markovian switching topologies and the absence of global gradient information. With proper selection of step-sizes, the authors prove the almost sure convergence of all agents’ local estimates to the same optimal solution when the union graph of the Markovian network’ states is strongly connected and the Markovian chain is irreducible. The convergence rate analysis is also given for specific cases. Simulations are given to demonstrate the results.
Similar content being viewed by others
References
Zhang Q and Zhang J F, Distributed parameter estimation over unreliable networks with markovian switching topologies, IEEE Transactions on Automatic Control, 2012, 57(10): 2545–2560.
Lei J, CHen H F, and Fang H T, Primal-dual algorithm for distributed constrained optimization, Systems & Control Letters, 2016, 96: 110–117.
Wei E, Ozdaglar A, and Jadbabaie A, A distributed newton method for network utility maximization — I: Algorithm, IEEE Transactions on Automatic Control, 2013, 58(9): 2162–2175.
Shi W, Ling Q, Wu G, et al, Extra: An exact first-order algorithm for decentralized consensus optimization, SIAM Journal on Optimization, 2015, 25(2): 944–966.
Dall’Anese E, Zhu H, and Giannakis G B, Distributed optimal power flow for smart microgrids, IEEE Transactions on Smart Grid, 2013, 4(3): 1464–1475.
Yi P, Hong Y, and Liu F, Initialization-free distributed algorithms for optimal resource allocation with feasibility constraints and application to economic dispatch of power systems, Automatica, 2016, 74: 259–269.
Nedic A and Ozdaglar A, Distributed subgradient methods for multi-agent optimization, IEEE Transactions on Automatic Control, 2009, 54(1): 48–61.
Lobel I and Ozdaglar A, Distributed subgradient methods for convex optimization over random networks, IEEE Transactions on Automatic Control, 2010, 56(6): 1291–1306.
Duchi J C, Agarwal A, and Wainwright M J, Dual averaging for distributed optimization: Convergence analysis and network scaling, IEEE Transactions on Automatic control, 2011, 57(3): 592–606.
Lei J, Chen H F, and Fang H T, Asymptotic properties of primal-dual algorithm for distributed stochastic optimization over random networks with imperfect communications, SIAM Journal on Control and Optimization, 2018, 56(3): 2159–2188.
Xu J, Zhu S, Soh Y C, and Xie L, Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes, 2015 54th IEEE Conference on Decision and Control (CDC), 2015, 2055–2060.
Pu S, Shi W, Xu J, et al, A push-pull gradient method for distributed optimization in networks, 2018 IEEE Conference on Decision and Control (CDC), 2018, 3385–3390.
Sayed A H, Adaptation, learning, and optimization over networks, Foundations and Trends in Machine Learning, 2014, 7(4–5): 311–801.
Nedich A, Convergence rate of distributed averaging dynamics and optimization in networks, Foundations and Trends in Systems and Control, 2015, 2(1): 1–100.
Nedic A and Liu J, Distributed optimization for control, Annual Review of Control, Robotics, and Autonomous Systems, 2018, 1: 77–103.
Yang T, Yi X L, Wu J F, et al., A survey of distributed optimization, Annual Reviews in Control, 2019, 47: 278–305.
Notarstefano G, Notarnicola I, Camisa A, et al., Distributed optimization for smart cyber-physical networks, Foundations and Trends in Systems and Control, 2019, 7(3): 253–383.
Yi P and Hong Y, Quantized subgradient algorithm and data-rate analysis for distributed optimization, IEEE Transactions on Control of Network Systems, 2014, 1(4): 380–392.
Xu J, Zhu S, Soh Y C, et al., Convergence of asynchronous distributed gradient methods over stochastic networks, IEEE Transactions on Automatic Control, 2017, 63(2): 434–448.
Nedic A, Asynchronous broadcast-based convex optimization over a network, IEEE Transactions on Automatic Control, 2010, 56(6): 1337–1351.
Lu J, Tang C Y, Regier P R, et al., Gossip algorithms for convex consensus optimization over networks, IEEE Transactions on Automatic Control, 2011, 56(12): 2917–2923.
Jakovetic D, Bajovic D, Sahu A K, et al., Convergence rates for distributed stochastic optimization over random networks, 2018 IEEE Conference on Decision and Control (CDC), 2018, 4238–4245.
Yi P, Lei J, and Hong Y, Distributed resource allocation over random networks based on stochastic approximation, Systems & Control Letters, 2018, 114: 44–51.
Huang M, Dey S, Nair G N, et al., Stochastic consensus over noisy networks with markovian and arbitrary switches, Automatica, 2010, 46(10): 1571–1583.
Matei I, Baras J S, and Somarakis C, Convergence results for the linear consensus problem under markovian random graphs, SIAM Journal on Control and Optimization, 2013, 51(2): 1574–1591.
Li T and Wang J, Distributed averaging with random network graphs and noises, IEEE Transactions on Information Theory, 2018, 64(11): 7063–7080.
Xiao N, Xie L, and Fu M, Kalman filtering over unreliable communication networks with bounded markovian packet dropouts, International Journal of Robust and Nonlinear Control: IFAC-Affiliated Journal, 2009, 19(16): 1770–1786.
Lobel I, Ozdaglar A, and Feijer D, Distributed multi-agent optimization with state-dependent communication, Mathematical Programming, 2011, 129(2): 255–284.
Alaviani S S and Elia N, Distributed multi-agent convex optimization over random digraphs, IEEE Transactions on Automatic Control, 2019, 65(3): 986–998.
Chen H F, Stochastic Approximation and Its Applications, Springer Science & Business Media, New York, 2006.
Acknowledgements
The authors would like to thank Professor LEI Jinlong for her helpful discussions, and thank YU Yang for his help in the simulations.
Author information
Authors and Affiliations
Corresponding authors
Additional information
The paper was sponsored by Shanghai Sailing Program under Grant Nos. 20YF1453000 and 20YF1452800, the National Natural Science Foundation of China under Grant No. 62003239, and the Fundamental Research Funds for the Central Universities under Grant Nos. 22120200047 and 22120200048.
This paper was recommended for publication by Editor LI Chanying.
Rights and permissions
About this article
Cite this article
Yi, P., Li, L. Distributed Nonsmooth Convex Optimization over Markovian Switching Random Networks with Two Step-Sizes. J Syst Sci Complex 34, 1324–1344 (2021). https://doi.org/10.1007/s11424-020-0071-3
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-020-0071-3