Zusammenfassung
Diese Arbeit bietet einen Überblick über Methoden, die verteilte und spieltheoretische Optimierungsprobleme in Multi-Agenten-Systemen lösen. Alle betrachteten Methoden basieren auf der Annahme, dass die kritischen Informationen im System auf die einzelnen Agenten verteilt sind und kein Agent Zugriff auf die Gesamtinformation besitzt. Jeder Agent kann allerdings Metadaten seiner lokalen Information seinen Nachbarn preisgeben, sodass ein gemeinsames Ziel erreicht werden kann. Die Nachbarschaften sind dabei durch einen Kommunikationsgraphen festgelegt. Der Fokus liegt daher auf kommunikationsbasierten Verfahren, die schnelle Konvergenzraten aufweisen. Dabei müssen die einzelnen Zielfunktionen der Agenten streng konvex und ihre Gradienten Lipschitz-stetig sein. In der Literatur wurden schnelle Methoden für die verteilte Optimierung bereits ausgiebig behandelt. Es bleiben jedoch viele offene Fragen im Bereich der spieltheoretischen Optimierung. Diese Arbeit verfolgt das Ziel, einen strukturierten Vergleich zwischen den bekannten Ergebnissen für diese Optimierungsprobleme zu schaffen und potentielle Richtungen für die zukünftige Forschung zu formulieren.
Abstract
This work provides an overview of the methods that solve distributed and game-theoretic optimization problems in multi-agent systems. All considered methods are based on the assumption that each agent has access only to some local information but not to the whole information in the system. However, each agent can communicate its local information with neighbors to achieve a common goal. The neighborhoods are defined through a communication graph. The focus is therefore on communication-based processes that possess rapid convergence to a system’s optimum. The individual cost functions of the agents are assumed to be strictly convex with Lipschitz continuous gradients. Some fast methods for distributed optimization have been already presented in the literature. However, there are many open questions related to game-theoretic optimization. This work aims to provide a structured comparison between the known results for these two types of optimization problems and formulate potential directions for the future research.
Über den Autor / die Autorin
Tatiana Tatarenko received the Dipl.-Math. degree in mathematics with focus on Statistics and Stochastic Processes from Lomonosov Moscow State University, Moscow, Russia, in 2011, and the Ph.D. degree from Technische Universität (TU) Darmstadt, Germany, in 2017. She is currently a Postdoctoral Researcher with the Control Methods and Robotics Laboratory at TU Darmstadt, Germany. Her main research interests include distributed optimization, game theoretic learning, and stochastic processes in networked multiagent systems.
Literaturverzeichnis
1. D.P. Bertsekas. Nonlinear Programming. Athena Scientific, 1999.Search in Google Scholar
2. P. Bianchi, G. Fort and W. Hachem. Performance of a distributed stochastic approximation algorithm. IEEE Transactions on Information Theory, 59(11):7405–7418, Nov 2013.10.1109/TIT.2013.2275131Search in Google Scholar
3. Z. Deng and X. Nian. Distributed generalized nash equilibrium seeking algorithm design for aggregative games over weight-balanced digraphs. IEEE Transactions on Neural Networks and Learning Systems, 30(3):695–706, March 2019.10.1109/TNNLS.2018.2850763Search in Google Scholar PubMed
4. P. Frihauf, M. Krstić and T. Başar. Nash equilibrium seeking in noncooperative games. Automatic Control, IEEE Transactions on, 57(5):1192–1207, 2012.10.1109/TAC.2011.2173412Search in Google Scholar
5. D. Gadjov and L. Pavel. A passivity-based approach to nash equilibrium seeking over networks. IEEE Transactions on Automatic Control, 64(3):1077–1092, March 2019.10.1109/TAC.2018.2833140Search in Google Scholar
6. D. Jakovetic, J. Xavier and J. M. F. Moura. Cooperative convex optimization in networked systems: Augmented lagrangian algorithms with directed gossip communication. IEEE Transactions on Signal Processing, 59(8):3889–3902, Aug 2011.10.1109/TSP.2011.2146776Search in Google Scholar
7. J. Koshal, A. Nedić and U. Shanbhag. A Gossip Algorithm for Aggregative Games on Graphs. In Decision and Control (CDC), 2012 IEEE 51st Annual Conference on, pages 4840–4845. IEEE, 2012.10.1109/CDC.2012.6426136Search in Google Scholar
8. S. Lee and A. Nedic. Distributed random projection algorithm for convex optimization. IEEE Journal of Selected Topics in Signal Processing, 7(2):221–229, April 2013.10.1109/JSTSP.2013.2247023Search in Google Scholar
9. N. Li and J. Marden. Designing Games for Distributed Optimization. IEEE Journal of Selected Topics in Signal Processing, 7(2):230–242, 2013.10.1109/CDC.2011.6161053Search in Google Scholar
10. I. Lobel and A.E. Ozdaglar. Distributed subgradient methods for convex optimization over random networks. IEEE Trans. Automat. Contr., 56(6):1291–1306, 2011.10.1109/TAC.2010.2091295Search in Google Scholar
11. J. R. Marden and J. S. Shamma. Revisiting log-linear learning: Asynchrony, completeness and payoff-based implementation. Games and Economic Behavior, 75(2):788–808, 2012.10.1109/ALLERTON.2010.5707044Search in Google Scholar
12. A. Nedić and A. Olshevsky. Distributed optimization over time-varying directed graphs. IEEE Trans. Automat. Contr., 60(3):601–615, 2015.10.1109/TAC.2014.2364096Search in Google Scholar
13. A. Nedic, A. Ozdaglar and P. A. Parrilo. Constrained consensus and optimization in multi-agent networks. IEEE Transactions on Automatic Control, 55(4):922–938, April 2010.10.1109/TAC.2010.2041686Search in Google Scholar
14. A. Nedić and A.E. Ozdaglar. Distributed subgradient methods for multi-agent optimization. IEEE Trans. Automat. Contr., 54(1):48–61, 2009.10.1109/TAC.2008.2009515Search in Google Scholar
15. A. Nedić and A. Olshevsky. Stochastic gradient-push for strongly convex functions on time-varying directed graphs. IEEE Transactions on Automatic Control, 61(12):3936–3947, Dec 2016.10.1109/TAC.2016.2529285Search in Google Scholar
16. A. Nedić, A. Olshevsky and W. Shi. Achieving geometric convergence for distributed optimization over time-varying graphs. SIAM Journal on Optimization, 27(4):2597–2633, 2017.10.1137/16M1084316Search in Google Scholar
17. Yu. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course, volume 87. Springer, 2004.10.1007/978-1-4419-8853-9Search in Google Scholar
18. Yu. Nesterov and L. Scrimali. Solving strongly monotone variational and quasi-variational inequalities. Discrete and Continuous Dynamical Systems – A, 31(4):1383–1396, 2006.10.3934/dcds.2011.31.1383Search in Google Scholar
19. D. Paccagnan, M. Kamgarpour and J. Lygeros. On aggregative and mean field games with applications to electricity markets. In 2016 European Control Conference (ECC), pages 196–201, June 2016.10.1109/ECC.2016.7810286Search in Google Scholar
20. J.-S. Pang and F. Facchinei. Finite-dimensional variational inequalities and complementarity problems : vol. 1. Springer series in operations research. Springer, New York, Berlin, Heidelberg, 2003.10.1007/b97543Search in Google Scholar
21. G. Qu and N. Li. Accelerated distributed nesterov gradient descent for smooth and strongly convex functions. In 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 209–216, Sep. 2016.10.1109/ALLERTON.2016.7852231Search in Google Scholar
22. S.S. Ram, V.V. Veeravalli and A. Nedić. Distributed non-autonomous power control through distributed convex optimization. In INFOCOM, pages 3001–3005. IEEE, 2009.10.1109/INFCOM.2009.5062275Search in Google Scholar
23. F. Salehisadaghiani and L. Pavel. Distributed Nash equilibrium seeking: A gossip-based algorithm. Automatica, 72:209–216, 2016.10.1016/j.automatica.2016.06.004Search in Google Scholar
24. F. Salehisadaghiani, W. Shi and L. Pavel. Distributed nash equilibrium seeking under partial-decision information via the alternating direction method of multipliers. Automatica, 103:27–35, 2019.10.1016/j.automatica.2019.01.012Search in Google Scholar
25. K. Scaman, F. Bach, S. Bubeck, Y.T. Lee and L. Massoulié. Optimal algorithms for smooth and strongly convex distributed optimization in networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 3027–3036, International Convention Centre, Sydney, Australia, 06–11 Aug 2017. PMLR.Search in Google Scholar
26. G. Scutari, F. Facchinei, L. Lampariello and P Song. Distributed methods for constrained nonconvex multi-agent optimization-part I: theory. CoRR, abs/1410.4754, 2014.Search in Google Scholar
27. W. Shi, Q. Ling, G. Wu and W. Yin. EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization. SIAM Journal on Optimization, 25(2):944–966, 2015.10.1137/14096668XSearch in Google Scholar
28. K. Srivastava and A. Nedić. Distributed asynchronous constrained stochastic optimization. IEEE Journal of Selected Topics in Signal Processing, 5(4):772–790, Aug 2011.10.1109/JSTSP.2011.2118740Search in Google Scholar
29. M. Stanković, K. Johansson and D. Stipanović. Distributed Seeking of Nash Equilibria with Applications to Mobile Sensor Networks. IEEE Transactions on Automatic Control, 57(4):904–919, 2012.10.1109/TAC.2011.2174678Search in Google Scholar
30. B. Swenson, S. Kar and J. Xavier. Empirical centroid fictitious play: An approach for distributed learning in multi-agent games. IEEE Transactions on Signal Processing, 63(15):3888–3901, Aug 2015.10.1109/TSP.2015.2434327Search in Google Scholar
31. T. Tatarenko. Independent log-linear learning in potential games with continuous actions. IEEE Transactions on Control of Network Systems, PP(99):1, 2017.10.1109/TCNS.2017.2655723Search in Google Scholar
32. T. Tatarenko and M. Kamgarpour. Learning generalized nash equilibria in a class of convex games. IEEE Transactions on Automatic Control, pages 1, 2018.10.1109/TAC.2018.2841319Search in Google Scholar
33. T. Tatarenko and A. Nedić. Geometric convergence of distributed gradient play in games with unconstrained action sets. CoRR, arXiv: https://arxiv.org/abs/1907.07144, 2019.Search in Google Scholar
34. T. Tatarenko, W. Shi and A. Nedić. Accelerated gradient play algorithm for distributed nash equilibrium seeking. In 2018 IEEE Conference on Decision and Control (CDC), pages 3561–3566, Dec 2018.10.1109/CDC.2018.8619479Search in Google Scholar
35. T. Tatarenko, W. Shi and A. Nedić. Geometric convergence of gradient play algorithms for distributed nash equilibrium seeking. CoRR, arXiv:1809.07383, 2018.Search in Google Scholar
36. T. Tatarenko and B. Touri. Non-convex distributed optimization. IEEE Transactions on Automatic Control, 62(8):3744–3757, Aug 2017.10.1109/TAC.2017.2648041Search in Google Scholar
37. K. Tsianos, S. Lawlor and M. Rabbat. Push-Sum Distributed Dual Averaging for Convex Optimization. In Proceedings of the 51st IEEE Annual Conference on Decision and Control, pages 5453–5458, 2012.10.1109/CDC.2012.6426375Search in Google Scholar
38. G. Tychogiorgos, A. Gkelias and K.K. Leung. A non-convex distributed optimization framework and its application to wireless ad-hoc networks. Wireless Communications, IEEE Transactions on, 12(9):4286–4296, September 2013.10.1109/TW.2013.072313.120739Search in Google Scholar
39. M. Ye and G. Hu. Distributed nash equilibrium seeking by a consensus based approach. IEEE Transactions on Automatic Control, 62(9):4811–4818, Sep. 2017.10.1109/TAC.2017.2688452Search in Google Scholar
40. M. Zhu and S. Martínez. Distributed coverage games for energy-aware mobile sensor networks. SIAM J. Control and Optimization, 51(1):1–27, 2013.10.1137/100784163Search in Google Scholar
© 2020 Walter de Gruyter GmbH, Berlin/Boston
