Abstract
This work provides methodological approaches to solve convex optimization problems arising in multi-agent systems which can be reformulated in terms of a so called N-cluster game. We consider different settings of information available to each agent in the system. First, we present a centralized algorithm, which requires a central coordinator having full access to information about agents’ actions and gradients of their cost functions, to demonstrate how the standard gradient descent method can be applied to achieve an optimal output in N-cluster games. After that we relax the full information setting and assume that only partial information is available to each agent. Focus lies on the following two cases. In the first case, the agents have access to their gradient functions and are allowed to exchange information with their local neighbors over a communication graph that connects the whole system. In the second case, the agents do not know the functional form of their objectives/gradients and can only access the current values of their objective functions at some query point. Moreover, the agents are allowed to communicate only with their local neighbors within the cluster to which they belong. For both settings we present the convergent optimization procedures and analyse their efficiency in simulations.
Zusammenfassung
Diese Arbeit stellt methodische Herangehensweisen zur Lösung von konvexen Optimierungsproblemen in Multi-Agenten-Systemen, formuliert als sogenannte Multi-Cluster Spiele, vor. In diesem Zusammenhang beschäftigen wir uns mit unterschiedlichen Aufteilungen von Informationen auf die Agenten. Zunächst stellen wir einen zentralen Algorithmus vor, der einen zentralen Koordinator mit uneingeschränktem Zugang zu den Aktionen der Agenten und den Gradienten ihrer Kostenfunktionen benötigt. Mit diesem Algorithmus soll demonstriert werden, wie die bekannte Methode des Gradientenabstiegs angewendet werden kann, um ein optimales Ergebnis bezüglich des N-Cluster Spiels zu erzeugen. Anschließend relaxieren wir die Annahme von uneingeschränkter Information und nehmen an, dass jedem Agenten nur ein Teil der Gesamtinformationen zur Verfügung steht. Hierbei liegt der Fokus auf den folgenden zwei Fällen. Im ersten Fall haben die Agenten Zugang zu den Gradienten ihrer eigenen Funktionen und Informationen können über einen das gesamte System vernetzenden Kommunikationsgraphen mit den direkten Nachbarn ausgetauscht werden. Im zweiten Fall kennen die Agenten die funktionale Form ihrer eigenen Zielfunktionen/Gradienten nicht und können den aktuellen Wert ihrer Zielfunktion nur an bestimmten Punkten abfragen. Zusätzlich ist es den Agenten nur erlaubt, Informationen mit den Agenten des eigenen Clusters auszutauschen. Für beide Fälle stellen wir konvergierende Optimierungsprozesse vor und analysieren deren Effizienz in Simulationen.
Dedicated to the 60th birthday of Prof. Dr.-Ing. Jürgen Adamy.
Funding source: Deutsche Forschungsgemeinschaft
Award Identifier / Grant number: 1984
Funding statement: This work was funded by the German research association (Deutsche Forschungsgemeinschaft / DFG) – priority programme 1984.
About the authors
Tatiana Tatarenko received the Dipl.-Math. degree in mathematics with focus on Statistics and Stochastic Processes from Lomonosov Moscow State University, Moscow, Russia, in 2011, and the Ph. D. degree from the Technical University of Darmstadt, Germany, in 2017. She is currently a leader of the research group “Distributed Optimisation and Game Theory” at the Control Methods and Robotics Lab under the direction of Prof. Dr.-Ing. Jürgen Adamy, Department of Electrical Engineering and Information Technology, TU Darmstadt. Her main fields of research are applied mathematics, distributed optimisation and game theory.
Jan Zimmermann received the degree Master of Science in Electrical Engineering and Information Science from the Technical University of Darmstadt, Germany in 2018. The same year he joined the Control Methods and Robotics Lab led by Prof. Dr. Jürgen Adamy as a Ph. D. student and started working in the research team of Dr. Tatiana Tatarenko. His current research interests include multi-agent-systems, distributed optimization, game theory and application of these fields in the context of smart grids.
References
1. Bach, F. R. and V. Perchet. 2016. Highly-smooth zero-th order online optimization. In: Conference on Learning Theory (COLT), pp. 257–283.Search in Google Scholar
2. Belgioioso, G., W. Ananduta, S. Grammatico and C. Ocampo-Martinez. 2020. Energy management and peer-to-peer trading in future smart grids: A distributed game-theoretic approach. In: 2020 European Control Conference (ECC), pp. 1324–1329.10.23919/ECC51009.2020.9143658Search in Google Scholar
3. Berahas, A. S., L. Cao, K. Choromanski and K. Scheinberg. 2021. A theoretical and empirical comparison of gradient approximations in derivative-free optimization. Foundations of Computational Mathematics. https://doi.org/10.1007/s10208-021-09513-z.10.1007/s10208-021-09513-zSearch in Google Scholar
4. Bravo, M., D. Leslie and P. Mertikopoulos. 2018. Bandit learning in concave n-person games. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18. Red Hook, NY, USA, Curran Associates Inc., pp. 5666–5676.Search in Google Scholar
5. Chung, K. L. Sep 1954. On a stochastic approximation method. Ann. Math. Statist. 25(3): 463–483.10.1142/9789812567130_0011Search in Google Scholar
6. Flaxman, A. D., A. T. Kalai and H. B. McMahan. 2005. Online convex optimization in the bandit setting: Gradient descent without a gradient. In: Proceedings of the Sixteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA’05. USA, Society for Industrial and Applied Mathematics, pp. 385–394.Search in Google Scholar
7. Gharesifard, B. and J. Cortés. 2013. Distributed convergence to nash equilibria in two-network zero-sum games. Automatica 49(6): 1683–1692.10.21236/ADA579430Search in Google Scholar
8. Hu, G. and Y. Pang. 2020. Nash equilibrium seeking in n-coalition games via a gradient-free method. arXiv preprint: 2008.12909.Search in Google Scholar
9. Jarrah, M., M. Jaradat, Y. Jararweh, M. Al-Ayyoub and A. Bousselham. 2015. A hierarchical optimization model for energy data flow in smart grid power systems. Information Systems 53: 190–200.10.1016/j.is.2014.12.003Search in Google Scholar
10. Kamgarpour, C. and G. Tatarenko. 2020. Bandit online learning of nash equilibria in monotone games. arXiv preprint: 2009.04258.Search in Google Scholar
11. Meng, M. and X. Li. 2020. On the linear convergence of distributed nash equilibrium seeking for multi-cluster games under partial-decision information. arXiv preprint: 2005.06923.Search in Google Scholar
12. Niyato, D., A. V. Vasilakos and Z. Kun. 2011. Resource and revenue sharing with coalition formation of cloud providers: Game theoretic approach. In: 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 215–224.10.1109/CCGrid.2011.30Search in Google Scholar
13. Pang, J.-S. and F. Facchinei. 2003. Finite-dimensional variational inequalities and complementarity problems: vol. 1. Springer series in operations research. Springer, New York, Berlin, Heidelberg.10.1007/b97543Search in Google Scholar
14. Scutari, G., D. Palomar, F. Facchinei and J. S. Pang. 2010. Convex optimization, game theory, and variational inequality theory. IEEE Signal Processing Magazine 27(3): 35–49.10.1109/MSP.2010.936021Search in Google Scholar
15. Shi, W., Q. Ling, G. Wu and W. Yin. 2015. EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization. SIAM Journal on Optimization 25(2): 944–966.10.1137/14096668XSearch in Google Scholar
16. Sun, C. and G. Hu. 2021. Distributed generalized nash equilibrium seeking of n-coalition games with full and distributive constraints. arXiv preprint: 2109.12515.10.1109/CDC45484.2021.9683359Search in Google Scholar
17. Tang, Y. and N. Li. 2019. Distributed zero-order algorithms for nonconvex multi-agent optimization. In: 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 781–786.10.1109/ALLERTON.2019.8919690Search in Google Scholar
18. Tatarenko, T. and M. Kamgarpour. 2019. Learning nash equilibria in monotone games. In: 2019 IEEE 58th Conference on Decision and Control (CDC), pp. 3104–3109.10.1109/CDC40024.2019.9029659Search in Google Scholar
19. Tatarenko, T., J. Zimmermann and J. Adamy. 2021. Gradient play in n-cluster games with zero-order information. arXiv preprint: 2107.12648.10.1109/CDC45484.2021.9683636Search in Google Scholar
20. Tellidou, A. C. and A. G. Bakirtzis. Nov 2007. Agent-based analysis of capacity withholding and tacit collusion in electricity markets. IEEE Transactions on Power Systems 22(4): 1735–1742.10.1109/TPWRS.2007.907533Search in Google Scholar
21. Ye, M., G. Hu and F. L. Lewis. 2018. Nash equilibrium seeking for N-coalition noncooperative games. Automatica 95: 266–272.10.1016/j.automatica.2018.05.020Search in Google Scholar
22. Ye, M., G. Hu and S. Xu. 2020. An extremum seeking-based approach for Nash equilibrium seeking in N-cluster noncooperative games. Automatica 114: 108815.10.1016/j.automatica.2020.108815Search in Google Scholar
23. Zeng, X., J. Chen, S. Liang and Y. Hong. 2019. Generalized Nash equilibrium seeking strategy for distributed nonsmooth multi-cluster game. Automatica 103: 20–26.10.1016/j.automatica.2019.01.025Search in Google Scholar
24. Zhao, C, J He, P Cheng and J Chen. 2017. Consensus-Based Energy Management in Smart Grid With Transmission Losses and Directed Communication. IEEE Transactions on Smart Grid 8(5): 2049–2061.10.1109/TSG.2015.2513772Search in Google Scholar
25. Zhou, J., Y. Lv, G. Wen, J. Lv and D. Zheng. 2021. Distributed nash equilibrium seeking in consistency-constrained multi-coalition games. arXiv preprint: 2106.10513.10.1109/TCYB.2022.3155687Search in Google Scholar PubMed
26. Zimmermann, J., T. Tatarenko, V. Willert and J. Adamy. 2021. Solving leaderless multi-cluster games over directed graphs. European Journal of Control 62: 14–21.10.1016/j.ejcon.2021.06.007Search in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston
