Summary
In extensive form noncooperative game theory, at each instant t, each agent i sets its state x i independently of the other agents, by sampling an associated distribution, q i(x i). The coupling between the agents arises in the joint evolution of those distributions. Distributed control problems can be cast the same way. In those problems the system designer sets aspects of the joint evolution of the distributions to try to optimize the goal for the overall system. Now information theory tells us what the separate q i of the agents are most likely to be if the system were to have a particular expected value of the objective function G(x 1, x 2, ...). So one can view the job of the system designer as speeding an iterative process. Each step of that process starts with a specified value of E(G), and the convergence of the q i to the most likely set of distributions consistent with that value. After this the target value for E q(G) is lowered, and then the process repeats. Previous work has elaborated many schemes for implementing this process when the underlying variables x i all have a finite number of possible values and G does not extend to multiple instants in time. That work also is based on a fixed mapping from agents to control devices, so that the the statistical independence of the agents’ moves means independence of the device states. This paper also extends that work to relax all of these restrictions. This extends the applicability of that work to include continuous spaces and Reinforcement Learning. This paper also elaborates how some of that earlier work can be viewed as a first-principles justification of evolution-based search algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Laughlin, D., Morari, M., Braatz, R.: Robust performance of cross-directional control systems for web processes. Automatica 29 (1993) 1394–1410
Wolfe, J., Chichka, D., Speyer, J.: Decentralized controllers for unmanned aerial vehicle formation flight. American Institute of Aeronautics and Astronautics 96 (1996) 3933
Mesbai, M., Hadaegh, F.: Graphs, matrix inequalities, and switching for the formation flying control of multiple spacecraft. In: Proceedings of the American Control Conference, San Diego, CA. (1999) 4148–4152
Fudenberg, D., Tirole, J.: Game Theory. MIT Press, Cambridge, MA (1991)
Basar, T., Olsder, G.: Dynamic Noncooperative Game Theory. Siam, Philadelphia, PA (1999) Second Edition.
Osborne, M., Rubenstein, A.: A Course in Game Theory. MIT Press, Cambridge, MA (1994)
Aumann, R., Hart, S.: Handbook of Game Theory with Economic Applications. North-Holland Press (1992)
Fudenberg, D., Levine, D.K.: The Theory of Learning in Games. MIT Press, Cambridge, MA (1998)
Wolpert, D.H.: Factoring a canonical ensemble. (2003) cond-mat/0307630.
Wolpert, D.H.: Information theory — the bridge connecting bounded rational game theory and statistical physics. In D. Braha, A.M., Bar-Yam, Y., eds.: Complex Engineering Systems. (2004)
Wolpert, D.H.: What information theory says about best response, binding contracts, and collective intelligence. In et al, A.N., ed.: Proceedings of WEHIA04, Springer Verlag (2004)
Wolpert, D.H., Bieniawski, S.: Theory of distributed control using product distributions. In: Proceedings of CDC04. (2004)
Macready, W., Wolpert, D.H.: Distributed optimization. In: Proceedings of ICCS 04. (2004)
Wolpert, D.H., Lee, C.F.: Adaptive metropolis hastings sampling using product distributions. Submitted to ICCS04 (2004)
Airiau, S., Wolpert, D.H.: Product distribution theory and semi-coordinate transformations. (2004) Submitted to AAMAS 04.
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA (1998)
Kaelbing, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4 (1996) 237–285
Crites, R.H., Barto, A.G.: Improving elevator performance using reinforcement learning. In Touretzky, D.S., Mozer, M.C., Hasselmo, M.E., eds.: Advances in Neural Information Processing Systems-8, MIT Press (1996) 1017–1023
Hu, J., Wellman, M.P.: Multiagent reinforcement learning: Theoretical framework and an algorithm. In: Proceedings of the Fifteenth International Conference on Machine Learning. (1998) 242–250
Antoine, N., Bieniawski, S., Kroo, I., Wolpert, D.H.: Fleet assignment using collective intelligence. In: Proceedings of 42nd Aerospace Sciences Meeting. (2004) AIAA-2004-0622.
Lee, C.F., Wolpert, D.H.: Product distribution theory for control of multi-agent systems. In: Proceedings of AAMAS 04. (2004)
Bieniawski, S., Wolpert, D.H.: Adaptive, distributed control of constrained multi-agent systems. In: Proceedings of AAMAS04. (2004)
Bieniawski, S., Wolpert, D.H., Kroo, I.: Discrete, continuous, and constrained optimization using collectives. In: Proceedings of 10th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, New York. (2004) in press.
Macready, W., Bieniawski, S., Wolpert, D.: Adaptive multi-agent systems for constrained optimization. (2004)
Bieniawski, S., Wolpert, D.H.: Using product distributions for distributed optimization. In: Proceedings of ICCS 04. (2004)
Cover, T., Thomas, J.: Elements of Information Theory. Wiley-Interscience, New York (1991)
Mackay, D.: Information theory, inference, and learning algorithms. Cambridge University Press (2003)
Jaynes, E.T., Bretthorst, G.L.: Probability Theory: The Logic of Science. Cambridge University Press (2003)
Kirkpatrick, S., Gelatt, C.D.J., Vecchi, M.P.: Optimization by simulated annealing. Science 220 (1983) 671–680
Diekmann, R., Luling, R., Simon, J.: Problem independent distributed simulated annealing and its applications. In: Applied Simulated Annealing. Springer (1993) 17–44
Catoni, O.: Solving scheduling problems by simulated annealing. SIAM Journal on Control and Optimization 36 (1998) 1539–1575
Vidal, R.V.V., ed.: Applied Simulated Annealing (Lecture Notes in Economics and Mathematical Systems). Springer (1993)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wolpert, D.H., Bieniawski, S. (2005). Distributed Adaptive Control: Beyond Single-Instant, Discrete Control Variables. In: Monitoring, Security, and Rescue Techniques in Multiagent Systems. Advances in Soft Computing, vol 28. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-32370-8_3
Download citation
DOI: https://doi.org/10.1007/3-540-32370-8_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23245-2
Online ISBN: 978-3-540-32370-9
eBook Packages: EngineeringEngineering (R0)