skip to main content
10.1145/1082473.1082593acmconferencesArticle/Chapter ViewAbstractPublication PagesaamasConference Proceedingsconference-collections
Article

Reasoning about joint beliefs for execution-time communication decisions

Published: 25 July 2005 Publication History

Abstract

Just as POMDPs have been used to reason explicitly about uncertainty in single-agent systems, there has been recent interest in using multi-agent POMDPs to coordinate teams of agents in the presence of uncertainty. Although multi-agent POMDPs are known to be highly intractable, communication at every time step transforms a multi-agent POMDP into a more tractable single-agent POMDP. In this paper, we present an approach that generates "centralized" policies for multi-agent POMDPs at plan-time by assuming the presence of free communication, and at run-time, handles the problem of limited communication resources by reasoning about the use of communication as needed for effective execution. This approach trades off the need to do some computation at execution-time for the ability to generate policies more tractably at plan-time. In our algorithm, each agent, at run-time, models the distribution of possible joint beliefs. Joint actions are selected over this distribution, ensuring that agents remain synchronized. Communication is used to integrate local observations into the team belief only when those observations would improve team performance. We show, both through a detailed example and with experimental results, that our approach allows for effective decentralized execution while avoiding unnecessary instances of communication.

References

[1]
R. Becker, S. Zilberstein, V. Lesser, and C. V. Goldman. Transition-independent decentralized Markov decision processes. In International Joint Conference on Autonomous Agents and Multi-agent Systems, 2003.
[2]
D. S. Bernstein, S. Zilberstein, and N. Immerman. The complexity of decentralized control of Markov decision processes. In Uncertainty in Artificial Intelligence, 2000.
[3]
A. R. Cassandra. POMDP solver software. http://www.cassandra.org/pomdp/code/index.shtml.
[4]
R. Emery-Montemerlo, G. Gordon, J. Schneider, and S. Thrun. Approximate solutions for partially observable stochastic games with common payoffs. In International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2004.
[5]
D. Goldberg, V. Cicirello, M. B. Dias, R. Simmons, S. Smith, T. Smith, and A. Stentz. A distributed layered architecture for mobile robot coordination: Application to space exploration. In International NASA Workshop on Planning and Scheduling for Space, 2002.
[6]
C. V. Goldman and S. Zilberstein. Decentralized control of cooperative systems: Categorization and complexity analysis. Journal of AI Research, 2004.
[7]
E. A. Hansen, D. S. Bernstein, and S. Zilberstein. Dynamic programming for partially observable stochastic games. In National Conference on Artificial Intelligence, 2004.
[8]
L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable domains. Artificial Intelligence, 1998.
[9]
M. L. Littman, A. R. Cassandra, and L. P. Kaelbling. Learning policies for partially observable environments: Scaling up. In International Conference on Machine Learning, 1995.
[10]
R. Nair, D. Pynadath, M. Yokoo, M. Tambe, and S. Marsella. Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings. In International Joint Conference on Artificial Intelligence, 2003.
[11]
R. Nair, M. Roth, M. Yokoo, and M. Tambe. Communication for improving policy computation in distributed POMDPs. In International Joint Conference on Autonomous Agents and Multi-agent Systems, 2004.
[12]
R. Nair, M. Tambe, and S. Marsella. Team formation for reformation in multiagent domains like RoboCupRescue. In RoboCup-2002 International Symposium, 2003.
[13]
C. H. Papadimitriou and J. N. Tsitsiklis. The complexity of Markov decision processes. Mathematics of Operations Research, 1987.
[14]
L. Peshkin, K.-E. Kim, N. Meuleau, and L. P. Kaelbling. Learning to cooperate via policy search. In Uncertainty in Artificial Intelligence, 2000.
[15]
P. Poupart, L. E. Ortiz, and C. Boutilier. Value-directed sampling methods for monitoring POMDPs. In Uncertainty in Artificial Intelligence, 2001.
[16]
D. V. Pynadath and M. Tambe. The communicative multiagent team decision problem: Analyzing teamwork theories and models. Journal of AI Research, 2002.
[17]
M. Roth, D. Vail, and M. Veloso. A real-time world model for multi-robot teams with high-latency communication. In International Conference on Intelligent Robots and Systems, 2003.
[18]
S. Thrun. Monte Carlo POMDPs. In Neural Information Processing Systems, 2000.
[19]
P. Xuan and V. Lesser. Multi-agent policies: From centralized ones to decentralized ones. In International Joint Conference on Autonomous Agents and Multi-agent Systems, 2002.
[20]
P. Xuan, V. Lesser, and S. Zilberstein. Formal modeling of communication decisions in cooperative multiagent systems. In Workshop on Game-Theoretic and Decision-Theoretic Agents, 2000.

Cited By

View all
  • (2024)Approximate dec-POMDP solving using multi-agent A*Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/745(6743-6751)Online publication date: 3-Aug-2024
  • (2023)Learning scalable and efficient communication policies for multi-robot collision avoidanceAutonomous Robots10.1007/s10514-023-10127-347:8(1275-1297)Online publication date: 19-Aug-2023
  • (2022)Cyber-physical framework for UAV intelligent communicationsSCIENTIA SINICA Informationis10.1360/SSI-2021-0226Online publication date: 10-Nov-2022
  • Show More Cited By

Index Terms

  1. Reasoning about joint beliefs for execution-time communication decisions

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    AAMAS '05: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems
    July 2005
    1407 pages
    ISBN:1595930930
    DOI:10.1145/1082473
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 July 2005

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. POMDP
    2. communication
    3. distributed execution
    4. robot teams

    Qualifiers

    • Article

    Conference

    AAMAS05
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)10
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Approximate dec-POMDP solving using multi-agent A*Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/745(6743-6751)Online publication date: 3-Aug-2024
    • (2023)Learning scalable and efficient communication policies for multi-robot collision avoidanceAutonomous Robots10.1007/s10514-023-10127-347:8(1275-1297)Online publication date: 19-Aug-2023
    • (2022)Cyber-physical framework for UAV intelligent communicationsSCIENTIA SINICA Informationis10.1360/SSI-2021-0226Online publication date: 10-Nov-2022
    • (2021)Discrete Interactions in Decentralized Multiagent Coordination: A Probabilistic PerspectiveIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2020.304076913:4(1010-1022)Online publication date: Dec-2021
    • (2021)Toward System Theoretical Foundations for Human–Autonomy TeamsSystems Engineering and Artificial Intelligence10.1007/978-3-030-77283-3_5(77-92)Online publication date: 2-Nov-2021
    • (2020)Exploring Information Interactions in Decentralized Multiagent Coordination under Uncertainty2020 5th IEEE International Conference on Big Data Analytics (ICBDA)10.1109/ICBDA49040.2020.9101337(304-308)Online publication date: May-2020
    • (2020)Partially observable game-theoretic agent programming in GologInternational Journal of Approximate Reasoning10.1016/j.ijar.2019.12.017Online publication date: Jan-2020
    • (2020)Learning multi-agent communication with double attentional deep reinforcement learningAutonomous Agents and Multi-Agent Systems10.1007/s10458-020-09455-w34:1Online publication date: 25-Mar-2020
    • (2019)Coordination of Multiple Autonomous Agents Using Naturally Generated Languages in Task PlanningApplied Sciences10.3390/app91735719:17(3571)Online publication date: 1-Sep-2019
    • (2019)Personalized Change Awareness: Reducing Information Overload in Loosely-Coupled TeamworkArtificial Intelligence10.1016/j.artint.2019.05.005Online publication date: May-2019
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media