Skip to main content
SpringerLink
Log in

Balancing exploration and exploitation in incomplete Min/Max-sum inference for distributed constraint optimization

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Distributed Constraint Optimization Problems (DCOPs) are NP-hard and therefore the number of studies that consider incomplete algorithms for solving them is growing. Specifically, the Max-sum algorithm has drawn attention in recent years and has been applied to a number of realistic applications. Unfortunately, in many cases Max-sum does not produce high-quality solutions. More specifically, Max-sum does not converge and explores solutions of low quality when run on problems whose constraint graph representation contains multiple cycles of different sizes. In this paper we advance the state-of-the-art in incomplete algorithms for DCOPs by: (1) proposing a version of the Max-sum algorithm that operates on an alternating directed acyclic graph (Max-sum_AD), which guarantees convergence in linear time; (2) solving a major weakness of Max-sum and Max-sum_AD that causes inconsistent costs/utilities to be propagated and affect the assignment selection, by introducing value propagation to Max-sum_AD (Max-sum_ADVP); and (3) proposing exploration heuristic methods that evidently improve the algorithms performance further. We prove that Max-sum_ADVP converges to monotonically improving states after each change of direction, and that it is guaranteed to converge in pseudo-polynomial time to a stable solution that does not change with further changes of direction. Our empirical study reveals a large improvement in the quality of the solutions produced by Max-sum_ADVP on various benchmarks, compared to the solutions produced by the standard Max-sum algorithm, Bounded Max-sum and Max-sum_AD with no value propagation. It is found to be the best guaranteed convergence inference algorithm for DCOPs. The exploration methods we propose for Max-sum_ADVP improve its performance further. However, anytime results demonstrate that their exploration level is not as efficient as a version of Max-sum, which uses Damping.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30

Similar content being viewed by others

Notes

  1. Following [7] we use the terms “variable-node” and “function-node” to refer to nodes in the factor graph corresponding to variables and constraints, respectively.

  2. In contrast to previous papers on Max-sum, we present it using pseudo-code. This is following standard DCOP literature, e.g., [22, 26, 41]. Nevertheless, only the presentation is different; the algorithm itself is identical to the algorithm presented in [7, 30].

  3. We demonstrate the phenomenon for Max-sum_AD since it is easier to follow. In standard Max-sum, such inconsistent information concerning the conflicting assignment of some node is propagated in all directions and fed back to the node itself through cycles.

  4. We note that in standard Max-sum, the use of VP does not guarantee monotonicity since neighboring agents can replace assignments concurrently (as in DSA).

  5. We are aware that in the literature there exist different versions of simulated annealing. We have implemented a variety of them and present the most successful.

References

  1. Aji, S. M., & McEliece, R. J. (2000). The generalized distributive law. IEEE Transactions on Information Theory, 46(2), 325–343.

    Article  MathSciNet  MATH  Google Scholar 

  2. Arshad, M., & Silaghi, M. C. (2004). Distributed simulated annealing. Distributed constraint problem solving and reasoning in multi-agent systems, frontiers in artificial intelligence and applications series, 112 November 2004.

  3. Bejar, R., Domshlak, C., Fernandez, C., Gomes, K., Krishnamachari, B., Selman, B., et al. (2005). Sensor networks and distributed CSP: Communication, computation and complexity. Artificial Intelligence, 161(1–2), 117–148.

    Article  MathSciNet  MATH  Google Scholar 

  4. Brito, I., & Meseguer, P. (2010). Improving dpop with function filtering. In AAMAS (pp. 141–148).

  5. Brito, I., Meisels, A., Meseguer, P., & Zivan, R. (2009). Distributed constraint satisfaction with partially known constraints. Constraints, 14(2), 199–234.

    Article  MathSciNet  MATH  Google Scholar 

  6. Dechter, R. (1999). Bucket elimination: A unifying framework for reasoning. Artificial Intelligence, 113(1–2), 41–85.

    Article  MathSciNet  MATH  Google Scholar 

  7. Farinelli, A., Rogers, A., Petcu, A., & Jennings, N. R. (2008). Decentralized coordination of low-power embedded devices using the max-sum algorithm. In AAMAS (pp. 639–646).

  8. Gershman, A., Grubshtein, A., Rokach, L., Meisels, A., & Zivan, R. (2008). Scheduling meetings by agents. In DCR workshop at AAMAS 2008, Estoril, Portugal, May.

  9. Gershman, A., Meisels, A., & Zivan, R. (2009). Asynchronous forward bounding. Journal of Artificial Intelligence Research, 34, 25–46.

    MathSciNet  MATH  Google Scholar 

  10. Globerson, A., & Jaakkola, T. (2007). Fixing max-product: Convergent message passing algorithms for map lp-relaxations. In NIPS.

  11. Hatano, D., & Hirayama, K. (2013). Deqed: An efficient divide-and-coordinate algorithm for dcop. In IJCAI.

  12. Hazan, T., & Shashua, A. (2010). Norm-product belief propagation: Primal-dual message-passing for approximate inference. IEEE Transactions on Information Theory, 56(12), 6294–6316.

    Article  MathSciNet  Google Scholar 

  13. Heras, F., & Larrosa, J. (2006). Intelligent variable orderings and re-orderings in dac-based solvers for WCSP. Journal of Heuristics, 12(4–5), 287–306.

    Article  MATH  Google Scholar 

  14. Hirayama, K., & Yokoo, M. (2000). An approach to over-constrained distributed constraint satisfaction problems: Distributed hierarchical constraint satisfaction. In Proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 135–142).

  15. Khot, S. (2002). On the power of unique 2-prover 1-round games. In Proceedings of the thirty-fourth annual ACM symposium on theory of computing (pp. 767–775).

  16. Kiekintveld, C., Yin, Z., Kumar, A., & Tambe, M. (2010). Asynchronous algorithms for approximate distributed constraint optimization with quality bounds. In AAMAS (pp. 133–140).

  17. Kschischang, F. R., Frey, B. J., & Loeliger, H. A. (2001). Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47(2), 181–208.

    Article  MathSciNet  MATH  Google Scholar 

  18. Larrosa, J., & Schiex, T. (2004). Solving weighted csp by maintaining arc consistency. Artificial Intelligence, 159, 1–26.

    Article  MathSciNet  MATH  Google Scholar 

  19. Lazic, N., Frey, B., & Aarabi, P. (2010). Solving the uncapacitated facility location problem using message passing algorithms. In International conference on artificial intelligence and statistics (pp. 429–436).

  20. Maheswaran, R. T., Pearce, J. P., & Tambe, M. (2004). Distributed algorithms for dcop: A graphical-game-based approach. In PDCS) (pp. 432–439), September 2004.

  21. Maheswaran, R. T., Tambe, M., Bowring, E., Pearce, J. P., & Varakantham, P. (2004). Taking DCOP to the real world: Efficient complete solutions for distributed multi-event scheduling. In 3rd International joint conference on autonomous agents and multiagent systems (AAMAS 2004), 19–23 August 2004, New York (pp. 310–317).

  22. Modi, P. J., Shen, W., Tambe, M., & Yokoo, M. (2005). Adopt: asynchronous distributed constraints optimizationwith quality guarantees. Artificial Intelligence, 161(1–2), 149–180.

    Article  MathSciNet  MATH  Google Scholar 

  23. Netzer, A., Grubshtein, A., & Meisels, A. (2012). Concurrent forward bounding for distributed constraint optimization problems. Artificial Intelligence, 193, 186–216.

    Article  MathSciNet  MATH  Google Scholar 

  24. Okimoto, T., Joe, Y., Iwasaki, A., Yokoo, M., & Faltings, B. (2011). Pseudo-tree-based incomplete algorithm for distributed constraint optimization with quality bounds. In J. Lee, (Ed.), CP 2011, LNCS 6876 (pp. 660–674).

  25. Pearce, J. P., & Tambe, M. (2007). Quality guarantees on k-optimal solutions for distributed constraint optimization problems. In IJCAI (pp. 1446–1451), Hyderabad, India, January 2007.

  26. Petcu, A., & Faltings, B. (2005). A scalable method for multiagent constraint optimization. In IJCAI (pp. 266–271).

  27. Petcu, A., & Faltings, B. (2005). Approximations in distributed optimization. In P. van Beek (Ed.), CP 2005, LNCS 3709 (pp. 802–806).

  28. Ramchurn, S. D., Farinelli, A., Macarthur, K. S., & Jennings, N. R. (2010). Decentralized coordination in robocup rescue. The Computer Journal, 53(9), 1447–1461.

    Article  Google Scholar 

  29. Reeves, C. R. (Ed.). (1993). Modern heuristic techniques for combinatorial problems. New York, NY: Wiley.

    MATH  Google Scholar 

  30. Rogers, A., Farinelli, A., Stranders, R., & Jennings, N. R. (2011). Bounded approximate decentralized coordination via the max-sum algorithm. Artificial Intelligence, 175(2), 730–759.

    Article  MathSciNet  MATH  Google Scholar 

  31. Rollon, E., & Larrosa, J. (2012). Improved bounded max-sum for distributed constraint optimization. In CP (pp. 624–632).

  32. Smith, M., & Mailler, R. (2010). Getting what you pay for: Is exploration in distributed hill climbing really worth it? In IAT (pp. 319–326).

  33. Sontag, D., Meltzer, T., Globerson, A., Jaakkola, T., & Weiss, Y. (2008). Tightening lp relaxations for map using message passing. In UAI (pp. 503–510).

  34. Stranders, R., Farinelli, A., Rogers, A., & Jennings, N. R. (2009). Decentralized coordination of continuously valued control parameters using the max-sum algorithm. In AAMAS (pp. 601–608).

  35. Teacy, W. T. L., Farinelli, A., Grabham, N. J., Padhy, P., Rogers, A., & Jennings, N. R. (2008). Max-sum decentralized coordination for sensor systems. In AAMAS (pp. 1697–1698).

  36. Vinyals, M., Pujol, M., Rodríguez-Aguilar, J. A., & Cerquides, J. (2010). Divide-and-coordinate: Dcops by agreement. In AAMAS (pp. 149–156).

  37. Vinyals, M., Rodríguez-Aguilar, J. A., & Cerquides, J. (2011). Constructing a unifying theory of dynamic programming dcop algorithms via the generalized distributive law. Autonomous Agents and Multi-Agent Systems, 22(3), 439–464.

    Article  Google Scholar 

  38. Vinyals, M., Shieh, E., Cerquides, J., Rodriguez-Aguilar, J. A., Yin, Z., Tambe, M., & Bowring, E. (2011). Quality guarantees for region optimal dcop algorithms. In AAMAS (pp. 133–140). Tapei.

  39. Yanover, C., Meltzer, T., & Weiss, Y. (2006). Linear programming relaxations and belief propagation: An empirical study. Journal of Machine Learning Research, 7, 1887–1907.

    MathSciNet  MATH  Google Scholar 

  40. Yeoh, W., Felner, A., & Koenig, S. (2010). Bnb-adopt: An asynchronous branch-and-bound dcop algorithm. Artificial Intelligence Research (JAIR), 38, 85–133.

    MATH  Google Scholar 

  41. Zhang, W., Xing, Z., Wang, G., & Wittenburg, L. (2005). Distributed stochastic search and distributed breakout: Properties, comparishon and applications to constraints optimization problems in sensor networks. Artificial Intelligence, 161(1–2), 55–88.

    Article  MathSciNet  MATH  Google Scholar 

  42. Zivan, R., Okamoto, S., & Peled, H. (2014). Explorative anytime local search for distributed constraint optimization. Artificial Intelligence, 212, 1–26.

  43. Zivan, R., & Peled, H. (2012). Max/min-sum distributed constraint optimization through value propagation on an alternating DAG. In AAMAS (pp. 265–272).

  44. Zivan, R., Yedidsion, H., Okamoto, S., Glinton, R., & Sycara, K. P. (2015). Distributed constraint optimization for teams of mobile sensing agents. Autonomous Agents and Multi-Agent Systems, 29(3), 495–536.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer-Sheva, Israel

    Roie Zivan, Tomer Parash, Liel Cohen, Hilla Peled & Steven Okamoto

Authors
  1. Roie Zivan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomer Parash
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liel Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hilla Peled
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steven Okamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roie Zivan.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

This paper is an extension of our AAMAS paper [43]. Besides an extended description and examples, it includes a proof of the monotonic improvement of Max-sum_ADVP and its cross-phase convergence, proposes two new classes of exploration heuristics, one inspired by simulated annealing and the other interleaving converging and non-converging versions of the algorithm. Furthermore, we present an extended empirical study that reveals the advantages in using the proposed exploration heuristics.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zivan, R., Parash, T., Cohen, L. et al. Balancing exploration and exploitation in incomplete Min/Max-sum inference for distributed constraint optimization. Auton Agent Multi-Agent Syst 31, 1165–1207 (2017). https://doi.org/10.1007/s10458-017-9360-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-017-9360-1

Keywords

Search

Navigation