Abstract
Distributed Constraint Optimization Problems (DCOPs) are NP-hard and therefore the number of studies that consider incomplete algorithms for solving them is growing. Specifically, the Max-sum algorithm has drawn attention in recent years and has been applied to a number of realistic applications. Unfortunately, in many cases Max-sum does not produce high-quality solutions. More specifically, Max-sum does not converge and explores solutions of low quality when run on problems whose constraint graph representation contains multiple cycles of different sizes. In this paper we advance the state-of-the-art in incomplete algorithms for DCOPs by: (1) proposing a version of the Max-sum algorithm that operates on an alternating directed acyclic graph (Max-sum_AD), which guarantees convergence in linear time; (2) solving a major weakness of Max-sum and Max-sum_AD that causes inconsistent costs/utilities to be propagated and affect the assignment selection, by introducing value propagation to Max-sum_AD (Max-sum_ADVP); and (3) proposing exploration heuristic methods that evidently improve the algorithms performance further. We prove that Max-sum_ADVP converges to monotonically improving states after each change of direction, and that it is guaranteed to converge in pseudo-polynomial time to a stable solution that does not change with further changes of direction. Our empirical study reveals a large improvement in the quality of the solutions produced by Max-sum_ADVP on various benchmarks, compared to the solutions produced by the standard Max-sum algorithm, Bounded Max-sum and Max-sum_AD with no value propagation. It is found to be the best guaranteed convergence inference algorithm for DCOPs. The exploration methods we propose for Max-sum_ADVP improve its performance further. However, anytime results demonstrate that their exploration level is not as efficient as a version of Max-sum, which uses Damping.
Similar content being viewed by others
Notes
Following [7] we use the terms “variable-node” and “function-node” to refer to nodes in the factor graph corresponding to variables and constraints, respectively.
We demonstrate the phenomenon for Max-sum_AD since it is easier to follow. In standard Max-sum, such inconsistent information concerning the conflicting assignment of some node is propagated in all directions and fed back to the node itself through cycles.
We note that in standard Max-sum, the use of VP does not guarantee monotonicity since neighboring agents can replace assignments concurrently (as in DSA).
We are aware that in the literature there exist different versions of simulated annealing. We have implemented a variety of them and present the most successful.
References
Aji, S. M., & McEliece, R. J. (2000). The generalized distributive law. IEEE Transactions on Information Theory, 46(2), 325–343.
Arshad, M., & Silaghi, M. C. (2004). Distributed simulated annealing. Distributed constraint problem solving and reasoning in multi-agent systems, frontiers in artificial intelligence and applications series, 112 November 2004.
Bejar, R., Domshlak, C., Fernandez, C., Gomes, K., Krishnamachari, B., Selman, B., et al. (2005). Sensor networks and distributed CSP: Communication, computation and complexity. Artificial Intelligence, 161(1–2), 117–148.
Brito, I., & Meseguer, P. (2010). Improving dpop with function filtering. In AAMAS (pp. 141–148).
Brito, I., Meisels, A., Meseguer, P., & Zivan, R. (2009). Distributed constraint satisfaction with partially known constraints. Constraints, 14(2), 199–234.
Dechter, R. (1999). Bucket elimination: A unifying framework for reasoning. Artificial Intelligence, 113(1–2), 41–85.
Farinelli, A., Rogers, A., Petcu, A., & Jennings, N. R. (2008). Decentralized coordination of low-power embedded devices using the max-sum algorithm. In AAMAS (pp. 639–646).
Gershman, A., Grubshtein, A., Rokach, L., Meisels, A., & Zivan, R. (2008). Scheduling meetings by agents. In DCR workshop at AAMAS 2008, Estoril, Portugal, May.
Gershman, A., Meisels, A., & Zivan, R. (2009). Asynchronous forward bounding. Journal of Artificial Intelligence Research, 34, 25–46.
Globerson, A., & Jaakkola, T. (2007). Fixing max-product: Convergent message passing algorithms for map lp-relaxations. In NIPS.
Hatano, D., & Hirayama, K. (2013). Deqed: An efficient divide-and-coordinate algorithm for dcop. In IJCAI.
Hazan, T., & Shashua, A. (2010). Norm-product belief propagation: Primal-dual message-passing for approximate inference. IEEE Transactions on Information Theory, 56(12), 6294–6316.
Heras, F., & Larrosa, J. (2006). Intelligent variable orderings and re-orderings in dac-based solvers for WCSP. Journal of Heuristics, 12(4–5), 287–306.
Hirayama, K., & Yokoo, M. (2000). An approach to over-constrained distributed constraint satisfaction problems: Distributed hierarchical constraint satisfaction. In Proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 135–142).
Khot, S. (2002). On the power of unique 2-prover 1-round games. In Proceedings of the thirty-fourth annual ACM symposium on theory of computing (pp. 767–775).
Kiekintveld, C., Yin, Z., Kumar, A., & Tambe, M. (2010). Asynchronous algorithms for approximate distributed constraint optimization with quality bounds. In AAMAS (pp. 133–140).
Kschischang, F. R., Frey, B. J., & Loeliger, H. A. (2001). Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47(2), 181–208.
Larrosa, J., & Schiex, T. (2004). Solving weighted csp by maintaining arc consistency. Artificial Intelligence, 159, 1–26.
Lazic, N., Frey, B., & Aarabi, P. (2010). Solving the uncapacitated facility location problem using message passing algorithms. In International conference on artificial intelligence and statistics (pp. 429–436).
Maheswaran, R. T., Pearce, J. P., & Tambe, M. (2004). Distributed algorithms for dcop: A graphical-game-based approach. In PDCS) (pp. 432–439), September 2004.
Maheswaran, R. T., Tambe, M., Bowring, E., Pearce, J. P., & Varakantham, P. (2004). Taking DCOP to the real world: Efficient complete solutions for distributed multi-event scheduling. In 3rd International joint conference on autonomous agents and multiagent systems (AAMAS 2004), 19–23 August 2004, New York (pp. 310–317).
Modi, P. J., Shen, W., Tambe, M., & Yokoo, M. (2005). Adopt: asynchronous distributed constraints optimizationwith quality guarantees. Artificial Intelligence, 161(1–2), 149–180.
Netzer, A., Grubshtein, A., & Meisels, A. (2012). Concurrent forward bounding for distributed constraint optimization problems. Artificial Intelligence, 193, 186–216.
Okimoto, T., Joe, Y., Iwasaki, A., Yokoo, M., & Faltings, B. (2011). Pseudo-tree-based incomplete algorithm for distributed constraint optimization with quality bounds. In J. Lee, (Ed.), CP 2011, LNCS 6876 (pp. 660–674).
Pearce, J. P., & Tambe, M. (2007). Quality guarantees on k-optimal solutions for distributed constraint optimization problems. In IJCAI (pp. 1446–1451), Hyderabad, India, January 2007.
Petcu, A., & Faltings, B. (2005). A scalable method for multiagent constraint optimization. In IJCAI (pp. 266–271).
Petcu, A., & Faltings, B. (2005). Approximations in distributed optimization. In P. van Beek (Ed.), CP 2005, LNCS 3709 (pp. 802–806).
Ramchurn, S. D., Farinelli, A., Macarthur, K. S., & Jennings, N. R. (2010). Decentralized coordination in robocup rescue. The Computer Journal, 53(9), 1447–1461.
Reeves, C. R. (Ed.). (1993). Modern heuristic techniques for combinatorial problems. New York, NY: Wiley.
Rogers, A., Farinelli, A., Stranders, R., & Jennings, N. R. (2011). Bounded approximate decentralized coordination via the max-sum algorithm. Artificial Intelligence, 175(2), 730–759.
Rollon, E., & Larrosa, J. (2012). Improved bounded max-sum for distributed constraint optimization. In CP (pp. 624–632).
Smith, M., & Mailler, R. (2010). Getting what you pay for: Is exploration in distributed hill climbing really worth it? In IAT (pp. 319–326).
Sontag, D., Meltzer, T., Globerson, A., Jaakkola, T., & Weiss, Y. (2008). Tightening lp relaxations for map using message passing. In UAI (pp. 503–510).
Stranders, R., Farinelli, A., Rogers, A., & Jennings, N. R. (2009). Decentralized coordination of continuously valued control parameters using the max-sum algorithm. In AAMAS (pp. 601–608).
Teacy, W. T. L., Farinelli, A., Grabham, N. J., Padhy, P., Rogers, A., & Jennings, N. R. (2008). Max-sum decentralized coordination for sensor systems. In AAMAS (pp. 1697–1698).
Vinyals, M., Pujol, M., Rodríguez-Aguilar, J. A., & Cerquides, J. (2010). Divide-and-coordinate: Dcops by agreement. In AAMAS (pp. 149–156).
Vinyals, M., Rodríguez-Aguilar, J. A., & Cerquides, J. (2011). Constructing a unifying theory of dynamic programming dcop algorithms via the generalized distributive law. Autonomous Agents and Multi-Agent Systems, 22(3), 439–464.
Vinyals, M., Shieh, E., Cerquides, J., Rodriguez-Aguilar, J. A., Yin, Z., Tambe, M., & Bowring, E. (2011). Quality guarantees for region optimal dcop algorithms. In AAMAS (pp. 133–140). Tapei.
Yanover, C., Meltzer, T., & Weiss, Y. (2006). Linear programming relaxations and belief propagation: An empirical study. Journal of Machine Learning Research, 7, 1887–1907.
Yeoh, W., Felner, A., & Koenig, S. (2010). Bnb-adopt: An asynchronous branch-and-bound dcop algorithm. Artificial Intelligence Research (JAIR), 38, 85–133.
Zhang, W., Xing, Z., Wang, G., & Wittenburg, L. (2005). Distributed stochastic search and distributed breakout: Properties, comparishon and applications to constraints optimization problems in sensor networks. Artificial Intelligence, 161(1–2), 55–88.
Zivan, R., Okamoto, S., & Peled, H. (2014). Explorative anytime local search for distributed constraint optimization. Artificial Intelligence, 212, 1–26.
Zivan, R., & Peled, H. (2012). Max/min-sum distributed constraint optimization through value propagation on an alternating DAG. In AAMAS (pp. 265–272).
Zivan, R., Yedidsion, H., Okamoto, S., Glinton, R., & Sycara, K. P. (2015). Distributed constraint optimization for teams of mobile sensing agents. Autonomous Agents and Multi-Agent Systems, 29(3), 495–536.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Additional information
This paper is an extension of our AAMAS paper [43]. Besides an extended description and examples, it includes a proof of the monotonic improvement of Max-sum_ADVP and its cross-phase convergence, proposes two new classes of exploration heuristics, one inspired by simulated annealing and the other interleaving converging and non-converging versions of the algorithm. Furthermore, we present an extended empirical study that reveals the advantages in using the proposed exploration heuristics.
Rights and permissions
About this article
Cite this article
Zivan, R., Parash, T., Cohen, L. et al. Balancing exploration and exploitation in incomplete Min/Max-sum inference for distributed constraint optimization. Auton Agent Multi-Agent Syst 31, 1165–1207 (2017). https://doi.org/10.1007/s10458-017-9360-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10458-017-9360-1