Skip to main content
SpringerLink
Log in

Negotiation in exploration-based environment

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

When two parties need to split some reward between them, negotiation theory can predict what offers the parties will make and how the reward will be split. When a single party needs to evaluate several alternatives and choose the best among them, optimal-stopping-rule theories guide it as to how to perform the exploration, what to explore next and when to stop. We consider a model in which party A needs to choose one alternative, but has no information and no means of acquiring information on the value of each alternative. Party B, on the other hand, has no interest in what party A chooses, but can perform (costly) exploration to learn about the different alternatives. As both negotiation and exploration take time, the common deadline and discounting factor further tie the processes together. We study the combined model, providing a comprehensive game theoretic based analysis, enabling the extraction of the payments that need to be made between agents A and B, and the social welfare. Special emphasis is placed on studying the effect of interleaving negotiation and exploration, and when is this method preferred over separating the two. In addition to exploring the basic questions, we also consider the case in which one of the parties has some control over the parameters of the problem (e.g. the negotiation protocol), and show how it increases the utility of this party but decreases the overall welfare.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Much effort has been dedicated in recent years to developing automatic negotiators that can successfully negotiate with people, usually applying machine learning mechanisms [6, 7, 27, 37, 45, 46].

  2. While optimal stopping is usually discussed in the context of models such as the “secretary problem” [21], the latter does not involve search costs and the goal is to maximize the probability of finding the best candidate rather than minimizing cost, hence the great difference between the two.

  3. At some points in the paper we also consider the distribution of payoffs, but we do not assume this is a consideration for the players.

  4. The case of \(j=0\) is ill-defined as it enables an infinite negotiation.

  5. The choice of minimum or maximum is application-dependent. For example, if the opportunities are different production technologies, as in the R&D example, then the company will pick the one associated with the minimum cost. If the opportunities are interviewees (potential employees in the headhunting example), the value of each opportunity represents the company’s benefit from hiring her, and the firm will recruit the one associated with the maximum value among those interviewed.

  6. While the proper representation should use a single variable to represent the number of remaining periods, we prefer the use of \(T-t\) as it later coincides with the negotiation analysis.

  7. This equation is different from [84] in the sense that the cost is multiplied by \(\delta ^k\) because it is incurred after the exploration, whereas in [84] the exploration cost is incurred before the exploration takes place. This, however, does not qualitatively change the results reported in this paper.

  8. In this sense the reservation value is just a threshold, and if the value of this threshold is too small the agent halts the exploration.

  9. For the discrete case the calculation of the reservation value is similar, replacing the integral by a sum and the probability distribution function with discrete probability \(P_i\) in Eq. (1):

    $$\begin{aligned} \delta ^k c_i = \delta ^k \sum _{x=r_i}^{\infty }{(x-r_i)P_i(x)} - (1-\delta ^k)r_i. \end{aligned}$$
    (2)

  10. In some degenerate cases the optimal exploration itself is not unique and there is possible more than one expected-benefit-maximizing way to explore the opportunities (e.g., when two opportunities have the same reservation value). In this case we the agents can follow an arbitrary pre-defined sequencing rule for the opportunities associated with the same reservation value when constructing their offers.

  11. See Eq. (1) in [20], which defines the portion out of the \(\delta EV\) pie that is being divided between the two parties.

  12. Notice that in this case the payment is made at the time of the proposal, rather than at time \(t+j\).

  13. One specific case worth mentioning within this context is when \(\delta =1\). Here, the agent that is set to be the first to issue a proposal will prefer any odd negotiation horizon and the other agent will prefer any even horizon.

  14. Notice that while that EV is fixed for the legacy negotiation protocol, even if the exploration that takes place was affected by \(\delta \) this would not change the result, because the exploration’s expected benefit is also maximized for \(\delta =1\).

References

  1. Amgoud, L., Dimopoulos, Y., & Moraitis, P. (2007). A unified and general framework for argumentation-based negotiation. In: Proceedings of the 6th international joint conference on autonomous agents and multiagent systems (AAMAS-07), (pp. 158:1–158:8).

  2. Amgoud, L., & Vesic, S. (2012). A formal analysis of the role of argumentation in negotiation dialogues. Journal of Logic and Computation, 22(5), 957–978.

    Article  MathSciNet  MATH  Google Scholar 

  3. An, B., Gatti, N., & Lesser, V. (2013). Bilateral bargaining with one-sided uncertain reserve prices. Autonomous Agents and Multi-Agent Systems, 26(3), 420–455.

    Article  Google Scholar 

  4. Atakan, A. (2006). Assortative matching with explicit search costs. Econometrica, 74(3), 667–680.

    Article  MathSciNet  MATH  Google Scholar 

  5. Atlas, J., & Decker, K. (2010). Coordination for uncertain outcomes using distributed neighbor exchange. In Proceedings of the ninth international conference on autonomous agents and multiagent systems (AAMAS-10), (pp. 1047–1054).

  6. Azaria, A., Aumann, Y., & Kraus, S. (2012). Automated strategies for determining rewards for human work. In Twenty-Sixth AAAI Conference on Artificial Intelligence.

  7. Azaria, A., Aumann, Y., & Kraus, S. (2014). Automated agents for reward determination for human work in crowdsourcing applications. Autonomous Agents and Multi-Agent Systems, 28(6), 934–955.

    Article  Google Scholar 

  8. Baarslag, T., Fujita, K., Gerding, E. H., Hindriks, K. V., Ito, T., Jennings, N. R., et al. (2013). Evaluating practical negotiating agents: Results and analysis of the 2011 international competition. Artificial Intelligence, 198, 73–103.

    Article  Google Scholar 

  9. Baarslag, T., Hindriks, K., Jonker, C. M., Kraus, S., & Lin, R. (2012). The first automated negotiating agents competition (anac 2010). In T. Ito, M. Zhang, V. Robu, S. Fatima, & T. Matsuo (Eds.), New Trends in Agent-based Complex Automated Negotiations, volume 383 of Studies in Computational Intelligence (pp. 113–135). Berlin: Springer.

    Chapter  Google Scholar 

  10. Bac, M., & Raff, H. (1996). Issue-by-issue negotiations: The role of information and time preference. Games and Economic Behavior, 13(1), 125–134.

    Article  MathSciNet  MATH  Google Scholar 

  11. Barbulescu, L., Rubinstein, Z., Smith, S., & Zimmerman, T. (2010). Distributed coordination of mobile agent teams: the advantage of planning ahead. In: Proceedings of the ninth international conference on autonomous agents and multiagent systems (AAMAS-10) (pp. 1331–1338).

  12. Benhabib, J., & Bull, C. (1983). Job search: The choice of intensity. Journal of Political Economy, 91(5), 747–764.

    Article  Google Scholar 

  13. Buttner, R. (2006). A classification structure for automated negotiations. In: Proceedings of the IEEE/WIC/ACM international conference on Web Intelligence and Intelligent Agent Technology, WI-IATW ’06 (pp. 523–530).

  14. Chevaleyre, Y., Endriss, U., & Maudet, N. (2010). Simple negotiation schemes for agents with simple preferences: Sufficiency, necessity and maximality. Autonomous Agents and Multi-Agent Systems, 20(2), 234–259.

    Article  Google Scholar 

  15. Chhabra, M., Das, S., & Sarne, D. (2014). Expert-mediated sequential search. European Journal of Operational Research, 234(3), 861–873.

    Article  MathSciNet  MATH  Google Scholar 

  16. Dunne, P., Wooldridge, M., & Laurence, M. (2005). The complexity of contract negotiation. Artificial Intelligence, 164(1–2), 23–46.

    Article  MathSciNet  MATH  Google Scholar 

  17. Ephrati, E., & Rosenschein, J. (1995). A framework for the interleaving of execution and planning for dynamic tasks by multiple agents. From Reaction to Cognition (pp. 139–153). Berlin: Springer.

    Chapter  Google Scholar 

  18. Faratin, P., Sierra, C., & Jennings, N. (2002). Using similarity criteria to make issue trade-offs in automated negotiations. Artificial Intelligence, 142, 205–237.

    Article  MathSciNet  Google Scholar 

  19. Fatima, S., Wooldridge, M., & Jennings, N. (2004). An agenda-based framework for multi-issue negotiation. Artificial Intelligence, 152(1), 1–45.

    Article  MathSciNet  MATH  Google Scholar 

  20. Fatima, S., Wooldridge, M., & Jennings, N. (2006). Multi-issue negotiation with deadlines. Journal of Artificial Intelligence Research, 27(1), 381–417.

    MathSciNet  MATH  Google Scholar 

  21. Ferguson, T. S. (1989). Who solved the secretary problem? Statistical Science, 4(3), 282–289.

    Article  MathSciNet  MATH  Google Scholar 

  22. Fershtman, C. (1990). The importance of the agenda in bargaining. Games and Economic Behavior, 2(3), 224–238.

    Article  MathSciNet  MATH  Google Scholar 

  23. Gal, S., Landsberger, M., & Levykson, B. (1981). A compound strategy for search in the labor market. International Economic Review, 22(3), 597–608.

    Article  MATH  Google Scholar 

  24. Gatti, N. (2009). Extending the alternating-offers protocol in the presence of competition: models and theoretical analysis. Annals of Mathematics and Artificial Intelligence, 55, 189–236.

    Article  MathSciNet  MATH  Google Scholar 

  25. Grosz, B., & Kraus, S. (1996). Collaborative plans for complex group action. Artificial Intelligence, 86(2), 269–357.

    Article  MathSciNet  Google Scholar 

  26. Hadad, M., Kraus, S., & Hartman, I. B.-A. (2013). Cooperative planning with time constraints. Annals of Mathematics and Artificial Intelligence, forthcoming.

  27. Haim, G., Gal, Y., Gelfand, M., & Kraus, S. (2012). A cultural sensitive agent for human-computer negotiation. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems, AAMAS-12 (pp. 451–458).

  28. Harsanyi, J., & Selten, R. (1972). A generalized nash solution for two-person bargaining games with incomplete information. Management Science, 18(5), 80–106.

    Article  MathSciNet  MATH  Google Scholar 

  29. Hazon, N., Aumann, Y., Kraus, S., & Sarne, D. (2013). Physical search problems with probabilistic knowledge. Artificial Intelligence, 196, 26–52.

    Article  MathSciNet  MATH  Google Scholar 

  30. Jennings, N., Faratin, P., Lomuscio, A., Parsons, S., Sierra, C., & Wooldridge, M. (2001). Automated negotiation: Prospects, methods and challenges. International Journal of Group Decision and Negotiation, 10(2), 199–215.

    Article  Google Scholar 

  31. Jennings, N., Faratin, P., Lomuscio, A., Parsons, S., Wooldridge, M., & Sierra, C. (2001). Automated negotiation: Prospects, methods and challenges. Group Decision and Negotiation, 10(2), 199–215.

    Article  Google Scholar 

  32. Kephart, J., & Greenwald, A. (2002). Shopbot economics. Journal of Autonomous Agents and Multi-Agent Systems, 5(3), 255–287.

    Article  MathSciNet  MATH  Google Scholar 

  33. Kim, J., & Segev, A. (2003). A framework for dynamic ebusiness negotiation processes. In E-Commerce, 2003. CEC 2003. IEEE International Conference on, (pp. 84–91). IEEE.

  34. Krasteva, S., & Yildirim, H. (2012). On the role of confidentiality and deadlines in bilateral negotiations. Games and Economic Behavior, 75(2), 714–730.

    Article  MathSciNet  MATH  Google Scholar 

  35. Kraus, S. (1997). Negotiation and cooperation in multi-agent environments. Artificial Intelligence, 94, 79–97.

    Article  MATH  Google Scholar 

  36. Kraus, S. (2001). Strategic Negotiation in Multiagent Environments. Cambridge, USA: MIT Press.

    MATH  Google Scholar 

  37. Kraus, S., & Lehmann, D. (1995). Designing and building a negotiating automated agent. Computational Intelligence, 11(1), 132–171.

    Article  Google Scholar 

  38. Kraus, S., & Wilkenfeld, J. (1991). The function of time in cooperative negotiations. In: Proceedings of the ninth national conference on artificial intelligence, AAAI’91 (pp. 179–184).

  39. Kraus, S., & Wilkenfeld, J. (1991). Negotiations over time in a multi agent environment: Preliminary report. In: Proceedings of the twelfth international joint conference on artificial intelligence (IJCAI-91), (pp. 56–61).

  40. Kraus, S., Wilkenfeld, J., & Zlotkin, G. (1995). Multiagent negotiation under time constraints. Artificial Intelligence, 75(2), 297–345.

    Article  MathSciNet  MATH  Google Scholar 

  41. Larson, K., & Sandholm, T. (2001). Bargaining with limited computation: Deliberation equilibrium. Artificial Intelligence, 132(2), 183–217.

    Article  MathSciNet  MATH  Google Scholar 

  42. Larson, K., & Sandholm, T. (2002). An alternating offers bargaining model for computationally limited agents. In: Proceedings of the first international joint conference on autonomous agents and multiagent systems: Part 1, AAMAS ’02 (pp. 135–142), New York.

  43. Larson, K., & Sandholm, T. (2002). An alternating offers bargaining model for computationally limited agents. In: Proceedings of the first international joint conference on autonomous agents and multiagent systems (AAMAS-02) (pp. 135–142).

  44. Lin, R., & Chou, S. (2004). Mediating a bilateral multi-issue negotiation. Electronic Commerce Research and Applications, 3(2), 126–138.

    Article  MathSciNet  Google Scholar 

  45. Lin, R., Gal, Y., Kraus, S., & Mazliah, Y. (2014). Training with automated agents improves people’s behavior in negotiation and coordination tasks. Decision Support Systems, 60, 1–9.

    Article  Google Scholar 

  46. Lin, R., & Kraus, S. (2010). Can automated agents proficiently negotiate with humans? Communications of the ACM, 53(1), 78–88.

    Article  Google Scholar 

  47. Lin, R., Kraus, S., Wilkenfeld, J., & Barry, J. (2006). An automated agent for bilateral negotiation with bounssded rational agents with incomplete information. Frontiers in Artificial Intelligence and Applications, 141, 270–274.

    Google Scholar 

  48. Lippman, S., & McCall, J. (1976). The economics of job search: A survey. Economic Inquiry, 14(3), 155–189.

    Article  MathSciNet  Google Scholar 

  49. Lomuscio, A., Wooldridge, M., & Jennings, N. (2001). A classification scheme for negotiation in electronic commerce. In: Proceedings of agent mediated electronic commerce (pp. 19–33).

  50. Ma, A., & Manove, M. (1993). Bargaining with deadlines and imperfect player control. Econometrica, 61(6), 1313–39.

    Article  MATH  Google Scholar 

  51. McMillan, J., & Rothschild, M. (1994). Search. In: Handbook of Game Theory with Economic Applications (pp. 905–927).

  52. Miller, D., & Gat, E. (1991). proceedings of exploiting known topologies to navigate with low-computation sensing. In: Fibers’ 91, (pp. 425–435). International Society for Optics and Photonics.

  53. Morgan, P. (1983). Search and optimal sample sizes. The Review of Economic Studies, 50(4), 659.

    Article  MathSciNet  MATH  Google Scholar 

  54. Morgan, P., & Manning, R. (1985). Optimal search. Econometrica, 53(4), 923–944.

    Article  MathSciNet  MATH  Google Scholar 

  55. Musliner, D., Durfee, E., & Shin, K. (1995). World modeling for the dynamic construction of real-time control plans. Artificial Intelligence, 74(1), 83–127.

    Article  Google Scholar 

  56. Myerson, R. (1984). Two-person bargaining problems with incomplete information. Econometrica, 52(2), 461–487.

    Article  MathSciNet  MATH  Google Scholar 

  57. Nahum, Y., Sarne, D., Das, S., & Shehory, O. (2015). Two-sided search with experts. Autonomous Agents and Multi-Agent Systems, 29(3), 364–401.

    Article  Google Scholar 

  58. Nash, J. (1950). The bargaining problem. Econometrica, 18, 155–162.

    Article  MathSciNet  MATH  Google Scholar 

  59. Nisan, N., Roughgarden, T., Tardos, E., & Vazirani, V. V. (2007). Algorithmic game theory (Vol. 1). Cambridge, NY: Cambridge University Press.

    Book  MATH  Google Scholar 

  60. Osborne, M. J., & Rubinstein, A. (1994). A course in game theory. Cambridge, Mass: MIT press.

    MATH  Google Scholar 

  61. Paolucci, M., Shehory, O., & Sycara, K. (2000). Interleaving planning and execution in a multiagent team planning environment. Electronic Transaction Artificial Intelligence, 4, 23–43.

    Google Scholar 

  62. Raïffa, H. (1982). The art and science of negotiation. Cambridge, Mass: Belknap Press of Harvard University Press.

    Google Scholar 

  63. Rochlin, I., Aumann, Y., Sarne, D., & Golosman, L. (2014). Efficiency and fairness in team search with self-interested agents. In: International conference on Autonomous Agents and Multi-Agent Systems, AAMAS’14 (pp. 365–372).

  64. Rochlin, I., Sarne, D., & Mash, M. (2014). Joint search with self-interested agents and the failure of cooperation enhancers. Artificial Intelligence, 214, 45–65.

    Article  MathSciNet  MATH  Google Scholar 

  65. Rosenfeld, A., Zuckerman, I., Segal-Halevi, E., Drein, O., & Kraus, S. (2015). NegoChat-A: a chat-based negotiation agent with bounded rationality. Autonomous Agents and Multi-Agent Systems. doi:10.1007/s10458-015-9281-9.

  66. Rosenschein, J., & Zlotkin, G. (1994). Rules of encounter: Designing conventions for automated negotiation among computers. Cambridge, Mass: MIT Press.

    Google Scholar 

  67. Rothschild, M. (1974). Searching for the lowest price when the distribution of prices is unknown. Journal of Political Economy, 82(4), 689–711.

    Article  Google Scholar 

  68. Rubinstein, A. (1982). Perfect equilibrium in a bargaining model. Econometrica, 50(1), 97–109.

    Article  MathSciNet  MATH  Google Scholar 

  69. Rubinstein, A. (1985). A bargaining model with incomplete information about time preferences. Econometrica, 53(5), 1151–72.

    Article  MathSciNet  MATH  Google Scholar 

  70. Sandholm, T., & Vulkan, N. (1999). Bargaining with deadlines. In: Proceedings of the national conference on artificial intelligence (AAAI-99) (pp. 44–51).

  71. Sarne, D., & Kraus, S. (2008). Managing parallel inquiries in agents two-sided search. Artificial Intelligence, 172(4–5), 541–569.

    Article  MathSciNet  MATH  Google Scholar 

  72. Sarne, D., Manisterski, E., & Kraus, S. (2010). Multi-goal economic search using dynamic search structures. Autonomous Agents and Multi-Agent Systems, 21(2), 204–236.

    Article  Google Scholar 

  73. Selten, R. (1975). Reexamination of the perfectness concept for equilibrium points in extensive games. International Journal of Game Theory, 4, 25–55.

    Article  MathSciNet  MATH  Google Scholar 

  74. Sierra, C., Faratin, P., & Jennings, N. (1997). A service-oriented negotiation model between autonomous agents. In Proceedings of the 8th European workshop on modelling autonomous agents in a multi-agent world (MAAMAW) (pp. 17–35).

  75. Sierra, C., Jennings, N., Noriega, P., & Parsons, S. (1997). A framework for argumentation-based negotiation. In: Proceedings of the 4th international workshop on agent theories, architectures, and languages (ATAL) (pp. 177–192).

  76. Simmons, R. (1990). An architecture for coordinating planning, sensing, and action. In: Proceedings of DARPA workshop on innovative approaches to planning, scheduling and control (pp. 292–297).

  77. Smith, L. (2011). Frictional matching models. Annual Reviews in Economics, 3(1), 319–338.

    Article  Google Scholar 

  78. Stahl, I. (1972). Bargaining theory. Stockholm: Economics Research Institute, Stockholm School of Economics.

    Google Scholar 

  79. Stigler, G. (1961). The economics of information. Journal of Political Economy, 69(3), 213–225.

    Article  Google Scholar 

  80. Tan, T., & Werlang, S. (1988). A guide to knowledge and games. In: Proceedings of the 2nd conference on theoretical aspects of reasoning about knowledge, (pp. 163–177). Morgan Kaufmann Publishers Inc.

  81. Tong, X., Zhang, W., & Huang, H. (2012). Agent negotiation on resources with nonlinear utility functions. Computer Science and Information Systems, 9, 1697–1720.

    Article  Google Scholar 

  82. van den Berg, G. (1990). Nonstationarity in job search theory. Review of Economic Studies, 57, 255–277.

    Article  MathSciNet  MATH  Google Scholar 

  83. Wagner, T., Phelps, J., Guralnik, V., & VanRiper, R. (2004). An application view of Coordinators: Coordination managers for first responders. In: Proceedings of the nineteenth national conference on artificial intelligence (AAAI-04) (pp. 908–915).

  84. Weitzman, M. (1979). Optimal search for the best alternative. Econometrica, 47(3), 641–54.

    Article  MathSciNet  MATH  Google Scholar 

  85. Wooldridge, M. (2000). Reasoning about rational agents. Cambridge, Mass: MIT Press.

    MATH  Google Scholar 

  86. Zheng, R., Chakraborty, N., Dai, T., Sycara, K., & Lewis, M. (2013). Automated bilateral multiple-issue negotiation with no information about opponent. In Proceedings of the 46th Hawaii international conference on system sciences (HICSS) (pp. 520–527).

  87. Zlotkin, G., & Rosenschein, J. (1996). Mechanism design for automated negotiation, and its application to task oriented domains. Artificial Intelligence, 86(2), 195–244.

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This research was partially supported by ISF grants 1241/12 and 1083/13 and by BSF young.

Author information

Authors and Affiliations

  1. Department of Computer Science, Bar-Ilan University, Ramat Gan, Israel

    Israel Sofer, David Sarne & Avinatan Hassidim

Authors
  1. Israel Sofer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Sarne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Avinatan Hassidim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Sarne.

Additional information

Preliminary results of this work appeared in Proceedings of the Twenty-Sixth National Conference on Artificial Intelligence (AAAI-2012).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sofer, I., Sarne, D. & Hassidim, A. Negotiation in exploration-based environment. Auton Agent Multi-Agent Syst 30, 724–764 (2016). https://doi.org/10.1007/s10458-015-9303-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-015-9303-7

Keywords

Search

Navigation