Model of Decision Making for Agents Based on Relationship Coefficient

Abstract

This paper is concerned with the problem of interactions among agents and how agents make decisions under this framework. Researchers contribute a lot to various interactions involved in agents and propose models of decision making process to portray how the agents behave. Based on relationship coefficient, a framework of interaction among agents is presented which is ranging from collaboration to antagonism. Furthermore, a model of decision making is put forward according to this framework. This model is illustrated and contrasted with an instance of game theory.

1 Introduction

The researches on autonomous agents and Multi-Agent System (MAS) have aroused intense interesting and are a comparatively new and multi-disciplinary subject [1]. The study of the basic theories and applications involves computer science, economics, philosophy, sociology and ecology. Because of the opening and dynamic, it provides a wider realm to probe into and a new perspective. There are various interactions within human society. Examples of common types of interactions include: cooperation, coordination and negotiation [1]. There are some other types of interaction which are not included in these examples, such as competition and antagonism. As we can consider the interactions among agents similar with those in human society, that means the study of interaction should include the range from complete cooperation to complete hostility. Many researchers pay much attention to the protocols, models, processes of interactions under different condition. They overlap in the research scope and isolate on the method.

Stone [2] divides the MAS interaction into two major categories, benevolent and competitive, and discussed it mainly in the perspective of co-evolution and learning. Zhang [3] measures off the MAS cooperation depending on the agents’ goals and the degree of cooperation, ranging from complete cooperation to complete selfishness. Start from the sociality of agents, Jennings [4] proposes the socially responsible agents which are able to benefit from interactions with other agents, but are nevertheless willing to provide some resources for the benefit of the overall system some of the time. The agents’ actions can be classified by the comparison between value of personal benefit and that of social benefit. There are four kinds of actions defined: social actions, individual actions, divided actions and futile actions. Further on this work, Kalenka [5] observes a spectrum of potential decision making functions which ranges from the purely selfish (F₀) to the purely altruistic (F_n).

However, in either case, the classification of interaction is indistinct, and the decision making is rigid and isolate. There is not an explicit measure to include all sorts of interactions and tidy up the relationship among them. For example, how to describe a MAS in which some agents are cooperative while others are competitive or even antagonistic? Therefore, it is important and necessary to bring forward a clearer way to generalize all kinds of the interaction among agents. To achieve this, we introduce a relation coefficient and construct a framework of interaction by which we can explicitly depict the relationship between agents and tease out the common threads that make up the MAS interaction tapestry. Moreover, we propose a decision-making model for agents within this framework.

We begin in the following section by introducing the relation coefficient and constructing a framework of interaction. In Sect. 3, a decision-making model is proposed and a detailed explanation is given. Section 4 illustrates the use of the model on an example of game theory. In the end, comes the summary.

2 Interaction Framework for MAS Based on Relationship Coefficient (RC-h)

From the point of view of philosophy, nothing is completely isolated. There are various relationships between objects in the world. An entity would make a plan considering the relationship with others more or less. For instance, people belong to different organizations or play different roles in daily life and have different relationship with others. They could be relatives, friends, colleagues, adversaries, enemies or strangers. These could be abstracted as cooperation, selfishness, competition, antagonism and so on. These relationships can be considered as the basis of behavior decision. “Attitude is everything”. In the same way, agents interact with each other according to the relationship between them.

Therefore, how to describe and tidy up these relationships is the primary issue of establishing a decision making model. He [6, 7] brings up the concepts of Generalized Correlativity and Generalized Correlation Coefficient. Generalized correlativity consists of consistent and exclusive relation while a coefficient called generalized correlation coefficient (h) is used to portray the ups and downs of the relations. Consistent relation includes affinity and repulsion while exclusive relation involves execution and viability.

We introduce these concepts into the interactions among agents and use h as the relationship coefficient (RC) to depict the various relationships between agents. For the purpose of simplification and convenience, the domain of the relationship coefficient is extended to [−1, 1].

Briefly speaking, the interaction of agents can be classified into two major divisions: cooperation and non-cooperation. Correspondingly, the different range of value of h can be used to describe it, i.e. h ∈ [0, 1] and h ∈ [−1, 0] separately. Within the cooperation (h ∈ [0, 1]), it can be subdivide into collaboration (h ∈ [0.5, 1]) and selfishness (h ∈ [0, 0.5]) according to the desire to cooperate or be selfish. In collaboration, agents pay more attention to cooperate with other agents to fulfill goals together while in selfish situation, agents prefer to maximize their own profits. Similarly, in non-cooperation (h ∈ [−1, 0]), there are competition (h ∈ [−0.5, 0]) and antagonism (h ∈ [−1, 0.5]) according to the degree of contradiction and protection. In the competition, agents try their best to protect themselves, while in antagonism agents are more bellicose and aggressive. Figure 1 shows the relation between coefficient h and the interaction among agents.

Fig. 1.

This shows a fig imaging the relationship between RC-h and MAS interaction

The continuous change of h from 1 to −1 indicates the interaction changes of agents from collaboration to antagonism, passing through selfishness and competition. There are several particular points which attract more attention during this course. ① h = 1, complete collaboration. It is like the instances in Distributed Problem Solution (DPS). In such situation, agents have no local goal, they work together to solve the problem. They must cooperate with each other to achieve the global goal. Their cooperation is set in designing process. ② h = 0, complete selfishness. These agents are self-interested or rational. They perform to maximize their own profits. Many articles can be found in research on game theory which is a typical example. The main issues of rational agents are conflicts detecting and resolving. ③ h = −1, complete antagonism. In such condition, agents aim at destroying the adversary. Zero-sum is one of these paradigms. Besides, h = 0.5 and h = −0.5 are two other particular statuses. h = 0.5 is the boundary between selfishness and collaboration, while h = −0.5 is between competition and antagonism.

It can be seen this framework includes all kind of possible kinds of interactions among agents. Given such an interaction framework, we can delve the interaction systematically and tidy up the concepts and relationships within it clearly which were isolated and incomplete before.

3 Decision Making Model for Agent Based on RC-h

In an open and dynamic environment, the behaviors of situated agents show more complexity because of the autonomy, heterogeneity and diversity. The agents have so different structures, different statuses and different targets that it is rigid if they make decision according to same and inflexible strategy. When anonymous agents behave, they prefer to choose flexible action to adapt the changing of environment, objects and aims.

In our interaction framework mentioned above, all kinds of interactions are considered and included, from complete cooperation to complete antagonism. Now we present a decision-making model for agents based on this framework.

In order to discuss the interaction and decision-making model more explicitly, we make some necessary definition for MAS and agents, and then comes the model of decision-making. Here we only consider the interaction between two agents for facilitation.

Definition 1:

A multi-agent system M is a finite set of agents. M = {A₁, A₂,…,A_n}. H = {H₁, H₂,…, H_n} is the set called relationship coefficient set, where H_i = {h_i1, h_i2,…h_{i(i − 1)}, h_{i(i + 1)},…h_in} is the set of RC-h of agent A_i, h_ij (i ≠ j) represents the RC-h of the agent A_i towards another agent A_j, and it is possible h_ij ≠ h_ji. The finite set called action set Act = <a₁, a₂,…a_m> denotes the collection of all the possible actions that agents can perform. For simplification, we assume that all the agents have the same action set. a^* ∈ Act represents the final action that agent choose to perform after decision making.

In our decision-making model, when agents choose which action to perform, they consider not only the effect of the action on their own, but also on the other participant agent. So we introduce a vector to describe both of the effects.

Definition 2:

Impact vector I(a_k) = (I₁(a_k), I₂(a_k)) represents the impacts of action a_k ∈ Act (k = 1, 2, …, m) on the agent who performs it (I₁(a_k)) and the one who undertakes it (I₂(a_k)). Additionally, I₁(a_k), I₂(a_k) are real number, and $1 \geqslant \text{I}_{1} \left( {a_{k} } \right), \, \text{I}_{2} \left( {a_{k} } \right) \geqslant - 1$, where the positive value means the action does good to the agents while negative value means the action does harm to the agents, and 0 means no difference.

As we described above, there are various relationships between agents. Such relationships definitely influence the agents to make decision. That is when they are making decision they pay different attention to themselves and other participants depending on the relationships between them. Here, weighting factors are used to depict the different emphases on the impacts of action.

Definition 3:

Weighting vector Ω(h_ij) = (ω₁(h_ij), ω₂(h_ij)) denotes the weighting factors added to I(a_k), where ω₁(h_ij), ω₂(h_ij) imply the concern degree of the agent who perform the action on itself and on other’s and defined as following. It is function of h_ij.

ω₁(h_ij), ω₂(h_ij) are depending on RC-h which denotes that the different relationship between agents determine the different behaviors. We have:

Definition 4:

ω₁(h_ij), ω₂(h_ij) which are in Ω(h_ij) are decided by

$$ \left\{ {\begin{array}{*{20}l} {\varpi_{1} (h_{ij} ) = 1 - \left| {h_{ij} } \right|} \hfill \\ {\varpi_{2} (h_{ij} ) = h_{ij} } \hfill \\ \end{array} } \right. $$

(1)

When $h \leqslant 0$ the negative value of ω₂ is to distinguish the non-cooperation from cooperation. The relationship between h and ω₁, ω₂ is shown in Fig. 2.

Fig. 2.

This shows a figure depicting the relationship between RC-h and the weight ω1, ω2

So, when agents make decision, they take the whole impacts into account.

Definition 5:

The total impacts of the action a_k are described by:

$$ \text{SI}\left( {h_{ij} ,a_{k} } \right) \, = {\varvec{\Omega}}\left( {\varvec{h}_{{\varvec{ij}}} } \right) \, {\mathbf{I}}\left( {\varvec{a}_{\varvec{k}} } \right) = \omega_{1} \left( {h_{ij} } \right) \, \text{I}_{1} \left( {a_{k} } \right) + \omega_{2} \left( {h_{ij} } \right) \, \text{I}_{2} \left( {a_{k} } \right) $$

(2)

Definition 6:

The action a^* ∈ Act which the agent finally chooses to perform after decision making is the one which maximize the total impact SI(h_ij, a_k), that is

$$ a^{*} = \mathop {\arg \hbox{max} }\limits_{{a_{k} \, \in \,Act}} SI(h_{ij} ,a_{k} ),k = 1,2, \ldots ,m $$

(3)

The analysis of this model is as follows considering the formulas (1), (2) and (3) with the change of h_ij, and concentrate on several particular values of h_ij. To simplify, let A₁ be the agent who makes decision and A₂ be the agent with which A₁ interacts. h represents h₁₂.

• 1. $0 \,\leqslant\, h \,\leqslant\, 1$, A₁ is cooperative. When h decreases from 1 to 0, A₁ changes its state from cooperation to selfishness. Meanwhile, ω₁(h) which is the weight to the influence on itself increases from 0 to 1 continuously while ω₂(h) which is that on the other decreases from 1 to 0. These two processes mean A₁ pays more and more attention to the impact of actions on itself (i.e. I₁(a_k)) and pays less and less attention to that on A₂ (i.e. I₂(a_k)).

• 2. h = 1, A₁ is completely cooperative. At this point, ω₁(h) = 0, ω₂(h) = 1, no matter how much I₁(a_k) is, ω₁(h)I₁(a_k) = 0. It implies A₁ only thinks about how to maximize the benefit of A₂ and pays no attention to its own benefit. It’s possible that A₁ selects an action which would damage itself.

• 3. h = 0.5, A₁ is cooperative and also self-interested. At this point, ω₁(h) = ω₂(h)= 0.5. A₁ pays the equal attention to the interest of A₂ and itself.

• 4. h = 0, A₁ is completely selfish. Similar to item 2, ω₁(h) = 1, ω₂(h) = 0 and ω₂(h) I₂(a_k) = 0, it means A₁ only thinks about how to maximize the benefit of its own and pay no attention to that of A₂.

When −1 ≤ h ≤ 0, A₁ is non-cooperative, we can have the similar analysis.

4 A Simulation of Decision Making Based on the RC-h

The prisoner’s dilemma game is a famous example in game theory. It is an abstraction of social situations where each agent is faced with two alternative actions cooperating and defecting. The detail description is as followed:

Two persons have been arrested as suspects for criminal, and are interrogated in separate rooms. They are told

• 1. If both of them admit to the crime, they get 2 years of imprisonment

• 2. If both do not admit to the crime, they can only get 1 years each because of lack of evidence

• 3. If one of them admits and the other does not, the defector is released while the other serves 3 years.

Now the payoff matrix of this dilemma is showed in Fig. 3, where “cooperate” means not to admit to the crime, and “defect” means to admit. The values in the payoff matrix are not the numbers of the years of imprisonment, but express how good the result would be for each agent. The number lies in left-up corner of each cell is the payoff of Agent-A, and the one in right-down corner is that of Agent-B.

Fig. 3.

Payoff matrix of prisoner’s dilemma

From Fig. 3, we can tell there are four strategies: s₁(C,C), s₂(C,D), s₃(D,C) and s₄(D,D), where C means Cooperation and D means Defection. The first parameter of each strategy represents the action of agent-A, the second one denotes that of agent-B.

In the analysis of game theory, s₄(D,D) is called the dominant strategy and also the Nash equilibrium. However, it is obviously not the best choice. For if each agent choose to cooperate, the final strategy would be s₁(C,C), both utility is 3 which is higher then 1 when they decide to defect. The reason of such dilemma is the conflict between personal rational and social rational. Agents always prefer to the strategy which can bring them the maximum profit so as to controdict with that of collectivity or society. In multi-agent system, the interactions are various in different scenarios. The rational asked in game theory has its limit. For instance, in the DSP, the agents are required to be altruistic and have no local target for themself but only work for the global target. Therefore, a more flexible and wider applicable decision making method is needed to solve such situation.

According to the decision making model based on RC-h mentioned above, an experimental simulation is designed. Figure 4 shows for agent-A and agent-B respectively, the total impacts (SI) of four strategies s₁(C,C), s₂(C,D), s₃(D,C) and s₄(D,D) calculated by formula (1). Because of the symmetry of payoff matrix, the curves of Agent-A are similar with those of Agent-B. Figure 5 shows the final chosen actions of Agent-A and Agent-B according to the decision making rule (formula (2), also refer to Fig. 4). This graph shows that when h > 0.4, both agents choose to cooperate. This value is near h = 0.5 which is the dividing lines of selfishness and collaboration. It is necessary to point out this turning point will drift with the difference of payoff matrix.

Fig. 4.

The total impacts (SI) of the strategies of Agent-A and Agent-B

Fig. 5.

The final decision of Agent-A and Agent-B according to RC-h

Figure 6 combines the results of Agent-A and Agent-B, and comes out the final decision which is effected by both sides. From Fig. 6 we can tell h = 0.4 is a boundary, when both agents’ RC-h are less than it, the final result comes to s₄(D,D) which coincide with the action of “rational” agents. What is different is the cooperative behavior (s₁(C,C)) occurs when the RC-h of two parties are greater than 0.4, i.e. both sides are nearly in the condition of collaboration. Else, s₂(C,D) or s₃(D,C) turns up if one agent is towards cooperative while the other is opposite.

Fig. 6.

The synthetic result of Agent-A and Agent-B

This decision making model also can be applied on other instances of game theory or other fields. We would not extend here.

From the analysis and simulation mentioned above, we can clearly tell the decision making model based on relationship coefficient-h is more flexible and reasonable. It allows us to comprehend the interactions in agents and its result from a higher view. Meanwhile, adjusting the RC-h according to design targets or learning the RC-h from interaction history makes the agents adaptive to the changing environment.

5 Summary

This paper presented a framework of interaction in MAS based on relationship coefficient-h. It provided a typology of the types of MAS interactions, ranging from complete cooperation to complete antagonism, and quantify the interaction by h. Within this framework, a decision-making model for agents is proposed which enable agents choose their action depending on the relationship to others. Using this model, in open and dynamic surroundings, agents can perform more adaptive and satisfy the design goals. At the end, the model is illustrated and contrasted with an instance of game theory.