Collective decision-making for dynamic environments with visual occlusions

Abstract

For decades, both empirical and theoretical models have been proposed to explain the patterns and mechanisms of collective decision-making (CDM). The most-studied CDM scenario is the best-of-n problem in a static environment. However, natural environments are typically dynamic. In dynamic environments, the visual occlusions produced by other members of a large-scale group are also common. Hence, some agents of a group are less informed than others, and their state uncertainties increase. This paper develops a new model referred to as the generalized Ising model with dynamic confidence (GIM-C) to reduce the state uncertainty induced by visual occlusions. The proposed model first estimates the expected rewards of possible actions with dynamic confidence weighting. It then gives the probability of choosing each action based on the generalized Ising model with an external field defined by the last stage’s results. Numerical simulations demonstrate that GIM-C shares the key feature of social cohesion with previous CDM models. Furthermore, in order to illustrate the efficiency of the proposed GIM-C, the collecting foraging task is considered, where a large-scale group of agents is required to obtain rewards with the presence of a dynamic predator and visual occlusions. The good performance of GIM-C in the collecting foraging task demonstrates that dynamic confidence weighting is efficient in reducing the state uncertainty introduced by visual occlusions. The proposed GIM-C also demonstrates the importance of enhancing the influence of informed agents in CDM problems in a dynamic environment with visual occlusions.

Appendix: The collective foraging task

In the collective foraging task, we consider agents with three possible behaviors defined by functions that map environmental conditions and the states of neighbors to the velocity of an agent at the current time step.

• Feeding, gathering rewards from feeding areas. With the center $\mathbf {l}_\mathrm{fd}$ of the nearest feeding area and the location $\mathbf {l}_i$ of agent i, the desired velocity $\mathbf {v}_i^t$ of a feeding agent i at time t is defined as:

  $$\begin{aligned} \mathbf {v}_i^t = \varTheta \left[ \frac{\mathbf {l}_\mathrm{fd}-{\mathbf {l}_i}}{\left| \mathbf {l}_\mathrm{fd}-\mathbf {l}_i \right| } + \mathbf {v}_r + \mathbf {v}_g \right] \cdot v_\mathrm{fd} , \end{aligned}$$

  (27)

  where $\varTheta \left[ \right]$ is the normalization operator, $\mathbf {v}_r$ is a random vector whose norm $|\mathbf {v}_r|=1$, $\mathbf {v}_g$ is a component maintaining the density of agents and $v_\mathrm{fd}$ denotes the speed of a feeding agent. With the maximum range of repulsion between agents denoted by $r_\mathrm{rep}^\mathrm{max}$ and the intensity of repulsion by $f_\mathrm{rep}$, $\mathbf {v}_{g}$ is given by the following:

  $$\begin{aligned} \mathbf {v}_g = {\left\{ \begin{array}{ll} \sum _{j\in N_i^t} f_\mathrm{rep} \cdot \left( r_\mathrm{rep}^\mathrm{max} - \left| \mathbf {l}_{j}-\mathbf {l}_i \right| \right) \cdot \varTheta \left[ \mathbf {l}_{i}-\mathbf {l}_j \right] &{} \text {if} \left| \mathbf {l}_{i}-\mathbf {l}_j \right| < r_\mathrm{rep}^\mathrm{max} \\ 0 &{} \text {otherwise} \end{array}\right. } \end{aligned}$$

  (28)

• Avoiding, escaping from the predator. With the location of predator denoted by $\mathbf {l}_\mathrm{pd}$, the desired velocity $v_i$ of an avoiding agent i is given by:

  $$\begin{aligned} \mathbf {v}_i^t = {\left\{ \begin{array}{ll} \varTheta \left[ \mathbf {l}_i - \mathbf {l}_\mathrm{pd} + \mathbf {v}_g \right] \cdot v_\mathrm{avd} &{} \text {if the agent observes the predator} \\ \varTheta \left[ \frac{ \sum _{j\in N_i^{t}} \mathbf {v}_j^{t-1}}{|N_i^{t}|} +\mathbf {v}_g \right] \cdot v_\mathrm{avd} &{} \text {otherwise} \end{array}\right. }, \end{aligned}$$

  (29)

  where $v_\mathrm{avd}$ denotes the speed of an avoiding agent.

• Foraging, approaching the nearest feeding area. The velocity of foraging agent i is given by:

  $$\begin{aligned} \mathbf {v}_i^t = \varTheta \left[ \mathbf {l}_\mathrm{fd} - \mathbf {l}_i +\mathbf {v}_g \right] \cdot v_\mathrm{fr} , \end{aligned}$$

  (30)

  where $v_\mathrm{fr}$ is the speed of a foraging agent.

The parameters used to calculate the desired velocity are outlined in Table 3.

1.1 Expected reward estimates

The estimation of the expected reward of a possible action is defined as a function of observations. At each time step t, agent i estimates expected rewards of all possible actions with the following functions:

• Feeding. In order to keep agents close to the center of the feeding area, the estimated expected reward $u_i^t(a_\mathrm{fd})$ of feeding is given as follows:

  $$\begin{aligned} u_i^t(a_\mathrm{fd}) = {\left\{ \begin{array}{ll} c_\mathrm{fd}\left( r_\mathrm{fd} - \left| \mathbf {l}_\mathrm{fd}-\mathbf {l}_i \right| \right) &{} \text {if} \left| \mathbf {l}_\mathrm{fd}-\mathbf {l}_i \right| < r_\mathrm{fd} \\ 0 &{} \text {otherwise} \end{array}\right. } , \end{aligned}$$

  (31)

  where $c_\mathrm{fd}$ is a constant and $r_\mathrm{fd}$ is the radius of a feeding area.

• Avoiding. Denoting the location of the predator by $\mathbf {l}_\mathrm{pd}$, the estimated expected reward $u_i^t(a_\mathrm{avd})$ of avoiding is given by the following:

  $$\begin{aligned} u_i^t(a_\mathrm{avd}) = {\left\{ \begin{array}{ll} c_\mathrm{avd}\left( u_\mathrm{avd}^\mathrm{max} - \left| \mathbf {l}_\mathrm{pd}-\mathbf {l}_i \right| \right) &{} \text {if the agent observes the predator} \\ 0 &{} \text {otherwise} \end{array}\right. }, \end{aligned}$$

  (32)

  where $u_\mathrm{avd}^\mathrm{max}$ is the max expected reward of avoiding, $c_\mathrm{avd}$ is a constant.

• Foraging. The estimated expected reward of foraging $u_i^t(a_\mathrm{fr})$ is given by the following:

  $$\begin{aligned} u_i^t(a_\mathrm{fr})={\left\{ \begin{array}{ll} 10 &{} \text {if} |\mathbf {l}_\mathrm{fd} - \mathbf {l}_i| > r_\mathrm{fd} \\ 0 &{} \textit{otherwise} \end{array}\right. }. \end{aligned}$$

  (33)

The parameters used to estimates the expected reward of possible behaviors can be found in Table 3.

Table 3 Parameters used in numerical simulations

1.2 Predator

The predator randomly samples a set of locations from the map every 20 time steps and values each location $\mathbf {l}$ as follows:

$$\begin{aligned} u_l = \frac{\rho _\mathbf {l}}{|\mathbf {l}_\mathrm{pd} - \mathbf {l}|}, \end{aligned}$$

(34)

where $\rho _l$ is the agent density around $\mathbf {l}$.

Then, the predator chooses the location $\mathbf {l} = \arg \max u_l$ and moves toward it. It is assumed that the predator will be reinforced if it captures agents. Let the number of agents captured by the predator be $n_\mathrm{prey}$, the speed $v_p$ of predator is given by the following:

$$\begin{aligned} v_p = v_p^0 + 0.05\cdot n_\mathrm{prey}, \end{aligned}$$

(35)

where $v_p^0$ is the initial speed of the predator. The initial speed $v_p^0$ tested in this work is outlined in Table 3.

1.3 A.3 The reward function

The total reward of a feeding area is 1000. A feeding agent only consume 0.1 reward per time step. When the reward of a feeding area runs out, it refreshes at a random location after 20 time steps.

Denote the reward collected by all agents from the feeding areas by $r_\mathrm{feed}$, the number of agents being preyed by $n_\mathrm{prey}$ and the number of collisions among the agents by $n_\mathrm{coll}$. Assume that the cost of each preyed agent is 5. Then, the total reward R of the group is given by the following:

$$\begin{aligned} R= r_\mathrm{feed} - 5n_\mathrm{prey}-n_\mathrm{coll}. \end{aligned}$$

(36)