Privacy-concerned multiagent planning

Abstract

Coordinated sequential decision making of a team of cooperative agents can be described by principles of multiagent planning. Provided that the mechanics of the environment the agents act in is described as a deterministic transitions system, an appropriate planning model is MA-Strips. Multiagent planning modeled as MA-Strips prescribes exactly what information has to be kept private and which information can be communicated in order to coordinate toward shared or individual goals. We propose a multiagent planning approach which combines compilation for a classical state-of-the-art planner together with a compact representation of local plans in the form of finite-state machines. Proving soundness and completeness of the approach, the planner efficiency is further boosted up using distributed delete-relaxation heuristics and using an approximative local plan analysis. We experimentally evaluate applicability of our approach in full privacy setting where only public information can be communicated. We analyze properties of standard multiagent benchmarks from the perspective of classification of private and public information. We show that our approach can be used with different privacy settings and that it outperforms state-of-the-art planners designed directly for particular privacy classification.

Appendix: Proofs of main theorems

Theorem 1 Public plan $\sigma$ of $\varPi$ is extensible if and only if $\sigma$ is $\alpha$-extensible for every agent $\alpha$.

Proof

($\Rightarrow$). When $\sigma$ is extensible, there is $\pi \in \textsf {sols} ({\varPi })$ such that ${\pi }\mathop {\triangleright }{\star }=\sigma$. Let $\alpha$ be arbitrary but fixed. Let us construct plan $\pi _\alpha$ of ${\varPi }\mathop {\triangleright }{\alpha }$ from $\pi$ by removing internal actions of agents other than $\alpha$ and by applying projection to the remaining actions obtaining $\pi _\alpha = \langle {a}\mathop {\triangleright }{\alpha }: a\in \pi \ \text{ and } \ a\in \textsf {pub-actions} (\varPi )\cup \textsf {int-actions} (\alpha )\rangle$. Clearly ${\pi _\alpha }\mathop {\triangleright }{\star }=\sigma$ because $\pi _\alpha$ preserves the order of public actions. To prove $\pi _\alpha \in \textsf {sols} ({\varPi }\mathop {\triangleright }{\alpha })$, we first observe that no action b internal for $\beta \ne \alpha$ can change state $s$ of $\varPi$ in a way observable by $\alpha$, that is, ${\gamma (s,b)}\mathop {\triangleright }{\alpha }={s}\mathop {\triangleright }{\alpha }$. Hence, the sequence of states (of $\varPi$) which proves $\pi \in \textsf {sols} ({\varPi })$ can be easily transformed to a sequence of states of (${\varPi }\mathop {\triangleright }{\alpha }$) which proves $\pi _\alpha \in \textsf {sols} ({\varPi }\mathop {\triangleright }{\alpha })$. Thus $\sigma$ is $\alpha$-extensible. ($\Leftarrow$) For every agent $\alpha _i, \sigma$ is $\alpha _i$-extensible and thus there is some local solution $\pi _i$ such that $\pi _i\in \textsf {sols} ({{\varPi }\mathop {\triangleright }{\alpha _i}})$ and ${\pi _i}\mathop {\triangleright }{\star }=\sigma$. When more than one local solutions exist, we can choose an arbitrary from them. Now we construct a solution $\pi$ of $\varPi$ from local solutions $\pi _i$’s as follows. We split each $\pi _i$ at the positions of public actions from $\sigma$, and we join the corresponding internal parts of different plans together. Internal actions of different agents cannot interact through a shared fact (otherwise this fact would be public and these actions would be public too), and thus, we can join different internal parts in any order, preserving only the order of actions of individual agents.

Then we construct $\pi$ of $\varPi$ from $\sigma$ by translating ids from $\sigma$ to corresponding actions of $\varPi$ and by adding the joined parts between corresponding public actions in $\sigma$.

Clearly ${\pi }\mathop {\triangleright }{\star }=\sigma$ and $\pi \in \textsf {sols} ({\varPi })$. Hence, $\sigma$ is extensible. $\square$

Lemma 3 Let $\varPi$ be a classical $\textsc {Strips}$ problem, let $\varGamma$ be a PSM of $\varPi$, and let $\pi \in \textsf {sols} ({\varPi })$. Then $\varGamma \mathop {\oplus }\pi$ is correctly defined and $\textsf {accept} (\varGamma )\cup \{\pi \}\subseteq \textsf {accept} (\varGamma \mathop {\oplus }\pi )$.

Proof

Let us prove that $\varGamma \mathop {\oplus }\pi$ is correctly defined as specified by Definition 5. Properties (1)–(3) are trivial. Property (4) is satisfied because $\pi \in \textsf {sols} ({\varPi })$ and hence $G\subseteq s_n$ where $s_n$ is the last state from Definition 7. It follows from Definition 7 that both PSMs are defined on the same alphabet and that $\varGamma$ is a sub-automaton of $\varGamma \mathop {\oplus }\pi$. Hence, clearly $\textsf {accept} (\varGamma )\subseteq \textsf {accept} (\varGamma \mathop {\oplus }\pi )$. Moreover, the sequence of states $s_0, \ldots , s_n$ from Definition 7 proves that $\pi \in \textsf {accept} (\varGamma \mathop {\oplus }\pi )$. Hence, the claim. $\square$

Lemma 4 Let ${\varPi }\mathop {\triangleright }{\alpha }$ be a local problem of agent $\alpha$ and let $\varGamma$ be a PSM of ${\varPi }\mathop {\triangleright }{\alpha }$. Then ${\varGamma }\mathop {\triangleright }{\star }$ is a public PSM of $\varPi$ and $\textsf {accept} ({\varGamma }\mathop {\triangleright }{\star })=\{{\pi }\mathop {\triangleright }{\star }:\pi \in \textsf {accept} (\varGamma )\}$.

Proof

Let $\varDelta ={\varGamma }\mathop {\triangleright }{\star }$. Let $\delta _\varGamma$ be the transition function of $\varGamma$ and let $\delta _\varDelta$ be the transition function of $\varDelta$. Let us prove the inclusions ($\subseteq$) and ($\supseteq$) separately.

($\subseteq$) Let $\sigma \in \textsf {accept} (\varDelta )$ and let $\sigma =\langle \textit{id}_1,\ldots ,\textit{id}_n \rangle$. Let $s_0, \ldots , s_n$ be the sequence of states of $\varDelta$ which proves $\sigma \in \textsf {accept} (\varDelta )$. Now we can sequentially process these actions and construct a sequence of action ids $\pi '$ such that $\pi '\in \textsf {accept} (\varGamma )$ and ${\pi '}\mathop {\triangleright }{\star }=\sigma$ as follows. Thanks to the integer labels, we can unambiguously translate every state of $\varDelta$ to the state of $\varGamma$ using function $\rho ^{-1}$. Let $t_i=\rho ^{-1}(s_i)$ for $0\le i\le n$. We start with empty $\pi '$. We know that the transition from $s_{i-1}$ to $s_i$ labeled by $id _i$ has been added to $\varDelta$ by line 17 of Algorithm 3. Hence, there is state r of $\varGamma$ such that $r\in \textsf {int-closure} _{}(t_{i-1})$ and $\delta _\varGamma (r, id _i)=t_i$. Hence, there has to be a (possibly empty) sequence of internal action ids $\langle id _1', \ldots , id _l' \rangle$ which proves $r\in \textsf {int-closure} _{}(t_{i-1})$. We simply append $\langle id _1', \ldots , id _l', id _i \rangle$ to $\pi '$. In this way, we construct $\pi '$ by sequential processing of all action ids from $\sigma$. We know that $s_0$ is the initial state of $\varDelta$ and also that $t_0$ is the initial state of $\varGamma$. It holds that ${\pi '}\mathop {\triangleright }{\star }=\sigma$ because all the actions from $\sigma$ were added to $\pi '$ in the right order and the additionally added actions are internal. To prove the claim, it is now enough to check that $\pi '\in \textsf {accept} (\varGamma )$. When $t_n$ is an accepting state of $\varGamma$, we are done. Otherwise, $s_n$ is marked as an accepting state of $\varDelta$ by line 19 and therefore there exists some accepting state $r'$ of $\varGamma$ such that $t_n\in \textsf {int-closure} _{}(r')$. Finally, we append internal action ids which prove $t_n\in \textsf {int-closure} _{}(r')$ to $\pi '$. Thus $\pi '\in \textsf {accept} (\varGamma )$ and hence the claim.

($\supseteq$) Let $\pi \in \textsf {accept} (\varGamma )$ and let $\sigma ={\pi }\mathop {\triangleright }{\star }$. We simulate the plan $\pi = \langle id _1, \ldots , id _n \rangle$ in the state space of $\varGamma$. We shall show that this simulation directly corresponds to the simulation of $\sigma$ in $\varDelta$. Let $s_0,\ldots ,s_n$ be the sequence of states of $\varGamma$ which proves $\pi \in \textsf {accept} (\varGamma )$. Clearly the initial state of $\varGamma$ (that is, $s_0$) is translated by $\rho$ to the initial state of $\varDelta$. For a transition from $s_{i-1}$ to $s_i$ labeled by $id _i$ in $\varGamma$, we distinguish two following two cases. Either (1) $id _i$ is public or (2) internal. If $id _i$ is public, it is trivially added by line 17 to $\varDelta$ and thus we can follow the corresponding transition in $\varDelta$. If $id _i$ is internal, we find the first transition with public action $\delta (s_{j-1}, id _j) \rightarrow s_j, j>i$. Note that all internal actions are removed when doing a public projection. Thus, we can proceed similarly to the previous case having virtual transition from $\delta (s_{i-1}, id _i) \rightarrow s_j$ with the only difference that now the needed transition is added because $s_j \in \textsf {int-closure} _{}(s)$. It can happen that no such index j exists, i.e., the plan $\pi$ ends with a sequence of internal actions. In that case, the state $\rho (s_{i-1})$ is going to be added to the goal states at line 19. $\square$

Lemma 5 The intersection $\varDelta _1\cap \varDelta _2$ of two public PSMs $\varDelta _1$ and $\varDelta _2$ of $\varPi$ is a correctly defined public PSM of $\varPi$ and the following holds.

$$\begin{aligned} \textsf {accept} (\varDelta _1\cap \varDelta _2) = \textsf {accept} (\varDelta _1)\cap \textsf {accept} (\varDelta _2) \end{aligned}$$

Proof

Let $\varDelta _1=\langle \Sigma ,S_1,I,\delta _1,F_1 \rangle$ and $\varDelta _2=\langle \Sigma ,S_2,I,\delta _2,F_2 \rangle$. Let $\varDelta _0=\varDelta _1\cap \varDelta _2$. Let us first prove that the $\varDelta _1\cap \varDelta _2$ is a correctly defined PSM of MA-Strips problem $\varPi$ as specified in Definition 9. Properties (1) to (3) are trivially fulfilled. Property (4) is proved by Definition 12 (2) and property (5) by Definition 12 (3). Now let us prove the inclusions ($\subseteq$) and ($\supseteq$) separately.

($\subseteq$) Let $\sigma =\langle id _1,\ldots , id _n \rangle$ be a public plan such that $\sigma \in \textsf {accept} (\varDelta _0)$. Let $\langle s_0,l_0 \rangle , \ldots , \langle s_n,l_n \rangle$ be the sequence of states which proves $\sigma \in \textsf {accept} (\varDelta _0)$. Thanks to the distinctiveness property of an injective function $\cdot$, we can find $i_k$ and $j_k$ such that $l_k=i_k\cdot j_k$ for every $0\le k\le n$. It holds that $\langle s_n,i_n \rangle \in F_1$ and $\langle s_n,j_n \rangle \in F_2$ by Definition 12 (3). Hence, the sequence of states $\langle s_0,i_0 \rangle , \ldots , \langle s_n,i_n \rangle$ proves that $\sigma \in \textsf {accept} (\varDelta _1)$ by Definition 12 (2). Similarly, $\sigma \in \textsf {accept} (\varDelta _2)$ and hence the claim.

($\supseteq$) Let $\varDelta _0=\langle \Sigma ,S_0,I,\delta _1,F_0 \rangle$. Let $\sigma =\langle id _1,\ldots , id _n \rangle$ be a public plan such that $\sigma \in \textsf {accept} (\varDelta _1)\cap \textsf {accept} (\varDelta _2)$ Let $\langle s_0,i_0 \rangle , \ldots , \langle s_n,i_n \rangle$ be the sequence of states which proves $\sigma \in \textsf {accept} (\varDelta _1)$ and let $\langle q_0,j_0 \rangle , \ldots , \langle q_n,j_n \rangle$ be the sequence of states which proves $\sigma \in \textsf {accept} (\varDelta _2)$. We know that $\langle s_0,i_0 \rangle =\langle q_0,j_0 \rangle =I$. Hence, it is easy to check by Definition 9 (4) that $s_k=q_k$ for all $0\le k \le n$. Also we know that $\langle s_n,i_n \rangle \in F_1$ and $\langle s_n,j_n \rangle =\langle q_n,j_n \rangle \in F_2$. Hence, $\langle s_n,i_n\cdot j_n \rangle \in F_0$ by Definition 12 (3). Now the sequence of states $\langle s_0,i_0\cdot j_0 \rangle , \ldots , \langle s_n,i_n\cdot j_n \rangle$ proves that $\sigma \in \textsf {accept} (\varDelta _0)$ by Definition 12 (2) and hence the claim. $\square$

Theorem 2 Let $\varPi$ be an MA-Strips problem and $\varDelta =\mathtt {PsmPlanComplete}(\varPi )$. It holds that $\textsf {accept} (\varDelta ) = \{ {\pi }\mathop {\triangleright }{\star }: \pi \in \textsf {sols} ({\varPi })\}$.

Proof

Let $\varGamma _\alpha$ denote the complete PSM of ${\varPi }\mathop {\triangleright }{\alpha }$ and $\varDelta _\alpha ={\varGamma _\alpha }\mathop {\triangleright }{\star }$. Let us prove the inclusions ($\subseteq$) and ($\supseteq$) separately.

($\subseteq$) Let $\sigma \in \textsf {accept} (\varDelta )$. Then $\sigma \in \textsf {accept} (\varDelta _\alpha )$ for every $\alpha$ by Lemma 5. Hence, for every $\alpha$ there is $\pi _\alpha \in \textsf {accept} (\varGamma _\alpha )$ such that $\sigma ={\pi _\alpha }\mathop {\triangleright }{\star }$ by Lemma 4. By Lemma 2 $\pi _\alpha \in \textsf {sols} ({{\varPi }\mathop {\triangleright }{\alpha }})$ and thus $\sigma$ is $\alpha$-extensible for every $\alpha$. Hence, $\sigma$ is extensible by Theorem 1.

($\supseteq$) Let $\pi \in \textsf {sols} ({\varPi })$ and $\sigma ={\pi }\mathop {\triangleright }{\star }$. Hence, $\sigma$ is extensible and thus also $\alpha$-extensible for every $\alpha$ by Theorem 1. Hence, for every $\alpha$ there is $\pi _\alpha \in \textsf {sols} ({{\varPi }\mathop {\triangleright }{\alpha }})$ with ${\pi _\alpha }\mathop {\triangleright }{\star }=\sigma$. Clearly $\pi _\alpha \in \textsf {accept} (\varGamma _\alpha )$ by Lemma 2 and thus $\sigma \in \textsf {accept} (\varDelta _\alpha )$ by Lemma 4. Thus $\sigma \in \textsf {accept} (\varDelta _\alpha )$ for all $\alpha$ and hence the claim by Lemma 5. $\square$

Theorem 4 Let $\varPi$ be an MA-Strips problem, let $\alpha$ be an agent of $\varPi$, and let $\sigma$ be a public plan of $\varPi$. Then $\sigma$ is $\alpha$-extensible iff $\textsf {sols} ({{\varPi }\mathop {\circledast }_{\alpha }{\sigma }})\ne \emptyset$.

Proof

($\Rightarrow$) Let $\sigma$ be $\alpha$-extensible. Hence there is $\pi \in \textsf {sols} ({{\varPi }\mathop {\triangleright }{\alpha }})$ such that ${\pi }\mathop {\triangleright }{\star }=\sigma$. Clearly $\pi$ contains only public actions in the order given by $\sigma$. The rest are internal actions of $\alpha$. We construct $\pi _0$ from $\pi$ by replacing public actions by their respective landmark actions. It is easy to verify that $\pi _0\in \textsf {sols} ({{\varPi }\mathop {\circledast }_{\alpha }{\sigma }})$.

($\Leftarrow$) Let $\pi \in \textsf {sols} ({{\varPi }\mathop {\circledast }_{\alpha }{\sigma }})$. Clearly ${\pi }\mathop {\triangleright }{\star }=\sigma$. Let us construct $\pi _0$ be translating landmark actions back to their original actions. It still holds ${\pi _0}\mathop {\triangleright }{\star }=\sigma$, and it is easy to check that $\pi _0\in \textsf {sols} ({{\varPi }\mathop {\triangleright }{\alpha }})$. Hence, $\sigma$ is $\alpha$-extensible. $\square$