Model Checking Differentially Private Properties

Abstract

We introduce the branching time temporal logic for specifying differential privacy. Several differentially private mechanisms are formalized as Markov chains or Markov decision processes. Using our formal models, subtle privacy conditions are specified by . In order to verify privacy properties automatically, model checking problems are investigated. We give a model checking algorithm for Markov chains. Model checking properties on Markov decision processes however is shown to be undecidable.

D. Liu and L. Zhang—Partially supported by the National Natural Science Foundation of China (Grants No. 61532019, 61761136011, 61472473).

B.-Y. Wang—Partially supported by the Academia Sinica Thematic Project: Socially Accountable Privacy Framework for Secondary Data Usage.

A Proof of Theorem 1

Proof

The proof follows by a reduction from the emptiness problem for probabilistic automata. A probabilistic automaton [29] is a tuple $\mathcal {A}= (S,\varSigma ,M,s_0,B)$ where

• S is a finite set of states,

• $\varSigma$ is the finite set of input alphabet,

• $M: S\times \varSigma \times S \rightarrow [0,1]$ such that $\sum _{t\in S} M(s,\alpha ,t)=1$ for all $s\in S$ and $\alpha \in \varSigma$,

• $s_0\in S$ is the initial state,

• $B\subseteq S$ is a set of accepting states.

Each input alphabet $\alpha$ induces a stochastic matrix $M(\alpha )$ in the obvious way. Let $\lambda$ denote the empty string. For $\eta \in \varSigma ^*$ we define $M(\eta )$ inductively by: $M(\lambda )$ is the identity matrix, $M(x\eta ')=M(x)M(\eta ')$. Thus, $M(\eta )(s,s')$ denotes the probability of going from s to $s'$ after reading $\eta$. Let $v_B$ denote the characteristic row vector for the set B, and $v_{s_0}$ denote the characteristic row vector for the set $\{s_0\}$. Then, the accepting probably of $\eta$ by $\mathcal {A}$ is defined as $v_{s_0}\cdot M(\eta )\cdot (v_B)^c$ where $(v_B)^c$ denotes the transpose of $v_B$. The following emptiness problem is know to be undecidable [27]:

Emptiness Problem: Given a probabilistic automaton $\mathcal {A}= (S,\varSigma ,M,s_0,B)$, whether there exists $\eta \in \varSigma ^*$ such that $v_{s_0}\cdot M(\eta )\cdot (v_B)^c > 0$?

Now we establish the proof by reducing the emptiness problem to our model checking problem. Given the probabilistic automaton $\mathcal {A}= (S,\varSigma ,M,s_0,B)$, assume we have a primed copy $\mathcal {A}'= (S',\varSigma ,M',s_0',\emptyset )$.

Let $AP:=\{at_B\}$. Now we construct our MDP where $\wp (s,a,t)$ equals to M(s, a, t) if $s,t\in S$ and to $M'(s,a,t)$ if $s,t\in S'$. We define the neighbor relation $N_S:=\{(s_0,s_0'),(s_0',s_0)\}$ by relating states $s_0,s_0'$. The labelling function L is defined by $L(s)=\{at_B\}$ if $s\in B$ and $L(s)=\emptyset$ otherwise.

Now we consider the formula $\varPhi = \mathcal {D}_{{1}, {0}} (F at_B)$. For the reduction we prove $s_0\models \mathcal {D}_{{1}, {0}} (F at_B)$ iff for all $\eta \in \varSigma ^*$ it holds $v_{s_0}\cdot M(\eta )\cdot (v_B)^c \le 0$.

First we assume $s_0\models \mathcal {D}_{{1}, {0}} (F at_B)$. By semantics we have that for all query scheduler $\mathfrak {Q}\in \varSigma ^\omega$, ${\Pr }_{ N _{S}}^{M_{\mathfrak {Q}}}({s_0}, {F at_B}) \le e \cdot {\Pr }_{ N _{S}}^{M_{\mathfrak {Q}}}({s_0'}, {F at_B})$. Since the set of accepting state in the primed copy is empty, we have ${\Pr }_{ N _{S}}^{M_{\mathfrak {Q}}}({s_0'}, {F at_B})=0$, thus we have ${\Pr }_{ N _{S}}^{M_{\mathfrak {Q}}}({s_0}, {F at_B}) \le 0$. This implies $v_{s_0}\cdot M(\eta )\cdot (v_B)^c \le 0$ for all $\eta \in \varSigma ^*$.

For the other direction, assume that all $\eta \in \varSigma ^*$ it holds $v_{s_0}\cdot M(\eta )\cdot (v_B)^c \le 0$. We prove by contradiction. Assume that $s_0\not \models \mathcal {D}_{{1}, {0}} (F at_B)$. Since the relation $N_S=\{(s_0,s_0'),(s_0',s_0)\}$, there exists $(s_0,s_0')$, and a query scheduler $\mathfrak {Q}\in \varSigma ^\omega$ such that

$$ {\Pr }_{ N _{S}}^{M_{\mathfrak {Q}}}({s_0}, {F at_B}) \not \le e \cdot {\Pr }_{ N _{S}}^{M_{\mathfrak {Q}}}({s_0'}, {F at_B}) $$

which implies ${\Pr }_{ N _{S}}^{M_{\mathfrak {Q}}}({s_0}, {F at_B}) >0$. It is then easy to construct a finite sequence $\eta \in \varSigma ^*$ with $v_{s_0}\cdot M(\eta )\cdot (v_B)^c > 0$, a contradiction.