Differentially Private Range Query on Shortest Paths

Abstract

We consider range queries on a graph under the constraints of differential privacy and query ranges are defined as the set of edges on the shortest path of the graph. Edges in the graph carry sensitive attributes and the goal is to report the sum of these attributes on the shortest path for counting query or the minimum of the attributes in a bottleneck query. We use differential privacy to ensure that answering these queries does not violate the privacy of the sensitive edge attributes. Our goal is to design mechanisms that minimize the additive error of the output with the given privacy budget.

For this, we develop the first set of non-trivial results for private range queries on shortest paths. For counting range queries we can achieve an additive error of $\widetilde{O}(n^{1/3})$ for $\varepsilon$-DP and $\widetilde{O}(n^{1/4})$ for $(\varepsilon , \delta )$-DP. We present two algorithms where we control the final error by carefully balancing perturbation added to the edge attributes directly versus perturbation added to (a subset of) range query answers. Bottleneck range queries are easier and can be answered with polylogarithmic additive errors.

Appendices

A Range Query on All Paths

When we allow queries along any path in a graph and require differential privacy guarantees, the following result provides a lower bound of $\Omega (n)$ on the additive error. To show the lower bound, we first consider a range query formulated by the incidence matrix A, with m columns corresponding to the m edges in the graph G and rows corresponding to all queries. A query along path P is represented by a row in the matrix with an element of 1 corresponding to edge e if e is on P and 0 otherwise. We will then talk about the discrepancy of matrix A.

The classical notion of discrepancy of a matrix A is the minimum value of $||Ax||_{\infty }$, where x is a vector with elements taking values $+1$ or $-1$. And the hereditary discrepancy of A is the maximum discrepancy of A limited on any subset of columns. As shown in [36], both discrepancy and hereditary discrepancy of A provides a lower bound on the additive error of differentially private range query using incidence matrix A.

Theorem 4

A $(\varepsilon , \delta )$-differential privacy mechanism that answers range queries where ranges are defined on any path of an input graph has to incur additive error of $\Omega (n)$.

Proof

Consider a graph of $n+1$ vertices $v_1, v_2, \cdots , v_{n+1}$ and 2n edges. Between vertices $v_i$ and $v_{i+1}$ there are two parallel edges $e_i$ and $e'_i$. On this graph there are $2^n$ paths from $v_1$ to $v_{n+1}$. We consider only queries along these paths and the incidence matrix is a tall matrix A of $2^n$ rows and 2n columns, corresponding to the 2n edges in the graph. Now we take a submatrix of A with only the columns corresponding to edges $e_i$. This gives a matrix $A'$ of $2^n \times n$, with the rows corresponding to all subsets of [n]. $A'$ has discrepancy of $\Omega (n)$. To see that, consider the specific vector x that minimizes $||A'x||_{\infty }$. Suppose x has k entries of $+1$ and $n-k$ entries of $-1$. Without loss of generality, we assume $k\ge n/2$, The row of A that has value 1 corresponding to the positive entries of x and value 0 corresponding to the negative entries of x, gives a value of $k\ge n/2$. Thus $||Ax||_{\infty }$ is at least n/2. This means that the hereditary discrepancy of A is at least $\Omega (n)$.

By the same argument and use Corollary 1 in [36], we conclude that any $(\varepsilon , \delta )$-differentially private mechanism has to have error of $\Omega (n)$.

B Proof of Lemma 8 – $(\varepsilon , \delta )$ Algorithm for Tree Graphs

Proof

We first claim that we can answer all pairs shortest distance on a tree with $(\alpha ,\beta )$-accuracy for

$$ \alpha = O\left( \frac{1}{\varepsilon }\log n \sqrt{\log \left( \frac{n}{\beta }\right) \log \left( \frac{1}{\delta }\right) }\right) $$

showing the utility guarantee of Lemma 8. Specifically, if we wish to have high probability bounds for the shortest path distance errors, i.e., $\beta =O(1/n)$, the error is upper bounded by $O\left( \frac{1}{\varepsilon }\log ^{1.5} n \sqrt{\log \left( \frac{1}{\delta }\right) }\right)$.

In Sealfon’s algorithm [45], a tree rooted at $v_0$ is partitioned into subtrees each of at most n/2 vertices. Specifically, define $v^*$ to be the vertex with at least n/2 descendants but none of $v^*$’s children has more than n/2 descendants. The tree is partitioned into the subtrees rooted at the children of $v^*$, and a subtree of the remaining vertices rooted at $v_0$. In Sealfon’s algorithm a Laplace noise of ${{\textsf {Lap}}(\log n/\varepsilon )}$ is added to the shortest path distance from $v_0$ to $v^*$ and the edges from $v^*$ to each of its children. The algorithm then repeatedly privatizes each of the subtrees recursively. Using Sealfon’s algorithm, we know that for a given root node $v_0$, computing the single source (with the root being the source) shortest path distance requires adding at most $O(\log n)$ privatized edges. Further, their algorithm ensures that any edge can be in at most $\log n$ levels of recursion and hence can be used to compute $O(\log n)$ noisy answers. In other words, the number of adaptive compositions we need is $O(\log n)$.

We use the Gaussian mechanism to privatize the edges. Since we are concerned with approximate-DP guarantee, the variance of the noise required to preserve $(\varepsilon ,\delta )$-differential privacy is $\sigma ^2 := O\left( \frac{1}{\varepsilon ^2}\log (1/\delta ) \log n\right)$.

Fix a node u. Let $\widehat{d}(u,v_0)$ be the distance estimated by using Sealfon’s algorithm instantiated with the Gaussian mechanism instead of the Laplace mechanism. Now the noise added are zero mean. Therefore,

$$ \mathbb {E} [\widehat{d}(u,v_0)] = d(u,v_0). $$

Using the standard concentration of Gaussian distribution [50] implies that

$$\begin{aligned} \textsf{Pr} \left( \left| \widehat{d}(u,v_0) - \mathbb {E} \left[ \widehat{d}(u,v_0) \right] \right| > a \right)&\le 2 e^{-a^2/(2 \sigma ^2 \log n)}. \end{aligned}$$

Setting $a = \frac{C}{\varepsilon }\log n \sqrt{\log \left( \frac{2n}{\beta }\right) \log \left( \frac{1}{\delta }\right) }$ for some constant $C>0$, we have

$$\begin{aligned}&\textsf{Pr} \left( \left| \widehat{d}(u,v_0) - \mathbb {E} \left[ \widehat{d}(u,v_0) \right] \right| > \frac{C}{\varepsilon }\log n \sqrt{\log \left( \frac{2n}{\beta }\right) \log \left( \frac{1}{\delta }\right) } \right) \\ \le&2 e^{-C \log (2n/\beta )} \le \frac{\beta }{ n}. \end{aligned}$$

Now union bound gives that

$$ \textsf{Pr} \left( \max _{u \in \mathcal {V}} \left| \widehat{d}(u,v_0) - d(u,v_0) \right| \le \frac{C}{\varepsilon }\log n \sqrt{\log \left( \frac{n}{\beta }\right) \log \left( \frac{1}{\delta }\right) } \right) \ge 1- \beta . $$

We can now use the above result to answer all pair shortest paths by fixing a node $v^*$ to be the root note and compute a single source shortest distance with the root node being the source node. Once we have all these estimates, to compute all pair shortest distance, for any two vertices, $(u,v) \in \mathcal {V} \times \mathcal {V}$, we first compute the least common ancestor z of u and v. We then compute the distance as follows:

$$ \widehat{d}(u,v) = \widehat{d}(u,v^*) + \widehat{d}(v,v*) - 2 \widehat{d}(z,v^*). $$

Since each of these estimates can be computed with an absolute error:

$$O\left( \frac{1}{\varepsilon }\log n \sqrt{\log \left( \frac{n}{\beta }\right) \log \left( \frac{1}{\delta }\right) }\right) ,$$

we get the final additive error bound. That is,

$$ \Pr \left( \max _{u,v \in \mathcal {V}}|\widehat{\textsf{d}} (u,v)-\textsf{d}(u,v)| = O\left( \frac{1}{\varepsilon }\log n \sqrt{\log \left( \frac{n}{\beta }\right) \log \left( \frac{1}{\delta }\right) }\right) \right) \ge 1-\beta $$

completing the proof of the claim.

C Proof of Lemma 6

Proof

(Proof of Lemma 6). The lemma is proved by a simple application of the Chernoff bound. For each path P(u, v) with more than $\frac{n}{\zeta }$ edges, let $v'$ be the $\left( \frac{n}{\zeta }+1\right)$-th vertices on the path P(u, v) from u. Similarly, let $u'$ be the $\left( \frac{n}{\zeta }+1\right)$-th vertices on the path P(u, v) from v (traversing backward). We show that there must be two vertices sampled in S on both $P(u, v')$ and $P(u',v)$, which is sufficient to prove the lemma statement.

Define $X_{u,v'}$ as the random variable for the number of vertices on $P(u,v')$ that are sampled in S, and define $X_{z}$ for each $z \in P(u,v')$ as the indicator random variable for z to be sampled in S. It is straightforward to see that $X_{u,v'}=\sum _{z\in P(u,v')} X_{z}$. Since $P(u,v')$ has at least $\frac{n}{\zeta }$ vertices, and we are sampling $s=100\, \log {n}\cdot \zeta$ vertices uniformly at random as S, the expected number of vertices on $P(u,v')$ that are sampled is at least $100\, \log {n}$. Formally, we have

$$\begin{aligned} \mathop {\mathrm {{\mathbb {E}}}}\limits \left[ X_{u,v'}\right] \ge 100\, \log {n}\cdot \frac{\zeta }{n}\cdot \frac{n}{\zeta } = 100\, \log {n}. \end{aligned}$$

As such, by applying the multiplicative Chernoff bound, we have

$$\begin{aligned} \Pr \left[ X_{u,v'}\le 2\right]&\le \exp \left( -\frac{0.8^2\cdot 100\, \log {n}}{3}\right) \\&\le \frac{1}{n^{10}}. \end{aligned}$$

The same argument can be applied to $P(u',v)$ by defining $X_{u',v}$ as the total number of vertices that are sampled in S. We omit the repetitive details for simplicity. Finally, although the random variables for different (u, v) pairs are dependent, we can still apply a union bound regardless the dependence, and get the desired statement.

D A Remark on Range Query Shortest Path Lower Bound

For counting range queries with $(\varepsilon , \delta )$-DP guarantee, there is a lower bound of $\Omega (n^{1/6})$ on the additive error, adapted from the construction of the lower bound for private all pairs shortest distances [10]. Specifically, the construction uses a graph where vertices are points in the plane and edges map to line segments between two points that do not contain other vertices. The edge length is the Euclidean length and therefore the shortest path between two vertices is the path corresponding to a straight line. The range query problem can be now formulated as a (special case) of linear queries, as in Sect. 6 and Sect. A, where the matrix A corresponds to the incidence matrix of the shortest paths and the edges in the graph. It is known that this matrix has a discrepancy lower bound of $\Omega (n^{1/6})$ [34]. By the connection of the discrepancy and linear query lower bounds [36], this is a lower bound for our problem.