ProvDP: Differential Privacy for System Provenance Dataset

Abstract

Provenance-based Intrusion Detection System (PIDS) is getting widely deployed for safeguarding enterprises across various verticals against sophisticated cyberattacks such as Advanced Persistent Threat (APT) campaigns. Rich in detail, provenance data are essentially fine-grained activities of users logged at their endpoints. Therefore, a security breach of provenance data can result in the leak of private information pertaining to users and the enterprise they work for (e.g., the clients to which a user often communicates), raising significant privacy concerns.

In this work, we propose a novel privacy-preserving solution, ProvDP, specifically tailored to protect the privacy of provenance data and focusing on provenance graphs. Our approach introduces multiple technical contributions: (1) a novel method for converting provenance graphs to and from provenance trees, enabling us to harness properties of trees to develop privacy-preserving techniques while preserving the semantic value of provenance graphs, (2) a novel subtree differential privacy framework for providing privacy guarantee on these provenance trees, and (3) empirical evidence that the application of differential privacy does not diminish the detection accuracy of PIDSes. Our evaluation demonstrates that PIDS trained on differentially private data maintain utility while preserving privacy.

A Appendix

1.1 A.1 Extended Top-m Filter Algorithm

1.2 A.2 Prune Subtree at Single Point Algorithm

1.3 A.3 Prune Subtree at k Point Algorithm

1.4 A.4 Graft Subtree Algorithm

1.5 A.5 $\epsilon$-Edge-Differential Privacy ($\epsilon$-Edge-DP) Proof for Algorithm 1

The ETmF algorithm which is adapt the TmF [38] is composed of two mechanisms which are $\epsilon _1$-Edge-DP and $\epsilon _2$-Edge-DP, and ETmF is $\epsilon$-Edge-DP by sequential composition (based on Theorem 2.1 in [38]) where $\epsilon = \epsilon _1 + \epsilon _2$.

The first mechanism calculated the number of edges to be added $\tilde{m}$ by perturbing the number of edges, $m = |E|$ using Laplace noise under budget $\epsilon _2$ to produce perturbed $\tilde{m}$.

$$ \tilde{m} = \lceil m + Lap(\varDelta f/\epsilon _2)\rceil $$

Based on the definition of neighboring graphs in [38], two graphs are neighboring if they differ in a single edge, thus the global sensitivity of function f which is defined as $\varDelta f$ is 1.

$$ \tilde{m} = \lceil m + Lap(1/\epsilon _2)\rceil $$

Therefore, this step is $\epsilon _2$-Edge-DP (based on Theorem 2.1 in [38]) since Laplacian noise is added as a scaled parameter $\frac{\varDelta f}{\epsilon _2}$ with respect to global sensitivity function $\varDelta f$.

The second mechanism adds edges to the graph of size $\tilde{m}$, by first identifying candidate edges in the graph by adding Laplace noise under budget $\epsilon _1$ and then passing the candidate edges through a high pass filters with threshold $\theta$. The global sensitivity is also 1 in this case similar to first mechanism. By the assumption of edge independence, parallel composition (based on Theorem 2.2 in [38]) is applicable at edge level. Therefore, the second method is $\epsilon _1$-Edge-DP. Therefore, ETmF is $\epsilon$-Edge-DP by sequential composition (based on Theorem 2.1 in [38]) where $\epsilon = \epsilon _1 + \epsilon _2$.

In system provenance the edges are public knowledge, and with that assumption, TmF can be generalized to apply to any set of valid edges $E_{\text {valid}}$. We can brute-force enumerate over all possible edges and reject invalid edges to obtain our $E_{\text {valid}}$. To derive the new upper-bound for $\theta$, we calculate the theoretical maximum of $|E_\text {valid}|$ since $\theta$ is upper bounded by $\epsilon _t$. There ca be a directed edge between any two nodes. Therefore, the maximum number of edges is bounded by $C^n_2$ i.e.,  $n(n-1)/2$. Therefore, the upper bound of $\theta$ is $n(n-1)/2$.

1.6 A.6 $\epsilon _1$-Subtree-Differential Privacy ($\epsilon _1$-Subtree-DP) Proof for Algorithm 2

In Definition 1 we defined $\epsilon _1$-Subtree-Differential Privacy.

Definition 3

In Definition 1 we defined two trees ($T_1$ and $T_2$) as neighboring, if the differ by a single subtree, t. Now we define the global sensitivity of a function f as $\varDelta f = \underset{T_1, T_2, t}{\max }\ || f(T_1, t) - f(T_2, t)||_1$.

Using Definition 1, we consider a pair of tree neighbors $T_1, T_2$ who differ by one subtree t, $\Pr [R(T_1) = x]$ which is proportional to $\exp \left( \frac{\epsilon _1 f(T_2, t)}{S}\right)$ using exponential distribution [29, 30]. Therefore, the ratio of $\Pr [R(T_1) = x]$ and $\Pr [R(T_2) = x]$ is equivalent to:

$$ \frac{\exp \left( \frac{\epsilon _1 f(T_1, t)}{S}\right) }{\exp \left( \frac{\epsilon _1 f(T_2, t)}{S}\right) }. $$

This can be simplified to $e^{\epsilon _1}$ as the maximum value of $f(T_1, t) - f(T_2, t)$ is S,

$$ \exp \left( \frac{\epsilon _1 (f(T_1, t) - f(T_2, t))}{S}\right) \le \exp \left( \frac{\epsilon _1 S}{S}\right) = e^{\epsilon _1} $$

For Algorithm 2, we will describe the effect of subtree size at that node, degree of the node, subtree height, and subtree depth on the global sensitivity calculation. We define global sensitivity of function f with respect to size as $\varDelta f_{size} = \underset{v \in V \setminus V_{root}}{\max } \{|V| - \text {(number of nodes after pruning }T_v\text {)}\}$. It can be rewritten in terms of subtree $T_v$ pruned, $\varDelta f = \underset{v \in V}{\max }\ |T_v|$.

The height parameter does not contribute to the global sensitivity if there is more than one longest alternative root-to-leaf paths. If node v which is on one of the longest root-to-leaf paths is pruned, the other longest root-to-leaf paths survive. Therefore, the height of the tree $\tilde{T}$ remains same as the height of T. If and only if there is one unique longest root-to-leaf path and the pruned node v is a member of the path, the sensitivity with respect to height becomes $\varDelta f_{height} \leftarrow \max (height(T) - height(\tilde{T}), height(T_v))$. Therefore, the upper bound of $\varDelta f_{height} \leftarrow height(T) - 1$.

The depth does not contribute to the global sensitivity as the depth is not a property of a tree rather it is a property of the node which is pruned. Therefore, it will contribute towards the local sensitivity of the pruned subtree. Let us consider $L_T$ is the set of leaf nodes of the tree T. Therefore, the upper limit of the local sensitivity with respect to the depth becomes $\varDelta f_{depth} \leftarrow \max _{v \in L_T} depth(v)$.

The branching factor $b_T$ of a tree T is defined as the maximum over the degrees of all the nodes in a tree, $b_T \leftarrow \max _{v \in T} {degree(v)}$. Let us define the least common ancestor $LCA_b \leftarrow lca(v_1, v_2, ..., v_k) \mid degree_{i= 1...k}(v_i) = b_T$. The branching factor of the tree reduces if and only if we prune a node which is either $LCA_b$ or its ancestors. Therefore, the sensitivity with respect to the branching factor becomes $\varDelta f_{branching\_factor} \leftarrow b_T$. Please note if the $LCA_b$ is living at level 1 of the tree and only node at level 1 then the above mentioned scenario occurs.

Finally, we calculate the global sensitivity $\varDelta f$ by taking a weighted sum of $\varDelta f_{size}$, $\varDelta f_{height}$, $\varDelta f_{depth}$, and $\varDelta f_{branching\_factor}$.

$$ \varDelta f \leftarrow \alpha \cdot \varDelta f_{size} + \beta \cdot \varDelta f_{height} + \gamma \cdot \varDelta f_{depth} + \eta \cdot \varDelta f_{branching\_factor} $$

Special Scenario: Lets us assume that the session behaviors have similar structures. Therefore, a tree T contains user activities that are uniform in nature across different sessions and T contains multiple similar subtrees.

The subtree size strictly decreases as depth increases, thus this v that gives the global sensitivity must be located at depth 1. At depth $\ge$ 2, the subtrees cannot be a candidate as there always exists a subtree rooted at depth 1 which will be larger in size by at least 1 node. Using our assumption of uniform user activity, all subtrees at depth 1 are of equal size, then the size of any subtree rooted at depth 1 will be $\frac{|V|-1}{degree(v_{root})}$, where $degree(v_{root})$ is the number of user behavior contained in the tree. For height and branching factor since the subtree are not unique the effect on global sensitivity is nullified. The sensitivity due to the depth is maximum when we prune at leaf. In such scenario, the depth will be the $height - 1$ which is equal to $\frac{|V|-1}{degree(v_{root})}$.

These two cases cannot occur together. We can either prune at level 1 or we can prune at leaf. Therefore, $\varDelta f$ for this special scenario:

$$ \varDelta f \leftarrow \max (\alpha , \gamma ) * \frac{|V|-1}{degree(v_{root})} $$

1.7 A.7 $\epsilon _{k}$-Subtree-Differential Privacy ($\epsilon _{k}$-Subtree-DP) Proof for Algorithm 3

In Definition 2 we defined $\epsilon _{k}$-Subtree-Differential Privacy. Using the sequential composition theorem (Theorem 3) from [29] which states any sequence of computations that each provide differential privacy in isolation also provide differential privacy in sequence. Mathematically, we denote it as, if a randomizer R provide $\epsilon _i$- differential privacy, then the sequence of R provides $\sum _{i} \epsilon _i$-differential privacy.

In k-subtree pruning algorithm (Algorithm 3), we perform subtree pruning (Algorithm 2) in sequence k times on the tree T. In Sect. A.6 we showed that Algorithm 2 is $\epsilon _1$-Subtree-DP. Therefore applying Algorithm 2, k times in sequence will provide $\sum _{i=1}^{k} e_{1}$ Subtree differential privacy. We define $\epsilon _{k}$ as $\epsilon _{k} = \sum _{i=1}^{k} e_{1}$. Therefore, Algorithm 3 is $\epsilon _{k}$-Subtree-DP (using the sequential composition theorem from [29]).

1.8 A.8 $\epsilon _{k^\prime }$-Subtree-Differential Privacy ($\epsilon _{k^\prime }$-Subtree-DP) Proof for Algorithm 4

In the grafting algorithm (Algorithm 4), we perform grafting once which is $\epsilon _{2}$-Subtree-DP because in line 6, we add Laplacian noise (e.g.,  $Lap(1/\epsilon _2)$) to $s_v$ to get $\tilde{s}_v$ which affects the grafting probability. This step is $\epsilon _2$-Edge-DP (based on Theorem 2.1 in [38]) since Laplacian noise is added as a scaled parameter $\frac{\varDelta f}{\epsilon _2}$ with respect to global sensitivity function $\varDelta f$.

In the grafting algorithm (Algorithm 4), we perform subtree grafting in sequence $k^\prime$ times on the tree $\tilde{T}$. We showed that grafting one subtree is $\epsilon _2$-Subtree-DP. Therefore, applying it $k^\prime$ times in sequence will provide $\sum _{i=1}^{k^\prime } e_2$ Subtree differential privacy. We define $\epsilon _{k^\prime }$ as $\epsilon _{k^\prime } = \sum _{i=1}^{k^\prime } e_2$. Therefore, Algorithm 4 is $\epsilon _{k^\prime }$-Subtree-DP (using the sequential composition theorem from [29]).

1.9 A.9 $\epsilon$-Subtree-Differential Privacy ($\epsilon$-Subtree-DP) Proof

We proved Algorithm 3 is $\epsilon _{k}$-Subtree-DP in Sect. A.7 and Algorithm 4 is $\epsilon _{k^\prime }$-Subtree-DP in Sect. A.8. Since, the algorithm are invoked in sequence, ProvDP is $\epsilon$-Subtree-DP where $\epsilon = \epsilon _k + \epsilon _{k^\prime }$, using the sequential composition theorem from [29].

1.10 A.10 Implementation

Our ProvDP tool is implemented in Python, comprising approximately 1K lines of code. We developed custom Python functions to construct provenance trees from graphs, filter extraneous information, and abstract node entities. This preprocessing ensures that the raw graph data is optimally prepared for our downstream task, GNN-based anomaly detection [32]. For the GNN model, we utilize the DGL [2] library, specifically employing GAT layers. The GAT layer was chosen due to recent works [31, 34] showing explanation capability of the architecture which increases the user’s confidence. These layers aggregate features from neighboring nodes to generate refined node representations. The model architecture incorporates dropout and ReLU activation functions between layers to enhance generalization.