Improved Structured Encryption for SQL Databases via Hybrid Indexing

Abstract

We introduce a new technique for indexing joins in encrypted SQL databases called partially precomputed joins which achieves lower leakage and bandwidth than those used in prior constructions. These techniques are incorporated into state-of-the-art structured encryption schemes for SQL data, yielding a hybrid indexing scheme with both partially and fully precomputed join indexes. We then introduce the idea of leakage-aware query planning by giving a heuristic that helps the client decide, at query time, which index to use so as to minimize leakage and stay below a given bandwidth budget. We conclude by simulating our constructions on real datasets, showing that our heuristic is accurate and that partially-precomputed joins perform well in practice.

Appendices

A CJJ+’s Multimap/Dictionary Encryption Schemes

Fig. 13.

Algorithms for RH dictionary encryption scheme $\mathsf {Dye}_{\pi }$ and RR multimap encryption scheme $\mathsf {Mme}^{\mathrm {rr}}_{\pi }$.

In our StE scheme constructed using $\mathbf {SqlStE}$ (in Sect. 3) we use a specific RH dictionary encryption scheme to store the rows in $\mathsf {DB}$. We formalize this as $\mathsf {Dye}_{\pi }$ whose algorithms are in Fig. 13. The primitives (given as input to $\mathbf {SqlStE}$) used in $\mathsf {Dye}_{\pi }$ are symmetric encryption scheme $\mathsf {SE}$ and function family $\mathsf {F}$. Note that $\mathsf {Dye}_{\pi }\mathsf {.KS}=\mathsf {F}\mathsf {.KS}\times \mathsf {SE}\mathsf {.KS}$.

In our StI schemes such as $\mathsf {PpJn},\mathsf {PpSj},\mathsf {HybStI}$ we use a RR multimap $\mathsf {Mme}$ as a primitive. We give an example of such a scheme $\mathsf {Mme}^{\mathrm {rr}}_{\pi }$ which is also based on $\Pi _{\mathrm {bas}}$. Its algorithms are in algorithms are in Fig. 13. The primitives are as in $\mathsf {Dye}_{\pi }$ but we require that $\mathsf {SE}\mathsf {.KS}=\{0,1\}^{{\mathsf {F}\mathsf {.ol}}}$. Note that $\mathsf {Mme}^{\mathrm {rr}}_{\pi }\mathsf {.KS}=\mathsf {F}\mathsf {.KS}$.

BProof of Theorem 1

Theorem 1. Let $\mathsf {StE}=\mathbf {SqlStE}[\mathsf {StI},\mathsf {SE},\mathsf {F}]$ be a correct StE scheme for $\mathsf {SqlDT}$. Then given algorithms $\mathcal {L}^{\mathrm {i}},\mathcal {S}^{\mathrm {i}}$ and adversary $A$ we can define $\mathcal {L}$ as in Sect. 3.2 and construct $\mathcal {S},A_{\mathrm {s}},A_{\mathrm {f}},A_{\mathrm {i}}$ such that:

$$\mathbf {Adv}^{\mathrm {ss}}_{\mathsf {StE},\mathcal {L},\mathcal {S}}(A)\le \mathbf {Adv}^{\mathrm {ind\$}}_{\mathsf {SE}}(A_{\mathrm {s}}) + \mathbf {Adv}^{\mathrm {prf}}_{\mathsf {F}}(A_{\mathrm {f}}) + \mathbf {Adv}^{\mathrm {ss}}_{\mathsf {StI},\mathcal {L}^{\mathrm {i}},\mathcal {S}^{\mathrm {i}}}(()A_{\mathrm {i}}).$$

Proof

The adversaries, simulator and games $\mathrm {G}_0,\mathrm {G}_1,\mathrm {G}_2,\mathrm {G}_3$ are given in Fig. 14. Notice that the $\mathsf {EncRows}$ algorithm used in the adversaries and games is given at the top, and uses two oracles $\textsc {Enc},\textsc {Fn}$ which the algorithms define. Let b be the challenge bit selected in $\mathrm {G}^{\mathrm {ss}}_{{\mathsf {StE},\mathcal {L},\mathcal {S}}}(A)$.

Notice that we can express $\mathbf {Adv}^{\mathrm {ss}}_{\mathsf {StE},\mathcal {L},\mathcal {S}}(A)=\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {StE},\mathcal {L},\mathcal {S}}}(A) | b=1] - \Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {StE},\mathcal {L},\mathcal {S}}}(A) | b=0]=\Pr [\mathrm {G}_3]-\Pr [\mathrm {G}_0]$. In $b=1$ case, this follows directly from the definition of $A_{\mathrm {i}}$. In the $b=0$ case, this follows from the definition of $\mathcal {L}^{\mathrm {i}},\mathcal {S}^{\mathrm {i}}$.

The only difference between $\mathrm {G}_0$ and $\mathrm {G}_1$ is whether $\mathsf {IX},\mathsf {tk}_1,...\,,\mathsf {tk}_n$ are generated using $\mathsf {StI}$’s algorithms or $\mathcal {S}$. In both cases, $\mathbf {D}'$’s values are encrypted using $\mathsf {SE}\mathsf {.Enc}$. This is the same differentiation going on in the semantic security game so $\mathrm {G}^{\mathrm {ss}}_{{\mathsf {StI},\mathcal {L}^{\mathrm {i}},\mathcal {S}^{\mathrm {i}}}}(A_{\mathrm {i}})=\Pr [\mathrm {G}_1]-\Pr [\mathrm {G}_0]$. Similarly the difference between $\mathrm {G}_1$ and $\mathrm {G}_2$ is whether the values in $\mathbf {D}'$ are the output of $\mathsf {SE}\mathsf {.Enc}$ or random strings which is what is going on in the IND$-security game $\mathrm {G}^{\mathrm {ind\$}}_{\mathsf {SE}}(A_{\mathrm {s}})$, so $\mathbf {Adv}^{\mathrm {ind\$}}_{\mathsf {SE}}(A_{\mathrm {s}})=\Pr [\mathrm {G}_2]-\Pr [\mathrm {G}_1]$. Once again, the difference between $\mathrm {G}_2$ and $\mathrm {G}_3$ is whether the labels in $\mathbf {D}'$ (i.e. the tokens in $\mathsf {Dye}_{\pi }\mathsf {.Enc}$) are generated using $\mathsf {F}\mathsf {.Ev}$ or a random function which is what is going on in the PRF-security game $\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_{\mathrm {f}})$, so $\mathbf {Adv}^{\mathrm {prf}}_{\mathsf {F}}(A_{\mathrm {f}})=\Pr [\mathrm {G}_3]-\Pr [\mathrm {G}_2]$.

Combining all the above equations gives the desired bound on $\mathbf {Adv}^{\mathrm {ss}}_{\mathsf {StE},\mathcal {L},\mathcal {S}}(A)$.

CLeakage Profile and Security Proof for $\mathsf {PpSj}$

Fig. 14.

Simulator, adversaries and games used in the proof of Theorem 1.

Fig. 15.

Leakage profile (top) and simulator (bottom) for $\mathsf {PpSj}$. In $\mathcal {L}^{\mathrm {p}}$, $\mathsf {RS},\mathcal {L},\mathsf {HF},\mathsf {HQ}$ compute the recursion structure leakage, $\mathsf {Mme}$’s leakage profile, hashset filtering results and hashset query pattern respectively, as discussed in Sect. 4.2. In $\mathcal {S}^{\mathrm {p}}$, $\mathcal {S}$ is a simulator for $\mathsf {Mme}$.

Fig. 16.

Adversaries used in proof of Theorem 3. Note that when $A_{\mathrm {f}}$ is run it randomly selects one of $A_1,A_2$ and runs it.

Theorem 3

Let $\mathcal {L},\mathcal {S}$ be the leakage algorithm and simulator for $\mathsf {Mme}$ respectively. Let $\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}$ be as defined in Fig. 15 and let $\mathsf {F}$ be the function family used. Then for all adversaries $A$ there exists adversaries $A_{\mathrm {m}},A_{\mathrm {f}}$ such that:

$$\mathbf {Adv}^{\mathrm {ss}}_{\mathsf {PpSj},\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}}(A)\le \mathbf {Adv}^{\mathrm {ss}}_{\mathsf {Mme},\mathcal {L},\mathcal {S}}(A_{\mathrm {m}})+(p+1)\cdot \mathbf {Adv}^{\mathrm {prf}}_{\mathsf {F}}(A_{\mathrm {f}}).$$

Here, p is the number of distinct predicates used in constructing $\mathsf {HS}$.

Proof

Adversary $A_{\mathrm {m}}$ is given in Fig. 16. In the same diagram, we see $A_1,A_2$ which are both PRF adversaries playing $\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}$. We define $A_{\mathrm {f}}$ to randomly pick one at run time and use it.

Now we can proceed via a standard hybrid argument. Let $b_{\mathrm {p}},b_{\mathrm {f}},b_{\mathrm {m}}$ be the challenge bits in $\mathrm {G}^{\mathrm {ss}}_{{\mathsf {PpSj},\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}}}$, $\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}$ and $\mathrm {G}^{\mathrm {ss}}_{{\mathsf {Mme},\mathcal {L},\mathcal {S}}}$ respectively.

From the various advantage definitions, we have that $\mathbf {Adv}^{\mathrm {ss}}_{\mathsf {PpSj},\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}}(A)$ $=\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {PpSj},\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}}}(A)| b_{\mathrm {p}}=1]-\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {PpSj},\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}}}(A)|b_{\mathrm {p}}=0]$, $\mathbf {Adv}^{\mathrm {ss}}_{\mathsf {Mme},\mathcal {L},\mathcal {S}}(A_{\mathrm {m}})$ $=\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {Mme},\mathcal {L},\mathcal {S}}}(A_{\mathrm {m}})| b_{\mathrm {m}}=1]-\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {Mme},\mathcal {L},\mathcal {S}}}(A_{\mathrm {m}})| b_{\mathrm {m}}=0]$, and $\mathbf {Adv}^{\mathrm {prf}}_{\mathsf {F}}(A_1)$ $=\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_1)|b_{\mathrm {f}}=1]-\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_1)|b_{\mathrm {f}}=0]$. Notice also that $\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_2)|b_{\mathrm {f}}$ $=0,c=i]=\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_2)|b_{\mathrm {f}}=1,c=i+1]$ for $i\in [p-1]$ and $\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_2)|b_{\mathrm {f}}$ $=1,c=j]-\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_2)|b_{\mathrm {f}}=1,c=j]\le \mathbf {Adv}^{\mathrm {prf}}_{\mathsf {F}}(A_2)$ for $j\in [p]$. This means that

$$p\cdot \mathbf {Adv}^{\mathrm {prf}}_{\mathsf {F}}(A_2)\ge \Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_2)|b_{\mathrm {f}}=1,c=p]-\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_1)|b_{\mathrm {f}}=0,c=1].$$

Notice that $A_{\mathrm {m}}$ in $\mathrm {G}^{\mathrm {ss}}_{{\mathsf {Mme},\mathcal {L},\mathcal {S}}}$ uses the game to simulate multimap encryption and performs the rest itself as it happens in the “real world” of $\mathrm {G}^{\mathrm {ss}}_{{\mathsf {PpSj},\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}}}(A)$. This gives $\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {PpSj},\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}}}(A)| b_{\mathrm {p}}=1]=\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {Mme},\mathcal {L},\mathcal {S}}}(A_{\mathrm {m}})| b_{\mathrm {m}}=1]$. Similarly, $A_1$ simulates multimap encryption as in the “ideal world” of $\mathrm {G}^{\mathrm {ss}}_{{\mathsf {Mme},\mathcal {L},\mathcal {S}}}$ and defers the filtering key production to $\textsc {Fn}$ which gives us $\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {Mme},\mathcal {L},\mathcal {S}}}(A_{\mathrm {m}})| b_{\mathrm {m}}=0]=\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_1)|b_{\mathrm {f}}=1]$. When $A_2$ plays $\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_2)$, if $c=p$ then all the $K_i$ will be randomly selected. This means $\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_1)|b_{\mathrm {f}}=0]=\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_2)|b_{\mathrm {f}}=1,c=p]$. Over p hybrids, we get to the version where all the $\mathsf {F}\mathsf {.Ev}(K_i,\cdot )$ (where $K_i$ is not revealed to the adversary) are simulated with random functions, giving us $\Pr [\mathrm {G}^{\mathrm {prf}}_{\mathsf {F}}(A_1)|b_{\mathrm {f}}=0,c=1]=\Pr [\mathrm {G}^{\mathrm {ss}}_{{\mathsf {PpSj},\mathcal {L}^{\mathrm {p}},\mathcal {S}^{\mathrm {p}}}}(A)|b_{\mathrm {p}}=0]$ because this selects all of $\mathsf {HS}$ elements as $\mathcal {S}^{\mathrm {p}}$ does.