Boolean Searchable Symmetric Encryption with Worst-Case Sub-linear Complexity

Abstract

Recent work on searchable symmetric encryption (SSE) has focused on increasing its expressiveness. A notable example is the OXT construction (Cash et al., CRYPTO ’13) which is the first SSE scheme to support conjunctive keyword queries with sub-linear search complexity. While OXT efficiently supports disjunctive and boolean queries that can be expressed in searchable normal form, it can only handle arbitrary disjunctive and boolean queries in linear time. This motivates the problem of designing expressive SSE schemes with worst-case sub-linear search; that is, schemes that remain highly efficient for any keyword query.

In this work, we address this problem and propose non-interactive highly efficient SSE schemes that handle arbitrary disjunctive and boolean queries with worst-case sub-linear search and optimal communication complexity. Our main construction, called IEX, makes black-box use of an underlying single keyword SSE scheme which we can instantiate in various ways. Our first instantiation, IEX-2Lev, makes use of the recent 2Lev construction (Cash et al., NDSS ’14) and is optimized for search at the expense of storage overhead. Our second instantiation, IEX-ZMF, relies on a new single keyword SSE scheme we introduce called ZMF and is optimized for storage overhead at the expense of efficiency (while still achieving asymptotically sub-linear search). Our ZMF construction is the first adaptively-secure highly compact SSE scheme and may be of independent interest. At a very high level, it can be viewed as an encrypted version of a new Bloom filter variant we refer to as a Matryoshka filter. In addition, we show how to extend IEX to be dynamic and forward-secure. To evaluate the practicality of our schemes, we designed and implemented a new encrypted search framework called Clusion. Our experimental results demonstrate the practicality of IEX and of its instantiations with respect to either search (for IEX-2Lev) and storage overhead (for IEX-ZMF).

S. Kamara—Work done in part at Microsoft Research.

T. Moataz—Work done in part at IMT Atlantique and Colorado State.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

A structured encryption (STE) scheme encrypts a data structure in such a way that it can be privately queried. An STE scheme is secure if it reveals nothing about the structure and query beyond a well-specified and “reasonable” leakage profile [13, 15]. STE schemes come in two forms: response-revealing and response-hiding. The former reveals the query response in plaintext whereas the latter does not. An important special case of STE is searchable symmetric encryption (SSE) which encrypts search structures such as inverted indexes [10, 11, 13, 15, 23, 24] or search trees [19, 23]. Another example is graph encryption which encrypts various kinds of graphs [13, 27]. STE has received a lot of attention from Academia and Industry due to: (1) its potential applications to cloud storage and database security; and (2) the fact that, among a host of different encrypted search solutions (e.g., property-preserving encryption, fully-homomorphic encryption, oblivious RAM, functional encryption) it seems to provide the best tradeoffs between security and efficiency.

In recent years, much of the work on STE has focused on supporting more complex structures and queries. A notable example in the setting of SSE is the work of Cash et al. which proposed the first SSE scheme to support conjunctive queries in sub-linear time [11]. Their scheme, OXT, is also shown to support disjunctive and even boolean queries. Faber et al. later showed how to extend OXT to achieve even more complex queries including range, substring, wildcard and phrase queries. Another example is the BlindSeer project from Pappas et al. [30] and Fisch et al. [17] which present a solution that supports boolean and range queries as well as stemming in sub-linear time.

Naive Solutions. Any boolean query $\phi (w_1, \dots , w_q)$, where $w_1, \dots , w_q$ are keywords and $\phi$ is a boolean formula, can be handled using a single-keyword SSE scheme in a naive way. In the case of response-revealing schemes it suffices to search for each keyword and have the server take the intersection and unions of the result sets appropriately. The issue with this approach, of course, is that the server learns more information than necessary: namely, it learns the result sets ${{\mathsf {D}}}{{\mathsf {B}}}(w_1)$, ..., ${{\mathsf {D}}}{{\mathsf {B}}}(w_q)$ whereas it should only learn the set ${{\mathsf {D}}}{{\mathsf {B}}}(\phi (w_1, \dots , w_q))$. For response-hiding schemes, one can search for each keyword and compute the intersections and unions at the client. The problem with this approach is that the parties communicate more information than necessary: namely, the server sends elements within the intersections of the result sets multiple times. With this in mind, any boolean SSE solution should improve on one of the naive approaches depending on whether it is response-hiding or response-revealing.

Worst-Case Sub-linear Search Complexity. While OXT achieves sub-linear search complexity for conjunctive queries, its extension to disjunctive and arbitrary boolean queries does not. More precisely, OXT remains sub-linear only for queries in searchable normal form (SNF) which have the form $w_1 \wedge \phi (w_2, \dots , w_q)$, where $w_1$ through $w_q$ are keywords and $\phi$ is an arbitrary boolean formula. For non-SNF queries, OXT requires linear time in the number of documents. This motivates the following natural question: can we design SSE schemes that support arbitrary disjunctive and arbitrary boolean queries with sub-linear search complexity? In other words, can we design solutions for these queries that are efficient even in the worst-case?

1.1 Our Contributions and Techniques

In this work, we address this problem and propose efficient disjunctive and boolean SSE schemes with worst-case sub-linear search complexity and optimal communication overhead. Our schemes are non-interactive and, as far as we know, the first to achieve optimal communication complexity. To do this we make several contributions which we summarize below

Worst-Case Disjunctive Search. Our first solution, which we call IEX, is a worst-case sub-linear disjunctive SSE scheme. While it leaks more than the naive response-hiding solution, we stress that it achieves optimal communication complexity which, for response-hiding schemes, is the main tradeoff we seek. In addition, it leaks less than OXT (when used for disjunctive queries) while achieving worst-case efficiency.

The underlying idea behind IEX’s design is best expressed in set-theoretic terms where we view the result of a disjunctive query $w_1 \vee \cdots \vee w_q$ as the union of the results of each individual term. More precisely, if we denote by ${{\mathsf {D}}}{{\mathsf {B}}}(w)$ the set of document identifiers that contain the query w, then ${{\mathsf {D}}}{{\mathsf {B}}}(w_1 \vee \cdots \vee w_q) = {{\mathsf {D}}}{{\mathsf {B}}}(w_1) \cup \cdots \cup {{\mathsf {D}}}{{\mathsf {B}}}(w_q)$. Using the naive response-hiding approach, one could use a single-keyword response-hiding scheme to query each keyword and compute the union at the client but, as discussed above, this would incur poor communication complexity. Our approach is different and, intuitively speaking, makes use of the inclusion-exclusion principle as follows. Consider a three-term query $w_1 \vee w_2 \vee w_3$. Instead of searching for ${{\mathsf {D}}}{{\mathsf {B}}}(w_1)$, ${{\mathsf {D}}}{{\mathsf {B}}}(w_2)$, ${{\mathsf {D}}}{{\mathsf {B}}}(w_3)$ and computing the union, we compute ${{\mathsf {D}}}{{\mathsf {B}}}(w_1)$ and remove from it

$$\begin{aligned} {{\mathsf {D}}}{{\mathsf {B}}}(w_1) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_2) \quad \text {and} \quad {{\mathsf {D}}}{{\mathsf {B}}}(w_1) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_3). \end{aligned}$$

We then compute ${{\mathsf {D}}}{{\mathsf {B}}}(w_2)$ and remove from it ${{\mathsf {D}}}{{\mathsf {B}}}(w_2) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_3)$. Finally, we take the union of the remaining sets and add ${{\mathsf {D}}}{{\mathsf {B}}}(w_3)$. It follows by the inclusion-exclusion principle that this results in exactly ${{\mathsf {D}}}{{\mathsf {B}}}(w_1) \cup {{\mathsf {D}}}{{\mathsf {B}}}(w_2) \cup {{\mathsf {D}}}{{\mathsf {B}}}(w_3)$. If we could somehow support the intersection and removal operations at the server, then we could achieve optimal communication complexity. Note that this high-level approach is “purely disjunctive” in the sense that it does not rely on transforming the query into another form as done in OXT. The avoidance of SNF in particular is what enables us to achieve worst-case efficiency.

We stress that the intuition provided thus far is only a very high-level conceptual explanation of our approach and cannot be translated directly to work on encrypted data. The challenge is that no SSE scheme we are aware of directly supports the kind of set operations needed to implement this idea. Therefore, a major part of our contribution is in designing and analyzing such a scheme.

Boolean Search. While IEX is naturally disjunctive, we show that it also supports boolean queries. Similarly to the disjunctive case, we explain our high-level approach in set-theoretic terms. First, recall that any boolean query can be written in conjunctive normal form (CNF) so it has the form $\varDelta _1 \wedge \cdots \wedge \varDelta _\ell$, where each $\varDelta _i = w_{i,1} \vee \cdots \vee w_{i,q}$ is a disjunction. Given a response-hiding disjunctive-search scheme like IEX, a naive approach for CNF queries is to execute disjunctive searches for each disjunction $\varDelta _1, \dots , \varDelta _\ell$ and have the client perform the intersection of the results. This approach is problematic, however, because it requires more communication than necessary. To avoid this we take the following alternative approach. We note that the result ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1 \wedge \cdots \wedge \varDelta _\ell )$ is a subset of ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1)$ and that it can be computed by progressively keeping only the identifiers in ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1)$ that are also included in ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _2)$ through ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _\ell )$. Again, we stress that this description is only a high-level conceptual explanation of our approach and requires more work to instantiate over encrypted data.

The IEX Structure. As mentioned above, a major challenge in this work is the design of an encrypted structure that supports the set-theoretic operations needed to implement the strategies discussed above. To achieve this, IEX makes use of a more complex structure than the traditional encrypted inverted index. In particular, IEX combines several instantiations of two kinds of structures: dictionaries and multi-maps. A dictionary (i.e., a key-value store) maps labels to values whereas a multi-map (i.e., an inverted index) maps labels to tuples of values. More precisely, the IEX design consists of an encrypted global multi-map that maps every keyword w to its document identifiers ${{\mathsf {D}}}{{\mathsf {B}}}(w)$ and an encrypted dictionary that maps every keyword to a local multi-map for w. The local multi-map of a keyword w maps all the keywords v that co-occur with w to the identifiers of the documents that contain both v and w. At a high-level, with the encrypted global multi-map we can recover ${{\mathsf {D}}}{{\mathsf {B}}}(w_1)$. With the encrypted dictionary, we can recover the encrypted local multi-map for keywords $w_2$ through $w_\ell$. And, finally, by querying the (encrypted) local multi-map of a keyword w with a keyword v, we can recover the identifiers of the documents that contain both w and v. With these basic operations, we can then execute a full disjunctive query as discussed above.

Instantiations. IEX is an abstract construction that makes black-box use of encrypted multi-maps and dictionaries which, in turn, can be instantiated with several concrete constructions, e.g., [10, 13, 15, 23].¹ While its asymptotic complexity is not affected by how the building blocks are instantiated, its concrete efficiency is so we consider this choice carefully—especially how the local multi-maps are instantiated. We consider two instantiations. The first, IEX-2Lev, uses the 2Lev construction of Cash et al. [10] to encrypt the multi-maps (local and global). This particular instantiation is very efficient with respect to search time but produces large encrypted structures (e.g., 9.8 GB for datasets of 34M keyword/id pairs).

To address this we propose a second instantiation called IEX-ZMF which trades off efficiency for compactness. In fact, we show that IEX-ZMF is an order of magnitude more compact than IEX-2Lev (e.g., producing 0.9 GB EDBs for datasets with 34M keyword/id pairs). This compactness is achieved by encrypting IEX’s local multi-maps with a new construction called ZMF which may be of independent interest and that we detail below.²

The ZMF Scheme. ZMF is a multi-map encryption scheme that is inspired by and has similarities to the classic Z-IDX construction of Goh [19]. Its core design as well as its security are very different, however. While Z-IDX produces a collection of non-adaptively-secure fixed-size encrypted Bloom filters, ZMF produces a collection of adaptively-secure variable-sized encrypted Bloom filters. In addition, the hash functions used for each filter can all be derived from a fixed set of hash functions (even though the filters store a different number of elements). This last property is non-standard but is crucial for our approach to be practical as it allows us to generate constant-size tokens that can be used with every filter in the collection. We refer to such collections of Bloom filters as matryoshka filters and, as far as we know, they have not been considered in the past. As we detail in Sect. 7, encrypting matryoshka filters with adaptive security is quite challenging. For this, we rely on the random oracle model and on a non-standard use of online ciphers [4] which are streaming block ciphers in the sense that every ciphertext block depends only on the previous plaintext blocks. Note that like Z-IDX, ZMF has linear search time but we use it in our IEX construction only to encrypt the local multi-maps which guarantees that IEX-ZMF is still sub-linear.

Dynamism and Forward-Security. We extend IEX to be dynamic resulting in a new scheme DIEX. An important security property for dynamic SSE schemes is forward security which guarantees that updates to an encrypted structure cannot be correlated with previous queries. Forward security was introduced by Stefanov, Papamanthou and Shi [32] and recent work of Zhang, Katz and Papamanthou [34] has shown that it mitigates certain injection attacks on SSE schemes. One advantage of our DIEX construction is that it naturally inherits the forward-security of its underlying encrypted multi-maps and dictionaries. That is, if the underlying structures are forward-secure then so is DIEX.

Reduced Leakage. As we mentioned above, IEX leaks more than the naive response-hiding solution while achieving optimal communication complexity. We stress, however, that it leaks less than the naive response-revealing solution and than OXT. As an example, consider that if OXT is used to search for two conjunctions ${\mathbf {w}}= w_1 \wedge w_2$ and ${\mathbf {w}}' = w_3 \wedge w_2$ which share a common term, the server can recover the results for ${\mathbf {w}}'' = w_1 \wedge w_2 \wedge w_3$. In the case of disjunctions, OXT ’s leakage is equivalent to the naive response-revealing solution.

Experiments. To evaluate the efficiency of IEX and its instantiations we designed and built a new encrypted search framework called Clusion [22]. It is written in Java and leverages the Apache Lucene search library [1]. It also includes a Hadoop-based distributed parser and indexer we implemented to handle massive datasets. Our experiments show that IEX—specifically our IEX-2Lev instantiation—is very efficient and even achieves faster search times than those reported for a C++ implementation of OXT [11] on a comparable system. For example, for conjunctive, disjunctive and boolean queries with selectivity on the order of thousands, IEX-2Lev takes 12, 14.8 and 23.7 ms, respectively. For the same conjunctive query, OXT is reported to take 200 ms on a comparable system. Clearly, a C/C++ implementation of IEX would perform even better.

We also implemented IEX-ZMF to evaluate its efficiency and compactness. In our experiments, it produced EDBs of size 198 MB and 0.9 GB from datasets with 1.5M and 34M keyword/id pairs, respectively. This is highly compact in comparison to IEX-2Lev which produced 1.6 GB, 9.8 GB EDBs for 1.5M and 34M keyword/id pairs, respectively. We also evaluated the efficiency of IEX-ZMF and, as expected, its performance for setup, search and token size are worse than IEX-2Lev. For example, for a dataset with 34M keyword/id pairs, $\mathsf {EDB}$ setup takes 7.58 h to process compared to 31 mins for IEX-2Lev.

On a boolean query of the form $(w \vee x) \wedge (y \vee z)$, where the disjunctions had selectivity 2 K and 10 K, respectively, IEX-ZMF took 1610 ms whereas IEX-2Lev took only 23.7 ms. As expected due to its high degree of compactness, IEX-ZMF is slower than IEX-2Lev (this is the exact tradeoff we seek).

2 Related Work

SSE was first considered by Song, Wagner and Perrig [31]. Curtmola, Garay, Kamara and Ostrovsky [15] introduced the notion of adaptive-security for SSE and presented the first constructions that achieved optimal search time with a space-efficient index. STE was introduced by Chase and Kamara [13] who proposed constructions for two-dimensional arrays, graphs and web graphs.

In [19], Goh introduced the Z-IDX construction which has linear search complexity and produces highly compact indexes due to its use of Bloom filters. Here, we extract a general transformation implicitly used in the Z-IDX construction and use it in part to construct our ZMF scheme. Kamara, Papamanthou and Roeder gave the first optimal-time dynamic SSE scheme [24]. Cash et al. [11] proposed OXT; the first optimal-time conjunctive keyword search scheme. Faber et al. [16] extend OXT to handle range, substring, wildcard and phrase queries. Pappas et al. [30] and Fisch et al. [17] present solutions based on garbled circuits and Bloom filters that can support boolean formulas, ranges and stemming. In [30], the authors show how to build the first worst-case sub-linear time boolean encrypted search solution. Like Goh’s Z-IDX construction and our ZMF scheme, the solution makes use of Bloom filters. In addition, it is the first adaptively-secure construction based on Bloom filters. For a disjunctive query ${\mathbf {w}}$, the scheme has search complexity $O(\log (n)\cdot C \cdot {{\mathsf {D}}}{{\mathsf {B}}}({\mathbf {w}}))$, where n is the number of documents and C is the cost of a 2-party secure function evaluation of a function that takes as input a Bloom filter of size $O(\#W )$ (i.e., the number of unique keywords in ${{\mathsf {D}}}{{\mathsf {B}}}$) and a q-term disjunctive query. We note that unlike IEX and OXT, it does not achieve optimal communication complexity. Also, while its search is sub-linear it involves multiple rounds of interactions.

Ishai, Kushilevitz, Lu and Ostrovsky propose a two-server SSE scheme that hides the access pattern and supports various complex queries including ranges, stemming and substring [21]. Cash et al. [10] design several I/O-efficient SSE schemes including the 2Lev construction which we use in one of our IEX instantiations. Kurosawa and Ohtaki [26] designed the first UC secure SSE scheme. Kurosawa [25] designed a linear-time construction that handles arbitrary boolean queries while not disclosing the structure of the boolean query itself. Forward Secrecy was first considered by Stefanov, Papamanthou and Shi [32]. In [9], Bost introduced an efficient forward secure construction. In [12], Cash and Tessaro give lower bounds on the locality of SSE by showing tradeoffs between locality, space overhead and read efficiency. Recently, Asharov, Naor, Segev and Shahaf gave SSE constructions with optimal locality, optimal space overhead and nearly-optimal read efficiency [3]. Encrypted search can also be achieved with other primitives like property-preserving encryption [5, 6], functional encryption [7, 8, 29], oblivious RAM [20], full-homomorphic encryption [18] and multi-party computation [33].

Online ciphers were introduced by Bellare, Boldyreva et al. [4], where they propose several schemes including the HCB1 construction which we make use of in our ZMF implementation. More efficient constructions were later proposed by Andreeva et al. [2].

3 Preliminaries

Notation. The set of all binary strings of length n is denoted as $\{0,1\}^n$, and the set of all finite binary strings as $\{0,1\}^*$. [n] is the set of integers $\{1, \dots , n\}$, and $2^{[n]}$ is the corresponding power set. We write $x \leftarrow \chi$ to represent an element x being sampled from a distribution $\chi$, and $x \mathop {\leftarrow }\limits ^{\$}X$ to represent an element x being sampled uniformly at random from a set X. The output x of an algorithm $\mathcal {A}$ is denoted by $x \leftarrow \mathcal {A}$. Given a sequence ${\mathbf {v}}$ of n elements, we refer to its ith element as $v_i$ or ${\mathbf {v}}[i]$. If S is a set then $\#S$ refers to its cardinality. If s is a string then |s| refers to its bit length and $s_i$ to its ith bit. $s^{|n}$ denotes the string s padded with $n - |s|$ 0’s and $s_{|n}$ represents the first n bits of s. Given strings s and r, we refer to their concatenation as either $\langle s, r\rangle$ or $s \Vert r$. For an n-bit string s and for all nonnegative d, we denote by $s^{\Vert d}$ the string $\langle s_1^{|d}, \cdots , s_{n}^{|d} \rangle$. In this work, padding takes precedence over truncation; that is, $s^{\Vert d}_{|p} = (s^{\Vert d})_{|p}$.

Data Types. An abstract data type is a collection of objects together with a set of operations defined on those objects. Examples include sets, dictionaries (also known as key-value stores or associative arrays) and graphs. The operations associated with an abstract data type fall into one of two categories: query operations, which return information about the objects; and update operations, which modify the objects. If the abstract data type supports only query operations it is static, otherwise it is dynamic.

Data Structures. A data structure for a given data type is a representation in some computational model³ of an object of the given type. Typically, the representation is optimized to support the type’s query operation as efficiently as possible. For data types that support multiple queries, the representation is often optimized to efficiently support as many queries as possible. As a concrete example, the dictionary type can be represented using various data structures depending on which queries one wants to support efficiently. Hash tables support $\mathsf{Get}$ and $\mathsf{Put}$ in expected O(1) time whereas balanced binary search trees support both operations in worst-case $\log (n)$ time. For ease of understanding and to match colloquial usage, we will sometimes blur the distinction between data types and structures. So, for example, when referring to a dictionary structure or a multi-map structure what we are referring to is an unspecified instantiation of the dictionary or multi-map data type.

Basic Structures. We make use of several basic data types including arrays, dictionaries and multi-maps which we recall here. An array $\mathsf {A}$ of capacity n stores n items at locations 1 through n and supports read and write operations. We write $v = \mathsf {A}[i]$ to denote reading the item at location i and $\mathsf {A}[i] = v$ the operation of storing an item at location i. A dictionary ${\mathsf {DX}}$ of capacity n is a collection of n label/value pairs $\{(\ell _i, v_i)\}_{i\le n}$ and supports $\mathsf{Get}$ and $\mathsf{Put}$ operations. We write $v_i = {\mathsf {DX}}[\ell _i]$ to denote getting the value associated with label $\ell _i$ and ${\mathsf {DX}}[\ell _i] = v_i$ to denote the operation of associating the value $v_i$ in ${\mathsf {DX}}$ with label $\ell _i$. A multi-map $\mathsf {MM}$ with capacity n is a collection of n label/tuple pairs $\{(\ell _i, V_i)_i\}_{i\le n}$ that supports $\mathsf{Get}$ and $\mathsf{Put}$ operations. Similarly to dictionaries, we write $V_i = \mathsf {MM}[\ell _i]$ to denote getting the tuple associated with label $\ell _i$ and $\mathsf {MM}[\ell _i] = V_i$ to denote operation of associating the tuple $V_i$ to label $\ell _i$. We sometimes write $\mathsf {MM}^{-1}[v]$ to refer to the set of labels in $\mathsf {MM}$ associated with tuples that include the value v. Multi-maps are the abstract data type instantiated by an inverted index. In the encrypted search literature multi-maps are sometimes referred to as indexes, databases or tuple-sets (T-sets) [10, 11].

Document Collections. A document collection is a set of documents ${\mathbf {D}}= (D_1, \dots ,D_n)$, each document consisting of a set of keywords from some universe $W$. We assume the universe of keywords is totally ordered (e.g., using lexicographic order) and denote by $W [i]$ the ith keyword in $W$. We assume every document has an identifier that is independent of its contents and denote it $\mathsf{id}(D_i)$. We assume the existence of an efficient indexing algorithm that takes as input a data collection ${\mathbf {D}}$ and outputs a multi-map that maps every keyword w in $W$ to the identifiers of the documents that contain w. In previous work, this multi-map is referred to as an inverted index or as a database. For consistency, we refer to any multi-map derived in this way from a document collection as a database and denote it ${{\mathsf {D}}}{{\mathsf {B}}}$. Given a keyword w, we denote by ${{\mathsf {c}}}{{\mathsf {o}}}_{{\mathsf {D}}}{{\mathsf {B}}}(w) \subseteq W$ the set of keywords in $W$ that co-occur with w; that is, the keywords that are contained in documents that contain w. When ${{\mathsf {D}}}{{\mathsf {B}}}$ is clear from the context we omit ${{\mathsf {D}}}{{\mathsf {B}}}$ and write only ${{\mathsf {c}}}{{\mathsf {o}}}(w)$.

3.1 Cryptographic Primitives

Basic Cryptographic Primitives. A private-key encryption scheme is a set of three polynomial-time algorithms $\mathsf {SKE}= (\mathsf{Gen}, {\mathsf {Enc}}, {\mathsf {Dec}})$ such that $\mathsf{Gen}$ is a probabilistic algorithm that takes a security parameter k and returns a secret key $K$; ${\mathsf {Enc}}$ is a probabilistic algorithm takes a key $K$ and a message m and returns a ciphertext c; ${\mathsf {Dec}}$ is a deterministic algorithm that takes a key $K$ and a ciphertext c and returns m if $K$ was the key under which c was produced. Informally, a private-key encryption scheme is secure against chosen-plaintext attacks (CPA) if the ciphertexts it outputs do not reveal any partial information about the plaintext even to an adversary that can adaptively query an encryption oracle. We say a scheme is random-ciphertext-secure against chosen-plaintext attacks (RCPA) if the ciphertexts it outputs are computationally indistinguishable from random even to an adversary that can adaptively query an encryption oracle.⁴ In addition to encryption schemes, we also make use of pseudo-random functions (PRF) and permutations (PRP), which are polynomial-time computable functions that cannot be distinguished from random functions by any probabilistic polynomial-time adversary.

Online Ciphers. An online cipher (OC) is a block cipher that can encrypt data streams. In particular, with an OC the encryption of the ith block in a stream depends only on the 1st through ith message blocks. OCs were introduced by Bellare, Boldyreva, Knudsen and Namprempre [4]. More formally, we say that a cipher $\mathsf{OC}: \{0,1\}^{k} \times \{0,1\}^{n\times B} \rightarrow \{0,1\}^{n\times B}$, where $B>1$ is the block length, is B-online if there exists a function $X: \{0,1\}^{k} \times \{0,1\}^{n\times B} \rightarrow \{0,1\}^B$ such that for any $\mathbf{m } \in \{0,1\}^{n\times B}$,

$$\begin{aligned} \mathsf{OC}_K(\mathbf{m }) = \mathsf{OC}^1_K(\mathbf{m }) \Vert \dots \Vert \mathsf{OC}^n_K(\mathbf{m }), \end{aligned}$$

where $\mathsf{OC}^i_K(\mathbf{m }) = X(K, m_1, \dots , m_i)$ for all $i \in [n]$ and where $m_i$ is the ith block of $\mathbf{m }$. OCs cannot be pseudo-random permutations (see [4] for a simple distinguisher) but can satisfy the weaker requirement of being computationally indistinguishable from a random online permutation. An online permutation is simply a permutation on a domain $\{0,1\}^{n\times B}$ whose ith block depends only on the first i blocks of its input. We denote by $\mathsf {OPerm} _{n, B}$ the set of all online permutations over $\{0,1\}^{n\times B}$. Security for an online cipher $\mathsf{OC}: \{0,1\}^k \times \{0,1\}^{n\times B} \rightarrow \{0,1\}^{n \times B}$ then holds if for all $\textsc {ppt}$ adversaries $\mathcal {A}$,

$$\begin{aligned} \left| {\Pr \left[ \,{\mathcal {A}^{\mathsf{OC}_K(\cdot )} = 1: K\mathop {\leftarrow }\limits ^{\$}\{0,1\}^k}\,\right] } - {\Pr \left[ \,{\mathcal {A}^{f(\cdot )} = 1: f \mathop {\leftarrow }\limits ^{\$}\mathsf {OPerm} _{n, B}}\,\right] } \right| \le \mathsf {negl}(k). \end{aligned}$$

4 Definitions

Structured encryption schemes encrypt data structures in such a way that they can be privately queried. There are several natural forms of structured encryption. The original definition of [13] considered schemes that encrypt both a structure and a set of associated data items (e.g., documents, emails, user profiles etc.). In [14], the authors also describe structure-only schemes which only encrypt structures. Another distinction can be made between interactive and non-interactive schemes. Interactive schemes produce encrypted structures that are queried through an interactive two-party protocol, whereas non-interactive schemes produce structures that can be queried by sending a single message, i.e., the token. One can also distinguish between response-hiding and response-revealing schemes: the former reveal the response to queries whereas the latter do not.

STE schemes are used as follows. During a setup phase, the client constructs an encrypted data structure $\mathsf {EDS}$ under a key $K$. The client then sends $\mathsf {EDS}$ to the server. During the query phase, the client constructs and sends a token $\mathsf {tk}$ generated from its query q and the key $K$. The server then uses the token $\mathsf {tk}$ to query $\mathsf {EDS}$. If the scheme is response-revealing, it recovers a response r. On the other hand, if the scheme is response-hiding it recovers a message that it returns to the client who in turn decrypts it with a resolving algorithm.

Definition 1

(Structured encryption). A single-round response-hiding structured encryption scheme $\varSigma _\mathscr {T}= (\mathsf{Setup}, {\mathsf {Token}}, \mathsf{Query}, \mathsf{Resolve})$ for data type $\mathscr {T}$ consists of four polynomial-time algorithms that work as follows:

• $(K, \mathsf {EDS}) \leftarrow \mathsf{Setup}(1^k, \mathsf {DS})$: is a probabilistic algorithm that takes as input a security parameter $1^k$ and a structure $\mathsf {DS}$ of type $\mathscr {T}$ and outputs a secret key $K$ and an encrypted structure $\mathsf {EDS}$.

• $\mathsf {tk}\leftarrow {\mathsf {Token}}(K, q)$: is a (possibly) probabilistic algorithm that takes as input a secret key $K$ and a query q and returns a token $\mathsf {tk}$.

• $c \leftarrow \mathsf{Query}(\mathsf {EDS}, \mathsf {tk})$: is a (possibly) probabilistic algorithm that takes as input an encrypted structure $\mathsf {EDS}$ and a token $\mathsf {tk}$ and outputs a message c.

• $r \leftarrow \mathsf{Resolve}(K, c)$: is a deterministic algorithm that takes as input a secret key $K$ and a message c and outputs a response r.

We say that a structured encryption scheme $\varSigma$ is correct if for all $k \in \mathbb {N}$, for all ${\mathsf {poly}}(k)$-size structures $\mathsf {DS}$ of type $\mathscr {T}$, for all $(K, \mathsf {EDS})$ output by $\mathsf{Setup}(1^k, {{\mathsf {D}}}{{\mathsf {S}}})$ and all sequences of $m = {\mathsf {poly}}(k)$ queries $q_1, \dots , q_m$, for all tokens $\mathsf {tk}_i$ output by ${\mathsf {Token}}(K, q_i)$, for all messages c output by $\mathsf{Query}(\mathsf {EDS}, \mathsf {tk}_i)$, $\mathsf{Resolve}(K, c)$ returns the correct response with all but negligible probability. The syntax of a response-revealing STE scheme can be recovered by omitting the $\mathsf{Resolve}$ algorithm and having $\mathsf{Query}$ output the response.

Security. The standard notion of security for STE guarantees that an encrypted structure reveals no information about its underlying structure beyond the setup leakage $\mathcal {L}_\mathsf {S}$, and that the query algorithm reveals no information about the structure and the queries beyond the query leakage $\mathcal {L}_\mathsf {Q}$. If this holds for non-adaptively chosen operations then this is referred to as non-adaptive security. If, on the other hand, the operations are chosen adaptively, this leads to the stronger notion of adaptive security [15]. This notion of security was first formalized by Curtmola et al. in the context of searchable encryption [15] and later generalized to structured encryption in [13].

Definition 2

(Adaptive security [13, 15]). Let $\varSigma _\mathscr {T}= (\mathsf{Setup}, {\mathsf {Token}}, \mathsf{Query})$ be a structured encryption scheme for type $\mathscr {T}$ and consider the following probabilistic experiments where $\mathcal {A}$ is a stateful adversary, $\mathcal {S}$ is a stateful simulator, $\mathcal {L}_\mathsf {S}$ and $\mathcal {L}_\mathsf {Q}$ are leakage profiles and $z \in \{0,1\}^*$:

• ${\mathbf {Real}}_{\varSigma , \mathcal {A}}(k)$: given z the adversary $\mathcal {A}$ outputs a structure $\mathsf {DS}$ of type $\mathscr {T}$ and receives $\mathsf {EDS}$ from the challenger, where $(K, \mathsf {EDS}) \leftarrow \mathsf{Setup}(1^k, \mathsf {DS})$. The adversary then adaptively chooses a polynomial number of queries $q_1,\dots , q_m$. For all $i \in [m]$, the adversary receives $\mathsf {tk}_i \leftarrow {\mathsf {Token}}(K, q_i)$. Finally, $\mathcal {A}$ outputs a bit b that is output by the experiment.

• ${\mathbf {Ideal}}_{\varSigma , \mathcal {A}, \mathcal {S}}(k)$: given z the adversary $\mathcal {A}$ generates a structure $\mathsf {DS}$ of type $\mathscr {T}$ which it sends to the challenger. Given z and leakage $\mathcal {L}_\mathsf {S}(\mathsf {DS})$ from the challenger, the simulator $\mathcal {S}$ returns an encrypted data structure $\mathsf {EDS}$ to $\mathcal {A}$. The adversary then adaptively chooses a polynomial number of operations $q_1, \dots , q_m$. For all $i \in [m]$, the simulator receives query leakage $\mathcal {L}_\mathsf {Q}(\mathsf {DS}, q_i)$ and returns a token $\mathsf {tk}_i$ to $\mathcal {A}$. Finally, $\mathcal {A}$ outputs a bit b that is output by the experiment.

We say that $\varSigma$ is adaptively $(\mathcal {L}_\mathsf {S}, \mathcal {L}_\mathsf {Q})$-secure if for all ppt adversaries $\mathcal {A}$, there exists a ppt simulator $\mathcal {S}$ such that for all $z \in \{0,1\}^*$,

$$\begin{aligned} |{\Pr \left[ \,{{\mathbf {Real}}_{\varSigma , \mathcal {A}}(k) = 1}\,\right] } - {\Pr \left[ \,{{\mathbf {Ideal}}_{\varSigma , \mathcal {A}, \mathcal {S}}(k) = 1}\,\right] }| \le \mathsf {negl}(k). \end{aligned}$$

5 IEX: A Worst-Case Sub-linear Disjunctive SSE Scheme

Our main construction, IEX, makes black-box use of a dictionary encryption scheme $\varSigma _{\mathsf {DX}}= (\mathsf{Setup}, {\mathsf {Token}}, \mathsf{Get})$, a multi-map encryption scheme $\varSigma _\mathsf {MM}= (\mathsf{Setup}, {\mathsf {Token}}, \mathsf{Get})$, a pseudo-random function F, and of a private-key encryption scheme $\mathsf {SKE}= (\mathsf{Gen}, {\mathsf {Enc}}, {\mathsf {Dec}})$. The details of the scheme are provided in Fig. 1. At a high-level, it works as follows.

Setup. The $\mathsf{Setup}$ algorithm takes as input a security parameter k and an index ${{\mathsf {D}}}{{\mathsf {B}}}$. It makes use of two data structures: a dictionary ${\mathsf {DX}}$ and a global multi-map $\mathsf {MM}_g$. $\mathsf {MM}_g$ maps every keyword in $w \in W$ to an encryption of the identifiers in ${{\mathsf {D}}}{{\mathsf {B}}}(w)$. We refer to these encryptions as tags and they are computed by evaluating $\mathsf {SKE}.{\mathsf {Enc}}$ using as coins the evaluation of F on keyword w and the identifier. The global multi-map $\mathsf {MM}_g$ is then encrypted using $\varSigma _\mathsf {MM}$, resulting in $\mathsf {EMM}_g$.

For each keyword $w \in W$, the algorithm creates a local multi-map $\mathsf {MM}_w$, that maps the keywords $v \in {{\mathsf {c}}}{{\mathsf {o}}}(w)$ to tags of identifiers in ${{\mathsf {D}}}{{\mathsf {B}}}(v)\cap {{\mathsf {D}}}{{\mathsf {B}}}(w)$. Intuitively, the purpose of the local multi-map $\mathsf {MM}_w$ is to quickly find out which documents contain both w and v, for any $v \ne w$. The local multi-maps $\mathsf {MM}_w$ are then encrypted with $\varSigma _\mathsf {MM}$. This results in encrypted multi-maps $\mathsf {EMM}_w$ which are then stored in the dictionary ${\mathsf {DX}}$ such that ${\mathsf {DX}}[w] = \mathsf {EMM}_w$. In other words, it stores label/value pairs $(w, \mathsf {MM}_w)$ in ${\mathsf {DX}}$. Finally, ${\mathsf {DX}}$ is encrypted with $\varSigma _{\mathsf {DX}}$, resulting in an encrypted dictionary $\mathsf {EDX}$. The output of $\mathsf{Setup}$ includes the encrypted structures $\big (\mathsf {EDX}, \mathsf {EMM}_g\big )$ as well as their keys.

There are several optimizations possible for $\mathsf{Setup}$ that we omit in our formal description for ease of exposition. The first is that the encrypted local multi-maps can be stored “by reference” in the encrypted dictionary $\mathsf {EDX}$ instead of “by value”. More precisely, instead of storing the actual encrypted local multi-maps $\mathsf {EMM}_w$ in $\mathsf {EDX}$ one can just store a pointer to them. Another optimization is that, depending on how $\varSigma _\mathsf {MM}$ is designed, the keys for the local encrypted multi-maps could all be generated from a single key using a PRF (with a counter). This would reduce the size of $K$. This optimization can be easily applied to most known encrypted multi-map schemes including the ones from [10, 11, 13, 15, 23].

Token. The ${\mathsf {Token}}$ algorithm takes as input a key and a vector of keywords ${\mathbf {w}}= (w_1, \dots , w_q)$. For all $i \in [q-1]$ it creates a “sub-token” $\mathbf {tk}_i = (\mathsf {dtk}_i, \mathsf {gtk}_i, \mathsf {ltk}_{i+1},\dots ,\mathsf {ltk}_q)$ composed of a dictionary token $\mathsf {dtk}_i$, a global token $\mathsf {gtk}_i$ for $w_i$ and, for all keywords $w_{i+1}$ through $w_q$ in the disjunction, a local token $\mathsf {ltk}_j$ for $w_j$, with $i+1 \le j \le q$. Intuitively, the global token will allow the server to query the encrypted global multi-map $\mathsf {EMM}_g$ to recover tags of the ids in ${{\mathsf {D}}}{{\mathsf {B}}}(w_i)$. The dictionary token for $w_i$ will then allow the server to query the encrypted dictionary $\mathsf {EDX}$ to recover $w_i$’s local multi-map $\mathsf {EMM}_i$. Finally, the local tokens will allow the server to query $w_i$’s encrypted local multi-map $\mathsf {EMM}_i$ to recover the tags of the ids of the documents that contain both $w_i$ and $w_{i+1}$, $w_i$ and $w_{i+2}$, etc. As we will see next, this information will be enough for the server to find the relevant documents. For the last keyword $w_q$ in the disjunction, the algorithm only needs to create a global token.

Search. The ${\mathsf {Search}}$ algorithm takes as input $\mathsf {EDB}= (\mathsf {EDX}, \mathsf {EMM}_g)$ and a token $\mathsf {tk}= (\mathbf {tk}_1, \dots , \mathbf {tk}_{q-1},\mathsf {gtk}_q)$. For each sub-token $\mathbf {tk}_i = (\mathsf {dtk}_i, \mathsf {gtk}_i, \mathsf {ltk}_{i+1}, \dots , \mathsf {ltk}_q)$, the server does the following. It first uses $\mathsf {gtk}_i$ to query the global multi-map $\mathsf {EMM}_g$ and recover a set of identifier tags $T_i$ for ${{\mathsf {D}}}{{\mathsf {B}}}(w_i)$. It then uses $\mathsf {dtk}_i$ to query the encrypted dictionary $\mathsf {EDX}$ to recover the local multi-map $\mathsf {EMM}_i$ for $w_i$ and uses $\mathsf {ltk}_{i+1}$ to query $\mathsf {EMM}_i$ to recover the tags $T'$ for identifiers of the documents that contain both $w_i$ and $w_{i+1}$; that is, the tags for the set $I' = {{\mathsf {D}}}{{\mathsf {B}}}(w_i) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_{i+1})$. The server then removes $T_i'$ from $T_i$. It then repeats this process for all local tokens $\mathsf {ltk}_{i+2}$ to $\mathsf {ltk}_q$. Once it finishes processing all local tokens in $\mathbf {tk}_i$, it holds the set of tags for the set

$$\begin{aligned} {{\mathsf {D}}}{{\mathsf {B}}}\big (w_i\big ) \setminus \bigcup _{j = i}^{q-1} \bigg ({{\mathsf {D}}}{{\mathsf {B}}}\big (w_i\big ) \bigcap {{\mathsf {D}}}{{\mathsf {B}}}\big (w_{j+1}\big )\bigg ). \end{aligned}$$

(1)

Once it finishes processing all the sub-tokens, the server holds tags $T_1$ through $T_{q-1}$. For $\mathsf {gtk}_q$, the server just queries the global multi-map to recover $T_q$. Finally, it outputs the set

$$\begin{aligned} T = \bigcup _{i=1}^q T_i. \end{aligned}$$

(2)

Fig. 1.

Our disjunctive SSE scheme IEX.

5.1 Correctness and Efficiency

We now analyze the correctness and efficiency of our construction. The correctness of IEX follows from Eqs. (1) and (2) and from the inclusion-exclusion principle. Given a disjunctive query ${\mathbf {w}}= (w_1, \dots , w_q)$, by Eq. (2), IEX.${\mathsf {Search}}\big (\mathsf {EDB},{\mathsf {Token}}(K, {\mathbf {w}})\big )$ will output

$$\begin{aligned} T \nonumber&= \bigcup _{i=1}^q T_i \\ \nonumber&= \bigg (\bigcup _{i=1}^{q-1}T_i\bigg ) \bigcup T_q \\ \nonumber&= \left( \bigcup _{i=1}^{q-1} \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_i) \setminus \bigcup _{j=i}^{q-1} \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_i) \bigcap {{\mathsf {D}}}{{\mathsf {B}}}(w_{j+1})\right) \right) \right) \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q) \\ \nonumber&= \left( \bigcup _{i=1}^{q-2} \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_i) \setminus \bigcup _{j=i}^{q-1} \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_i) \bigcap {{\mathsf {D}}}{{\mathsf {B}}}(w_{j+1})\right) \right) \right) \\ \nonumber&\qquad \qquad \bigcup \underbrace{\left( {{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \setminus \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \bigcap {{\mathsf {D}}}{{\mathsf {B}}}(w_{q})\right) \right) \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q)}_{U} \\ \end{aligned}$$

(3)

where the first and third equalities hold by Eqs. (2) and (1), respectively. Note, however, that U equals ${{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q)$:

$$\begin{aligned} U&= {{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \bigcap \left( \overline{{{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1})} \bigcup \overline{{{\mathsf {D}}}{{\mathsf {B}}}(w_{q})} \right) \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q)\\&= \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \bigcap \overline{{{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1})}\right) \bigcup \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \bigcap \overline{{{\mathsf {D}}}{{\mathsf {B}}}(w_{q})} \right) \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q)\\&= \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \bigcap \overline{{{\mathsf {D}}}{{\mathsf {B}}}(w_{q})} \right) \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q)\\&= \left( {{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q) \right) \bigcap \left( \overline{{{\mathsf {D}}}{{\mathsf {B}}}(w_{q})} \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q)\right) \\&= {{\mathsf {D}}}{{\mathsf {B}}}(w_{q-1}) \bigcup {{\mathsf {D}}}{{\mathsf {B}}}(w_q) \end{aligned}$$

Repeating the same argument for $q-2$, $q-3$ and so on and plugging into Eq. (3), we get that $T = \bigcup _{i=1}^q {{\mathsf {D}}}{{\mathsf {B}}}(w_i)$.

Efficiency. The search complexity of IEX is $O(q^2 \cdot M)$, where $M = \max _{i \in [q]} \#{{\mathsf {D}}}{{\mathsf {B}}}(w_i)$ and q is the number of terms in the disjunction. Tokens are of size O(q). We also note that unlike BXT and OXT [11], IEX tokens are selectivity-independent in the sense that they do not depend on the size of the result. The IEX storage complexity is,

$$\begin{aligned} O\bigg ( \mathsf {strg}\bigg (\sum _w \#{{\mathsf {D}}}{{\mathsf {B}}}(w)\bigg ) + \sum _w \mathsf {strg}\bigg (\sum _{v\in {{\mathsf {c}}}{{\mathsf {o}}}(w)} \#{{\mathsf {D}}}{{\mathsf {B}}}(v) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w) \bigg ) \bigg ), \end{aligned}$$

where $\mathsf {strg}$ is the storage complexity of the underlying encrypted multi-map encryption scheme $\varSigma _\mathsf {MM}$.

A Storage Optimization. As we can see, the storage complexity of IEX can be large, especially if the underlying encrypted multi-maps are. This is indeed the case when they are instantiated with standard sub-linear constructions. We observe, however, that we can tradeoff storage complexity (and setup time) for the communication complexity of search as follows. When constructing a local multi-map $\mathsf {EMM}_w$ for a keyword w, we normally insert tags for the identifiers in ${{\mathsf {D}}}{{\mathsf {B}}}(w) \cap {{\mathsf {D}}}{{\mathsf {B}}}(v)$ for all $v \in {{\mathsf {c}}}{{\mathsf {o}}}(w)$. This is not necessary for correctness, however, so we can omit some of the co-occurring keywords from w’s local multi-map. The tradeoff is that this will increase the communication complexity of IEX’s search operation and, in particular, make it non-optimal.

To do this, we suggest using the following approach to decide whether to add a keyword $v \in {{\mathsf {c}}}{{\mathsf {o}}}(w)$ or not. Let $p < 1$ be a filtering parameter and let

$$\begin{aligned} T_{w, v} \mathop {=}\limits ^{def}\frac{\#{{\mathsf {D}}}{{\mathsf {B}}}(v) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w)}{\max (\#{{\mathsf {D}}}{{\mathsf {B}}}(w),\#{{\mathsf {D}}}{{\mathsf {B}}}(v))}. \end{aligned}$$

If $T_{w, v} > p$, then add v to $\mathsf {EMM}_w$ otherwise do not. With this filtering in place, the storage complexity of IEX is now

$$\begin{aligned} O\bigg ( \mathsf {strg}\bigg (\sum _w \#{{\mathsf {D}}}{{\mathsf {B}}}(w)\bigg ) + \sum _w \mathsf {strg}\bigg (\sum _{\begin{array}{c} v\in {{\mathsf {c}}}{{\mathsf {o}}}(w) \\ T_{w,v}>p \end{array}} \#{{\mathsf {D}}}{{\mathsf {B}}}(v) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w) \bigg ) \bigg ). \end{aligned}$$

In our experiments we set $p = 0.2$.

Remark. We note that when all the terms of the disjunctive query have selectivity O(n), IEX has linear search complexity. This is, however, the best one can do. On the other hand, the communication complexity of IEX remains optimal independently of the selectivity of the terms. This similarly applies to OXT but not to BlindSeer since it induces a logarithmic (multiplicative) overhead.

5.2 Security

The setup leakage of IEX consists of the setup leakage of its underlying building blocks. In particular, this includes the setup leakage of the encrypted global multi-map and of the encrypted dictionary. Assuming the use of standard optimal-time multi-map and dictionary encryption schemes [10, 13, 15, 23], this reveals the size of the database ${{\mathsf {D}}}{{\mathsf {B}}}$ as well as the total size of the local multi-maps stored in the dictionary. The query leakage of IEX for a query ${\mathbf {w}}$ includes, for each keyword $w_i \in W$, the query leakage of the encrypted dictionary and of the encrypted global multi-map. It also includes the query leakage of every queried local multi-map as well as their setup leakage. Again if instantiated with standard constructions, this will consist of the search and access patterns which, respectively, capture whether or not the same query has been searched for and (in our case) the tags. Finally, the query leakage also includes the number of documents containing ${{\mathsf {D}}}{{\mathsf {B}}}(w_i) \bigcap {{\mathsf {D}}}{{\mathsf {B}}}(w_{j+1})$, for all $j\ge i$ and $i \in [q-1]$.

We now give a precise description of IEX’s leakage profile and show that it is adaptively-secure with respect to it. Its setup leakage is

$$\begin{aligned} \mathcal {L}_\mathsf {S}^\mathsf {iex}({{\mathsf {D}}}{{\mathsf {B}}}) = \bigg (\mathcal {L}_\mathsf {S}^{\mathsf {dx}}({\mathsf {DX}}), \mathcal {L}^{{{\mathsf {m}}}{{\mathsf {m}}}}_\mathsf {S}(\mathsf {MM}_g)\bigg ), \end{aligned}$$

where $\mathcal {L}_\mathsf {S}^{\mathsf {dx}}({\mathsf {DX}})$ and $\mathcal {L}_\mathsf {S}^{{{\mathsf {m}}}{{\mathsf {m}}}}(\mathsf {MM}_g)$ are the setup leakages of the underlying dictionary and multi-map encryption schemes, respectively. Its query leakage is

$$\begin{aligned} \mathcal {L}_\mathsf {Q}^\mathsf {iex}&({{\mathsf {D}}}{{\mathsf {B}}}, {\mathbf {w}}) = \bigg ( \bigg ( \mathcal {L}_\mathsf {Q}^{\mathsf {dx}}\big ({\mathsf {DX}}, w_i\big ),\mathcal {L}_\mathsf {S}^{{{\mathsf {m}}}{{\mathsf {m}}}}(\mathsf {MM}_i),\\&\qquad \qquad \qquad \mathcal {L}_\mathsf {Q}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big (\mathsf {MM}_g, w_i\big ), \dots , \mathcal {L}_\mathsf {Q}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big (\mathsf {MM}_i, w_q\big ), \mathsf {TagPat}_i({{\mathsf {D}}}{{\mathsf {B}}}, {\mathbf {w}}) \bigg )_{i \in [q-1]}, \\&\qquad \qquad \qquad \mathcal {L}_\mathsf {Q}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big (\mathsf {MM}_g, w_q\big ), \mathsf {TagPat}_q({{\mathsf {D}}}{{\mathsf {B}}}, {\mathbf {w}}) \bigg ), \end{aligned}$$

where, for all $i \in [q]$,

$$\begin{aligned} \mathsf {TagPat}_i({{\mathsf {D}}}{{\mathsf {B}}}, {\mathbf {w}}) = \bigg ( \bigg (f_i\big (\mathsf{id}\big )\bigg )_{\mathsf{id}\in {{\mathsf {D}}}{{\mathsf {B}}}(w_i) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_{i+1})}, \dots , \bigg (f_i\big (\mathsf{id}\big )\bigg )_{\mathsf{id}\in {{\mathsf {D}}}{{\mathsf {B}}}(w_i) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_q)} \bigg ), \end{aligned}$$

and $f_i$ is a random function from $\{0,1\}^{|\mathsf{id}| + \log \#W }$ to $\{0,1\}^k$.

Theorem 1

If $\varSigma _{\mathsf {DX}}$ is adaptively $\big (\mathcal {L}_\mathsf {S}^{\mathsf {dx}}, \mathcal {L}_\mathsf {Q}^{\mathsf {dx}}\big )$-secure, $\varSigma _\mathsf {MM}$ is adaptively $\big (\mathcal {L}_\mathsf {S}^{{{\mathsf {m}}}{{\mathsf {m}}}}, \mathcal {L}_\mathsf {Q}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big )$-secure, $\mathsf {SKE}$ is RCPA-secure and F is pseudo-random, then IEX is $(\mathcal {L}^\mathsf {iex}_\mathsf {S}, \mathcal {L}_\mathsf {Q}^\mathsf {iex})$-secure.

The proof of Theorem 1 is deferred to the full version of the paper.

6 Boolean Queries with IEX

While IEX is naturally disjunctive, it can also support boolean queries. The boolean variant is similar to IEX in that it uses the same encrypted structures (i.e., it has the same $\mathsf{Setup}$ algorithm) but different ${\mathsf {Token}}$ and ${\mathsf {Search}}$ algorithms. We refer to the boolean variant of IEX as BIEX. We now provide an overview of how BIEX works.

Overview of BIEX. Recall that any query can be written in conjunctive normal form (CNF) so it has the form $\varDelta _1 \wedge \cdots \wedge \varDelta _\ell$, where each $\varDelta _i = w_{i,1} \vee \cdots \vee w_{i,q}$ is a disjunction. Note that the result ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1 \wedge \cdots \wedge \varDelta _\ell )$ is the intersection of ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1)$ through ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _\ell )$. But this intersection does not have to be computed “directly” by executing a naive intersection operation. A better alternative (from a leakage point of view) is to compute the intersection by starting with ${{\mathsf {D}}}{{\mathsf {B}}}(w_1)$, keeping only the subset of identifiers of ${{\mathsf {D}}}{{\mathsf {B}}}(w_1)$ that are also in ${{\mathsf {D}}}{{\mathsf {B}}}(w_2)$, then keeping only the subset of identifiers that are also in ${{\mathsf {D}}}{{\mathsf {B}}}(w_3)$ and so on. This alternative approach requires only information about ${{\mathsf {D}}}{{\mathsf {B}}}(w_1)$ and the progressive subsets. Moreover, it uses operations that are already supported by the IEX structures.

How we do this exactly, is best explained through a concrete example. Suppose we have a $\textsf {CNF}$ query with $\varDelta _1 = w_1 \vee w_2$ and $\varDelta _2 = w_3$. The first step would be to perform a disjunctive query for $\varDelta _1$, resulting in tags for the identifiers in ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1)$. In the second step, we want to filter out and keep the tags of identifiers in ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_3)$. To find these tags, it suffices to query the local multi-maps of $w_1$ and $w_2$ on $w_3$. In the first case, $\mathsf {EMM}_{w_1}$ will return tags for ${{\mathsf {D}}}{{\mathsf {B}}}(w_1) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_3)$ and in the second case $\mathsf {EMM}_{w_2}$ will return tags for ${{\mathsf {D}}}{{\mathsf {B}}}(w_2) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_3)$. Finally, we take the union of both of these intersections and perform a final intersection with ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1)$. The final result equals ${{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_3)$. Figure 2 describes this process in more detail and for arbitrary boolean queries.

Correctness. To show correctness we need to show that, given a boolean query in $\textsf {CNF}$ form $\varDelta _1 \wedge \cdots \wedge \varDelta _\ell$ such that $\varDelta _i = w_{i,1} \vee \cdots \vee w_{i,q}$ (for simplicity we assume the disjunctions all have q terms), BIEX.${\mathsf {Search}}$ outputs

$$\begin{aligned} \bigcap _{i\in [\ell ]}\bigcup _{j\in [q]} {{\mathsf {D}}}{{\mathsf {B}}}(w_{i,j}). \end{aligned}$$

(4)

Looking at the description of BIEX.${\mathsf {Search}}$ in Fig. 2, one can see that every time Step 4(d)i is invoked it outputs

$$\begin{aligned} \bigcup _{j \in [q]}{{\mathsf {D}}}{{\mathsf {B}}}(w_{i,t}) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,j}), \end{aligned}$$

for all $t \in [q]$ and $i \in [\ell ]$. Note that this stems from the fact that $\varSigma _\mathsf {MM}.\mathsf{Get}(\mathsf {EMM}_j,\mathsf {ltk}_{t,i,j})$ outputs ${{\mathsf {D}}}{{\mathsf {B}}}(w_{i,t}) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,j})$ for every $j \in [q]$.

Also, based on the correctness of IEX we know that the search for the first disjunction will output $\bigcup _{j \in [q]}{{\mathsf {D}}}{{\mathsf {B}}}(w_{1,j})$ (with no redundant identifiers). So we have the final result of the query

$$\begin{aligned} I_\ell = \bigcup _{j \in [q]}{{\mathsf {D}}}{{\mathsf {B}}}(w_{1,j}) \bigcap \underbrace{\bigg ( \bigcup _{j, l \in [q]} \big ({{\mathsf {D}}}{{\mathsf {B}}}(w_{2,j}) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,l})\big ) \bigg ) \bigcap \cdots \bigg (\bigcup _{j, l \in [q]} \big ({{\mathsf {D}}}{{\mathsf {B}}}(w_{\ell ,j}) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,l})\big )\bigg )}_{\ell -1 \ \text {terms}} \end{aligned}$$

(5)

On the other hand, note that for all $i \in [\ell ]$ we have by Morgan’s laws that

$$\begin{aligned} \bigcup _{j\in [q]} {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,j}) \bigcap \bigcup _{j\in [q]} {{\mathsf {D}}}{{\mathsf {B}}}(w_{i,j})&= \bigcup _{j\in [q]} {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,j}) \bigcap \bigcup _{j\in [q]} {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,j}) \bigcap \bigcup _{j\in [q]} {{\mathsf {D}}}{{\mathsf {B}}}(w_{i,j})\\&= \bigcup _{j\in [q]} {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,j}) \bigcap \bigg ( \bigcup _{j,l \in [q]} \big ({{\mathsf {D}}}{{\mathsf {B}}}(w_{i,j}) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_{1,l})\big ) \bigg ) \end{aligned}$$

That is, we can recursively apply the above result on Eq. (5) for all $l \in [\ell ]$ to obtain Eq. (4).

Efficiency. The storage complexity of BIEX is the same as IEX. Its search complexity is

$$\begin{aligned} O\bigg (q^2 \cdot \bigg (\max _{w \in \varDelta _1}\#{{\mathsf {D}}}{{\mathsf {B}}}(w) +\ell \cdot \#{{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1)\bigg )\bigg ). \end{aligned}$$

The term $q^2\cdot \max _{w\in \varDelta _1}\#{{\mathsf {D}}}{{\mathsf {B}}}(w)$ is the time to search for the first disjunction and the second term $q^2\cdot \ell \cdot \#{{\mathsf {D}}}{{\mathsf {B}}}(\varDelta _1)$ is the total number of local multi-map queries.

We can clearly see from the search complexity of BIEX that we can achieve better efficiency if the selectivity of the first disjunction is as small as possible. In practice, therefore, the first disjunction should be the one with the smallest selectivity; similarly to how the first keyword is chosen in OXT. Note that if the first disjunction in the $\textsf {CNF}$ form of the boolean query matches the entire database then the search complexity of BIEX will be linear while the optimal complexity might be sub-linear (the communication complexity of BIEX will remain optimal, however). It is not obvious to us how to improve this without pre-computing every possible query as it seems almost inherent to the query itself. With this in mind, it follows that BIEX has a sub-linear worst-case search complexity when the first disjunction’s selectivity is sub-linear.

The communication complexity of BIEX is optimal since the final set $I_\ell$ does not contain any redundant identifiers. Finally, note that it is non-interactive and token size is independent of the query’s selectivity.

Security. The setup leakage of BIEX is the same as IEX’s. Its query leakage includes the query leakage of IEX on the first disjunction and the query leakage of the encrypted local multi-maps when queried on all the terms of disjunctions $\varDelta _2, \dots , \varDelta _\ell$. Finally, it also includes the number of documents that match the terms of the first disjunction and the terms of remaining disjunctions.

We now give a precise description of the leakage profile of BIEX and show that it is adaptively-secure with respect to it. The setup leakage is

$$\begin{aligned} \mathcal {L}_\mathsf {S}^\mathsf {iexb}({{\mathsf {D}}}{{\mathsf {B}}}) = \mathcal {L}_\mathsf {S}^\mathsf {iex}({{\mathsf {D}}}{{\mathsf {B}}}), \end{aligned}$$

where $\mathcal {L}_\mathsf {S}^\mathsf {iex}({{\mathsf {D}}}{{\mathsf {B}}})$ is the setup leakages of IEX. Given a CNF query $\varDelta _1 \wedge \cdots \wedge \varDelta _\ell$, the query leakage is

$$\begin{aligned} \mathcal {L}_\mathsf {Q}^\mathsf {iexb}\bigg ({{\mathsf {D}}}{{\mathsf {B}}}, \bigwedge _{i=1}^\ell \varDelta _i \bigg ) =&\bigg ( \mathcal {L}_\mathsf {Q}^\mathsf {iex}\big ({{\mathsf {D}}}{{\mathsf {B}}}, \varDelta _1\big ),\bigg ( \mathcal {L}_\mathsf {Q}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big (\mathsf {MM}_i, w_{l,1}\big ), \cdots , \mathcal {L}_\mathsf {Q}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big (\mathsf {MM}_i, w_{l,q}\big ), \\&\mathsf {TagPat}_{i,l}\bigg ({{\mathsf {D}}}{{\mathsf {B}}}, \bigwedge _{i=1}^\ell \varDelta _i\bigg ) \bigg )_{\begin{array}{c} i \in [q] \\ l \in [2,\cdots , \ell ] \end{array}}\bigg ). \end{aligned}$$

where,

$$\begin{aligned} \mathsf {TagPat}_{i,l}\bigg ({{\mathsf {D}}}{{\mathsf {B}}}, \bigwedge _{i=1}^\ell \varDelta _i\bigg ) = \bigg ( \bigg (f_i(\mathsf{id})\bigg )_{{{\mathsf {D}}}{{\mathsf {B}}}(w_{1,i}) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w_{l, 1})}, \dots , \bigg (f_i(\mathsf{id})\bigg )_{{{\mathsf {D}}}{{\mathsf {B}}}(w_{1, i}) \cap {{\mathsf {D}}}{{\mathsf {B}}}_(w_{l, q})} \bigg ) \end{aligned}$$

and $f_i$ is a random function from $\{0,1\}^{n + \log \#W }$ to $\{0,1\}^k$.

Theorem 2

If $\varSigma _{\mathsf {DX}}$ is adaptively $\big (\mathcal {L}_\mathsf {S}^{\mathsf {dx}}, \mathcal {L}_\mathsf {Q}^{\mathsf {dx}}\big )$-semantically secure and $\varSigma _\mathsf {MM}$ is adaptively $\big (\mathcal {L}_\mathsf {S}^{{{\mathsf {m}}}{{\mathsf {m}}}}, \mathcal {L}_\mathsf {Q}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big )$-secure, then BIEX is adaptively $(\mathcal {L}^\mathsf {iexb}_\mathsf {S}, \mathcal {L}_\mathsf {Q}^\mathsf {iexb})$-secure.

The proof of Theorem 2 is similar (at a high-level) to the proof of Theorem 1.

Fig. 2.

The scheme BIEX.

7 ZMF: A Compact and Adaptively-Secure SSE Scheme

The main limitation of IEX is its storage complexity of

$$\begin{aligned} O\bigg ( \mathsf {strg}_g\big (\mathsf {MM}_g\big ) + \sum _w \mathsf {strg}_\ell \big (\mathsf {MM}_w \big ) \bigg ), \end{aligned}$$

where $\mathsf {strg}_g$ and $\mathsf {strg}_\ell$ are the storage complexity of the global and local EMMs, respectively. If the latter are instantiated with standard sub-linear-time constructions such as [10, 11, 15, 24], we have

$$\begin{aligned} O\bigg ( \sum _w \#{{\mathsf {D}}}{{\mathsf {B}}}(w) + \sum _w \sum _{v\in {{\mathsf {c}}}{{\mathsf {o}}}(w)} \#{{\mathsf {D}}}{{\mathsf {B}}}(v) \cap {{\mathsf {D}}}{{\mathsf {B}}}(w) \bigg ), \end{aligned}$$

(6)

which does not compare favorably to standard single-keyword search solutions which require only $O\big (\sum _w \#{{\mathsf {D}}}{{\mathsf {B}}}(w)\big ),$ or to the OXT construction of [11] which requires

$$\begin{aligned} O\bigg ( \sum _w \#{{\mathsf {D}}}{{\mathsf {B}}}(w) + \log \left( \frac{1}{\varepsilon }\right) \cdot \sum _w \#{{\mathsf {D}}}{{\mathsf {B}}}(w) \bigg ) \end{aligned}$$

when XSet is instantiated with a Bloom filter with a false positive rate of $\varepsilon$. In particular, note that the second term in the asymptotic expression above hides a constant of 1, which makes OXT reasonably compact.

Our Approach. The main storage inefficiency in IEX comes from the local EMMs which contribute the second term in Eq. (6). Ideally, we could improve things if we could use more compact local EMMs. Unfortunately, all known sub-linear constructions require $O(\sum _w \#{{\mathsf {D}}}{{\mathsf {B}}}(w))$ storage. We observe, however, that for local EMMs sub-linear search is not necessary since in practice the number of label/tuple pairs they store is small in comparison to the total number of documents n. So, for our purposes, a linear-time construction would work as long as it was compact. In [19], Goh proposed a very compact construction called Z-IDX based on Bloom filters. Specifically, it needs only

$$\begin{aligned} O\bigg ( \log \left( \frac{1}{\varepsilon }\right) \cdot \sum _{v\in {\mathbf {V}}} \#\mathsf {MM}^{-1}[v] \bigg ) \end{aligned}$$

bits of storage, where ${\mathbf {V}}$ is the value space of the multi-map and $\varepsilon$ is the false positive rate. If we could encrypt the local EMMs of IEX with Z-IDX, the former’s storage would be

$$\begin{aligned} O\bigg ( \sum _w \#{{\mathsf {D}}}{{\mathsf {B}}}(w) + \log \left( \frac{1}{\varepsilon }\right) \cdot \sum _w \#{{\mathsf {c}}}{{\mathsf {o}}}(w) \bigg ), \end{aligned}$$

which is much more competitive with OXT (note that the second term here also has a constant of 1). Unfortunately, this approach does not work because Z-IDX is not adaptively secure. Nevertheless, we show how to construct a highly compact scheme that is. In the following, we first recall how Z-IDX works.

Goh’s ZIDX Scheme. Like any SSE scheme, Z-IDX can be viewed as a STE scheme and, in particular, as a multi-map encryption scheme. Conceptually, we observe that Z-IDX can be abstracted into two parts: (1) a compiler that transforms an underlying set encryption scheme into a multi-map encryption scheme; and (2) a concrete set encryption scheme based on Bloom filters and PRFs. We refer to the former as the Z-IDX transformation and describe it in detail in Fig. 3. Given a set encryption scheme $\varSigma _\mathsf {SET}$, it produces a multi-map encryption scheme $\varSigma _\mathsf {MM}$ that works as follows. The $\varSigma _\mathsf {MM}.\mathsf{Setup}$ algorithm takes as input a multi-map $\mathsf {MM}$ that maps labels to tuples of values from ${\mathbf {V}}$. It creates $\#{\mathbf {V}}$ sets $(S_v)_{v\in {\mathbf {V}}}$ such that $S_v$ holds the labels in $\mathsf {MM}$ that map to v. It then encrypts each set $S_v$ with $\varSigma _\mathsf {SET}$ resulting in an encrypted set $\mathsf {ESET}_v$. The encrypted multi-map $\mathsf {EMM}$ is simply the collection of encrypted sets $(\mathsf {ESET}_v)_{v\in {\mathbf {V}}}$. A $\varSigma _\mathsf {MM}$ token for a label $\ell$ is a $\varSigma _\mathsf {SET}$ token for $\ell$ and $\varSigma _\mathsf {MM}.\mathsf{Get}$ uses the token to test each set in $\mathsf {EMM}= (\mathsf {ESET}_v)_{v\in {\mathbf {V}}}$ and outputs v if the test succeeds.

Note that for $\varSigma _\mathsf {MM}$ to work, $\varSigma _\mathsf {SET}$ must satisfy a stronger STE form than what is described in Definition 1. In particular, it must be what we call multi-structure in the sense that the tokens produced with a key $K$ can be used to query all the structures encrypted under $K$. We provide formal syntax and security definitions of multi-structure STE schemes in the full version of the paper. The main difference between standard and multi-structure STE schemes are that in the latter the $\mathsf{Setup}$ algorithm is replaced with a key generation algorithm $\mathsf{Gen}(1^k)$ that takes as input a security parameter and outputs a secret key $K$; and an encryption algorithm ${\mathsf {Enc}}(K, \mathsf {DS})$ that takes as input a secret key $K$ and a data structure $\mathsf {DS}$ and outputs an encrypted structure $\mathsf {EDS}$.

Fig. 3.

The Z-IDX transformation.

Adaptive Security. From our abstract perspective, the reason Z-IDX is not adaptively-secure is because the Z-IDX transformation is (implicitly) applied to a set encryption scheme that is not adaptively-secure. We show in Theorem 3 below, however, that if the transformation is applied to an adaptively-secure set encryption scheme then the result is adaptively-secure as well. More precisely, we show that if the set encryption scheme is adaptively $(\mathcal {L}^\mathsf {set}_\mathsf {S}, \mathcal {L}^\mathsf {set}_\mathsf {Q})$-secure then the Z-IDX transformation yields a multi-map encryption scheme with the following leakage profile:

$$\begin{aligned} \mathcal {L}^{{{\mathsf {m}}}{{\mathsf {m}}}}_\mathsf {S}(\mathsf {MM}) = \left( \left( \mathcal {L}_\mathsf {S}^\mathsf {set}\big (\mathsf {MM}^{-1}[v]\big )\right) _{v \in {\mathbf {V}}}, \#{\mathbf {V}}\right) , \end{aligned}$$

and

$$\begin{aligned} \mathcal {L}^{{{\mathsf {m}}}{{\mathsf {m}}}}_\mathsf {Q}(\mathsf {MM}, q)= \mathcal {L}_\mathsf {Q}^{\mathsf {set}}\left( \left( \mathsf {MM}^{-1}[v]\right) _{v\in {\mathbf {V}}}, q\right) . \end{aligned}$$

Theorem 3

If $\varSigma _\mathsf {SET}$ is adaptively $(\mathcal {L}^\mathsf {set}_\mathsf {S}, \mathcal {L}^\mathsf {set}_\mathsf {Q})$-secure then the scheme $\varSigma _\mathsf {MM}$ that results from applying the Z-IDX transformation to it is adaptively $(\mathcal {L}^{{{\mathsf {m}}}{{\mathsf {m}}}}_\mathsf {S}, \mathcal {L}^{{{\mathsf {m}}}{{\mathsf {m}}}}_\mathsf {Q})$-secure.

Due to space constraints, the proof of Theorem 3 will appear in the full version of the paper.

7.1 An Adaptively-Secure and Multi-structure Set Encryption Scheme

In this Section, we construct an adaptively-secure, highly-compact and multi-structure set encryption scheme. Then, by applying the Z-IDX transformation to it we get an adaptively-secure and highly-compact multi-map encryption scheme which we then use in IEX.

Adaptive Security. The main difficulty in designing adaptively-secure encrypted structures is supporting equivocation during simulation. Roughly speaking, the issue is that during the ${\mathbf {Ideal}}(k)$ experiment the simulator first needs to simulate an encrypted structure for the adversary and later needs to be able to simulate tokens that work correctly with the simulated structure produced in the first step. The challenge in supporting equivocation is that at the time the encrypted structure is simulated, the simulator has no information about the adversary’s queries so it is not clear how to simulate the structure in a way that will work correctly at query time. So to handle equivocation, the construction needs to be carefully designed and, typically, needs expensive cryptographic primitives. Fortunately, as first shown by Chase and Kamara [13], in the setting of symmetric STE, equivocation can be achieved very efficiently based only on XOR and PRF operations.

Our Base Scheme. One possible way to design an encrypted Bloom filter is as follows. Let ${\mathbf {U}}$ be a universe of elements. Given a set $S \subseteq {\mathbf {U}}$, insert the value $F_{K}(a)$, for all $a \in S$, in a standard Bloom filter, where F is a pseudo-random function. The token for an element $a \in {\mathbf {U}}$ is $\mathsf {tk}= F_{K}(a)$ and the Bloom filter can be queried by doing a standard Bloom filter test on $\mathsf {tk}$.

The main problems with this construction are that: (1) it reveals information about the size of S; and (2) it is not adaptively-secure. To achieve adaptive security, we can encrypt the Bloom filter by XORing each of its bits with a pad generated from another pseudo-random function G. This encryption step both hides the size of S and allows for equivocation. Now the token $\mathsf {tk}$ for an element $a \in {\mathbf {U}}$ includes $F_{K_1}(a)$ and the pads for locations $H_1(F_{K_1}(a))$ through $H_\lambda (F_{K_1}(a))$, where $(H_1, \dots , H_\lambda )$ are the hash functions used for the Bloom filter.

For this to work, however, the pads have to be designed carefully. More precisely, correctness requires that the pads only depend on the locations that they mask otherwise two (or more) elements $a_1$ and $a_2$ that collide under one of the hash functions will produce different masks for the same location. To get such location-dependent pads we compute them as $G_{K_2}(\ell )$, where $\ell$ is the $\ell$th bit of the filter. Now, a token for element a is set to

$$\begin{aligned} \mathsf {tk}= \bigg ( F_{K_1}(a), G_{K_2}\bigg (H_1\big (F_{K_1}(a)\big )\bigg ), \dots , G_{K_2}\bigg (H_\lambda \big (F_{K_1}(a)\big )\bigg ) \bigg ). \end{aligned}$$

The base construction described so far is compact and adaptively-secure but not multi-structure.

Reusability. Recall that a multi-structure STE scheme can produce multiple encrypted structures $(\mathsf {EDS}_1, \dots , \mathsf {EDS}_n)$ under a single key $K$ in such a way that a single (constant-size) token $\mathsf {tk}$ can be used to query all the structures generated under key $K$. So to make our base scheme multi-structure, the pads have to be filter-dependent in addition to being location-dependent so that different pads are used for different filters even if they mask the same location. We do this by setting the pads to be the output of a random oracle applied to the pair $(G_{K_2}(\ell ), \mathsf{id})$ where $\mathsf{id}$ is the identifier of the filter. The purpose of the random oracle here is twofold. First, it enables the extraction of n (random) pads from pairs $(G_{K_2}(\ell ), \mathsf{id}_1)$ through $(G_{K_2}(\ell ), \mathsf{id}_n)$ without relying on n secret keys. This, in turn, means the tokens can be of size independent of n. Second, it allows the simulator to equivocate on the pads while, again, keeping the tokens independent of n.

While the base scheme is now compact, adaptively-secure and multi-structure, it produces very large tokens. The problem is that if two sets $S_1$ and $S_2$ have different sizes, then the parameters of their Bloom filters (i.e., the array sizes, number of hash functions and hash function ranges) have to be different. The consequence is that in our encrypted set scheme, we will need different sets of hash functions for each filter which, in turn, means the tokens will have to include multiple pads for every filter.

Matryoshka Filters. We solve this problem as follows. Instead of encrypting a set of standard Bloom filters as in our base construction, we encrypt a new filter-based structure we refer to as matryoshka filters (MF).⁵ MFs are essentially a set of nested Bloom filters of varying sizes whose hash functions are all derived from a fixed set of hash functions. More precisely, consider a sequence of sets $S_1, \dots , S_n \subseteq {\mathbf {U}}$ not necessarily of the same size. We assume for simplicity that the sets have size a multiple of 2. For some false negative rate $2^{-\lambda }$, choose $\lambda$ independent and ideal random hash functions $(H_1, \dots , H_\lambda )$ from ${\mathbf {U}}$ to $[(\lambda /\ln 2)\cdot \max _i\#S_i]$. We refer to these functions as the maximal hash functions and to their associated filter as the maximal filter. For every set $S_i$, construct a Bloom filter of size $[\lceil (\lambda /\ln 2)\cdot \#S_i\rceil ]$ with hash functions $(H_1^i, \dots , H_h^i)$ where, for all $j \in [\lambda ]$, $H_j^i(a) = H_j(a)_{\Vert p_i}$ with $p_i = \lceil \log \big ((\lambda /\ln 2)\cdot \#S_i\big )\rceil$. We refer to these hash functions as the derived functions and to their associated filters as the derived filters. Note that if the maximal hash functions are ideal random functions then so are the derived functions so the standard Bloom filter analysis holds.

Encrypting Matryoshka Filters. As mentioned above, our final solution consists of adapting our base scheme to encrypt matryoshka filters instead of standard Bloom filters. In other words, we XOR each bit of each matryoshka filter with location- and filter-dependent pads. The main difference with the base scheme is that here the pads also need to be nested; that is, given a pad for the maximal filter we need to be able to construct the pads for the derived filters. To support this, we make use of the properties of online ciphers; namely, that given an n-bit string s and a B-online cipher $\mathsf{OC}$, the following equality holds:

$$\begin{aligned} \mathsf{OC}_{K}\bigg (s^{\Vert B}_{|p\times B}\bigg ) = \mathsf{OC}_{K}\bigg (s^{\Vert B}\bigg )_{|p\times B}, \end{aligned}$$

(7)

where $p < n$. This can be derived as follows. From the correctness property of online ciphers, we have

$$\begin{aligned} \mathsf{OC}_{K}\bigg (s^{\Vert B}_{|p\times B} \bigg )&= \mathsf{OC}^1_K\bigg ( s^{\Vert B}_{|p\times B} \bigg ) \Vert \cdots \Vert \mathsf{OC}^p_K\bigg ( s^{\Vert B}_{|p\times B} \bigg ) \\&= X\bigg (K, s^{\Vert B}_{|B}\bigg ) \Vert \cdots \Vert X\bigg (K, s^{\Vert B}_{|p\times B}\bigg ), \end{aligned}$$

and

$$\begin{aligned} \mathsf{OC}_K\bigg ( s^{\Vert B} \bigg )&= \mathsf{OC}^1_K\bigg ( s^{\Vert B} \bigg ) \Vert \cdots \Vert \mathsf{OC}^n_K\bigg ( s^{\Vert B} \bigg ) \\&= X\bigg (K, s^{\Vert B}_{|B}\bigg ) \Vert \cdots \Vert X\bigg (K, s^{\Vert B}_{|n\times B}\bigg ), \end{aligned}$$

for some function X. It follows then that

$$\begin{aligned} \mathsf{OC}_K\bigg ( s^{\Vert B} \bigg )_{|p\times B}&= X\bigg (K, s^{\Vert B}_{|B}\bigg ) \Vert \cdots \Vert X\bigg (K, s^{\Vert B}_{|p\times B}\bigg ) \\&= \mathsf{OC}_{K}\bigg (s^{\Vert B}_{|p\times B} \bigg ). \end{aligned}$$

Now, to encrypt the $\ell$th bit of a matryoshka filter, we use a pad constructed as

$$\begin{aligned} {\mathsf {R}}\bigg ( \mathsf{OC}_K\bigg (\ell ^{\Vert B}_{|p\times B}\bigg ), \mathsf{id}(S) \bigg ), \end{aligned}$$

where ${\mathsf {R}}$ is a random oracle and $\mathsf{id}(S)$ is the identifier of the filter. Note that the pad is both filter- and location-dependent. In addition, if the server is provided the value $\mathsf{OC}_K\big ( \ell ^{\Vert B}\big )$ for the maximal filter, it follows by Eq. (7) that it can derive the above pad as

$$\begin{aligned} {\mathsf {R}}\bigg ( \mathsf{OC}_K\big (\ell ^{\Vert B}\big )_{|p\times B}, \mathsf{id}(S) \bigg ). \end{aligned}$$

The detailed description of our set encryption scheme is given in Fig. 4. In the Theorem below we show that it is adaptively-secure with the following leakage profile:

$$\begin{aligned} \mathcal {L}^\mathsf {est}_\mathsf {S}(S) = \#S \quad \text { and } \quad \mathcal {L}^\mathsf {set}_\mathsf {Q}\bigg (S_1, \dots , S_n, q\bigg ) = \bigg ( b_1, \dots , b_n, {{\mathsf {S}}}{{\mathsf {P}}}(q) \bigg ), \end{aligned}$$

where ${{\mathsf {S}}}{{\mathsf {P}}}$ is the search pattern; that is, if and when two queries are the same. More formally, if t queries have been made, ${{\mathsf {S}}}{{\mathsf {P}}}(q)$ outputs a t-bit string with a 1 at location i if q is equal to the ith query.

Fig. 4.

An adaptively-secure multi-structure set encryption scheme.

Theorem 4

If $\mathsf{OC}$ is secure, then the multi-structure set encryption scheme described in Fig. 4 is adaptively $(\mathcal {L}^\mathsf {set}_\mathsf {S}, \mathcal {L}^\mathsf {set}_\mathsf {Q})$-secure in the random oracle model.

The proof of Theorem 4 is in the full version of the paper.

7.2 The ZMF Multi-map Encryption Scheme

By applying the Z-IDX transformation to our multi-structure set encryption scheme from Fig. 4, we get a new adaptively-secure multi-map encryption scheme we call ZMF. We state its security formally in the following Corollary of Theorems 3 and 4. Its leakage profile is,

$$\begin{aligned} \mathcal {L}^\mathsf {zmf}_\mathsf {S}(\mathsf {MM}) = \bigg ( \bigg (\#\mathsf {MM}^{-1}[v] \bigg )_{v\in {\mathbf {V}}}, \#{\mathbf {V}}\bigg ) \quad \text { and } \quad \mathcal {L}^\mathsf {zmf}_\mathsf {Q}(\mathsf {MM}, q) = \bigg (b_1, \dots , b_{\#{\mathbf {V}}}, {{\mathsf {S}}}{{\mathsf {P}}}(q) \bigg ) \end{aligned}$$

where $b_i$ is 1 if $q \in \mathsf {MM}^{-1}[v_i]$ and 0 otherwise, and $v_i$ is the ith value in ${\mathbf {V}}$.

Corollary 1

The ZMF multi-map encryption scheme which results from applying the Z-IDX transformation to the set encryption scheme of Fig. 4 is $(\mathcal {L}^\mathsf {zmf}_\mathsf {S}, \mathcal {L}^\mathsf {zmf}_\mathsf {Q})$-adaptively secure.

8 DIEX: A Dynamic SSE Scheme

We describe our dynamic SSE construction DIEX. As far as we know, it is the first adaptively-secure dynamic SSE scheme that is forward-secure and supports Boolean search queries in sub-linear time. In particular, it supports the addition, deletion and editing of files. In the full version of the paper, we recall the syntax and security definitions for dynamic STE.

Overview. As a starting point, we describe a dynamic version of IEX that is not forward-secure. For this, we make two changes to our static construction. First, we replace the encrypted dictionary $\mathsf {EDX}$ and the global encrypted multi-map $\mathsf {EMM}_g$ with a dynamic encrypted dictionary $\mathsf {EDX}^+$ and a dynamic global encrypted multi-map $\mathsf {EMM}_g^+$. The encrypted local multi-maps remain static. Second, we require that these new structures be response-hiding. We provide a high level description of our construction which is described in detail in Fig. 5.

The DIEX.$\mathsf{Setup}$ algorithm is the same as the IEX.$\mathsf{Setup}$ with the exception that it uses a dynamic encrypted dictionary and a dynamic encrypted multi-map and outputs state information $st$. The DIEX.$\mathsf {Token}^{\mathsf {sr}}$ algorithm is similar to IEX.${\mathsf {Token}}$ with the exception that it is stateful. Here, the state is just used to generate tokens for the underlying dynamic dictionary and global multi-map encrypted structures. The DIEX.$\mathsf {Token}^{\mathsf {up}}$ algorithm works as follows. It takes as inputs the key $K$, the state $st$ and an update $u = (\mathsf{op}, \mathsf{id}, W _\mathsf{id})$ that consists of an operation $\mathsf{op}\in \{\mathsf{edit^{\mathsf {+}}}, \mathsf{edit^{\mathsf {-}}}\}$, the document identifier $\mathsf{id}$ being edited and a set of keywords $W _\mathsf{id}$ to add or delete based on $\mathsf{op}$. We have the following cases:

• if $u = (\mathsf{edit^{\mathsf {+}}}, \mathsf{id}, W _\mathsf{id})$, the client will update the global multi-map $\mathsf {EMM}_g$ with pairs $(w, \mathsf {tag}_\mathsf{id})$ for all $w \in W _\mathsf{id}$. Here, $\mathsf {tag}_\mathsf{id}:={\mathsf {Enc}}_{K_1}\big (\mathsf{id}; F_{K_2}\big (\mathsf{id}\Vert w\big )\big )$ as in IEX. This is done by generating update tokens $(\mathsf {utk}^w_g)_{w\in W _\mathsf{id}}$ for $\mathsf {EMM}_g$ using $\varSigma _\mathsf {MM}.\mathsf {Token}^{\mathsf {up}}$. For all $w \in W _\mathsf{id}$, the client generates a new local multi-map $\mathsf {MM}_w$ that maps all $v \in W _\mathsf{id}\setminus \{w\}$ to $\mathsf {tag}_\mathsf{id}$. It encrypts all these local multi-maps $(\mathsf {MM}_w)_{w\in W _\mathsf{id}}$ with $\varSigma _{\mathsf {DX}}.\mathsf{Setup}$, resulting in $(\mathsf {EMM}_w)_{w\in W _\mathsf{id}}$ and creates update tokens $(\mathsf {utk}^w_d)_{w\in W _\mathsf{id}}$ for $\mathsf {EDX}$. The algorithm outputs an update token

  $$\begin{aligned} \mathsf {utk}= \bigg ( \mathsf{op}, \big (\mathsf {utk}_d^w\big )_{w\in W _\mathsf{id}}, \big (\mathsf {utk}_g^w\big )_{w\in W _\mathsf{id}} \bigg ), \end{aligned}$$

  and $st= (st_d, st_g)$, where the former is the state maintained by $\varSigma _{\mathsf {DX}}$ and the latter is the state maintained by $\varSigma _\mathsf {MM}$.

• if $u = (\mathsf{edit^{\mathsf {-}}}, \mathsf{id}, W _\mathsf{id})$, the client only updates $\mathsf {EMM}_g$. Specifically, it removes all pairs $(w, \mathsf {tag}_\mathsf{id})$ for $w \in W _\mathsf{id}$. This can be done by computing tags as above and generating update tokens $(\mathsf {utk}_g^w)_{w\in W _\mathsf{id}}$ using $\varSigma _\mathsf {MM}.\mathsf {Token}^{\mathsf {up}}$. The algorithm outputs the update token

  $$\begin{aligned} \mathsf {utk}= \bigg (\mathsf{op}, (\mathsf {utk}^w_g)_{w \in W _{\mathsf{id}}} \bigg ), \end{aligned}$$

  and $st= st_g$ where $st_g$ is the state maintained by $\varSigma _\mathsf {MM}$.

The ${\mathsf {Update}}$ algorithm takes as input $\mathsf {EDB}$ and an update token $\mathsf {utk}$ and outputs $\mathsf {EDB}'$. If $\mathsf{op}= \mathsf{edit^{\mathsf {+}}}$, it uses the sub-tokens in $\mathsf {utk}$ to update $\mathsf {EMM}_g$ and $\mathsf {EDX}$. If $\mathsf{op}=\mathsf{edit^{\mathsf {-}}}$, it only updates $\mathsf {EMM}_g$. The ${\mathsf {Search}}$ algorithm is the same as IEX.${\mathsf {Search}}$. Recall that we do not update the local multi-maps already in $\mathsf {EDX}$. This is not necessary to for correctness because, during search, the server will take the intersection of the tags returned from the global multi-map $\mathsf {EMM}_g$ and from the appropriate local multi-maps. However, because $\mathsf {EMM}_g$ is properly updated, the intersection operation will filter out the old/stale tags from the local multi-map.

Forward Security. We note that DIEX is forward secure if its underlying structures are. Specifically, if $\varSigma _\mathsf {MM}$ and $\varSigma _{\mathsf {DX}}$ are forward secure then so is DIEX. This is easy to see from the fact the DIEX tokens only consist of $\varSigma _{\mathsf {DX}}$ and $\varSigma _\mathsf {MM}$ tokens so if the former can be simulated from the security parameter, then the latter can. Due to space constraints, the definition of forward security is differed to the full version of the paper. As a possible instantiation of a forward secure multi-map and dictionary encryption scheme, one can use the Sophos scheme of Bost [9].

Fig. 5.

The scheme DIEX.

Efficiency. The efficiency of DIEX depends on the underlying multi-map and dictionary encryption schemes. Using optimal constructions, the search complexity of DIEX is the same as IEX; that is, $O(q^2 \cdot M)$, where $M = \max _{i \in [q]}\#{{\mathsf {D}}}{{\mathsf {B}}}(w_i)$ and q is the number of terms in the disjunction.

Security. We show that DIEX is adaptively secure with respect to the following well-defined leakage profile. The setup and query leakages are the same as IEX so we only describe the update leakage. For an update $u = (\mathsf{edit^{\mathsf {+}}}, \mathsf{id}, W _\mathsf{id})$,

$$\begin{aligned} \mathcal {L}_\mathsf {U}\bigg ({{\mathsf {D}}}{{\mathsf {B}}}, u \bigg ) = \bigg ( \mathcal {L}_\mathsf {U}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big (\mathsf {MM}_g, (\mathsf{op}, w,\mathsf{id}) \big ), \mathcal {L}_\mathsf {U}^{\mathsf {dx}}\big ({\mathsf {DX}}, (\mathsf{op}, w,\mathsf{id}) \big ), \mathcal {L}_\mathsf {S}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big ( \mathsf {MM}_w\big )\bigg )_{w \in W _\mathsf{id}}. \end{aligned}$$

If $u=(\mathsf{edit^{\mathsf {-}}}, \mathsf{id}, W _\mathsf{id})$: $\mathcal {L}_\mathsf {U}^\mathsf {diex}\bigg ({{\mathsf {D}}}{{\mathsf {B}}}, u \bigg ) = \bigg ( \mathcal {L}_\mathsf {U}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big (\mathsf {MM}_g, (\mathsf{op}, w,\mathsf{id}) \big ) \bigg )_{w \in W _\mathsf{id}}.$

Theorem 5

If $\varSigma _{\mathsf {DX}}$ is adaptively $\big (\mathcal {L}_\mathsf {S}^{\mathsf {dx}}, \mathcal {L}_\mathsf {Q}^{\mathsf {dx}}, \mathcal {L}_\mathsf {U}^{\mathsf {dx}}\big )$-semantically secure and $\varSigma _\mathsf {MM}$ is adaptively $\big (\mathcal {L}_\mathsf {S}^{{{\mathsf {m}}}{{\mathsf {m}}}}, \mathcal {L}_\mathsf {Q}^{{{\mathsf {m}}}{{\mathsf {m}}}}, \mathcal {L}_\mathsf {U}^{{{\mathsf {m}}}{{\mathsf {m}}}}\big )$-secure, then DIEX is adaptively $(\mathcal {L}^\mathsf {diex}_\mathsf {S}, \mathcal {L}_\mathsf {Q}^\mathsf {diex}, \mathcal {L}_\mathsf {U}^\mathsf {diex})$-secure.

We defer the proof to the final version of this work.

9 Empirical Evaluation

To evaluate the practicality of our schemes, we designed and built an open source encrypted search framework called Clusion [22]. Due to space limitations, however, we defer the empirical analysis of our constructions to the full version of this work. There, we evaluate IEX-2Lev and IEX-ZMF which are instantiations of IEX with 2Lev and ZMF, respectively. We also evaluate our Boolean scheme BIEX. Our experiments report setup time, search time, storage and token size for all our constructions.