Computing Neural Networks with Homomorphic Encryption and Verifiable Computing

Abstract

The widespread use of machine learning and in particular of Artificial Neural Networks (ANN) raises multiple security and data privacy issues. Recent works propose to preserve data confidentiality during the inference process, available as an outsourced service, using Homomorphic Encryption techniques. However, their setting is based on an honest-but-curious service provider and none of them addresses the problem of result integrity. In this paper, we propose a practical framework for privacy-preserving predictions with Homomorphic Encryption (HE) and Verifiable Computing (VC). We propose here a partially encrypted Neural Network in which the first layer consists of a quadratic function and its homomorphic evaluation is checked for integrity using a VC scheme which is slight adaption of the one of Fiore et al. [13]. Inspired by the neural network model proposed by Ryffel et al. [26] which combines adversarial training and functional encryption for partially encrypted machine learning, our solution can be deployed in different application contexts and provides additional security guarantees.

We validate our work on the MNIST handwritten recognition dataset for which we achieve high accuracy (97.54%) and decent latency for a practical deployment (on average 3.8 s for both homomorphic evaluation and integrity proof preparation and 0.021 s for the verification).

Appendices

A Properties of VC

Let us now present a summary for some general properties of $\mathcal {VC}$ schemes, for more details see [13, 14]:

Correctness: The $\mathcal {VC}$ is correct if the client running the verification algorithm accepts, with a high probability, the output send by the server only when this one is correct.

Security: A $\mathcal {VC}$ scheme is secure if a malicious server cannot persuade the verification algorithm to accept an incorrect output.

Privacy: A $\mathcal {VC}$ scheme is private when the public outputs of the problem generation algorithm ProbGen over two different inputs are indistinguishable.

Function Privacy: This requirement guarantees that the public key PK, sampled via $(PK, SK) \leftarrow \mathbf{KeyGen} (f, \lambda )$, does not leak information on the encoded function f, even after a polynomial amount of runs of $\mathbf{ProbGen} _{SK}$ on adversarially chosen inputs.

Outsourceability: A $\mathcal {VC}$ can be outsourced if it allows efficient generation and efficient verification. (i.e. the time of $\left( \mathbf{ProbrGen} _{SK}(x)+\mathbf{Verify} (\sigma _y) \right)$ is in O(T), where T is the time required to compute f(x)).

Adaptive Security: The adaptive security for a $\mathcal {VC}$ scheme is defined by the security when the adversary chooses f after having seen many “encodings” of $\sigma _x$ for adaptively-chosen values x.

This type of schemes allows to compute $\sigma _x$ independently of f so we can calculate $\sigma _x$ before choosing f.

B Realization of PRF with Amortized Closed-Form Efficiency

Definition 2

A PRF (F.KG, F) is secure if, for every PPT adversary $\mathcal {A}$, we have that:

$\left| Pr[\mathcal {A}^{F_K(\cdot )}(\lambda , pp)=1]-Pr[\mathcal {A}^{\varPhi (\cdot )}(\lambda , pp)=1]\right| \le neg(\lambda )$ where: $(K,pp)\leftarrow KG(\lambda )$ and $\varPhi : \chi \rightarrow \mathcal {R}$ is a random function (i.e. it is not possible to distinguish between F and $\varPhi$).

Let $f:\mathbb {F}_q^n \rightarrow \mathbb {F}_q$ be an arithmetic circuit of degree 2, and, without loss of generality, parse $f(x_1,\ldots ,x_n)=\sum _{i,j}^{n}{\zeta _{i,j}\cdot x_i\cdot x_j}+ \sum _{k=1}^{n}{\zeta _{k}\cdot x_k}$

for some $\zeta _{i,j},\zeta _k \in \mathbb {F}_q$. we $\hat{f}:(\mathbb {G}_1\times \mathbb {G}_2)^n\rightarrow \mathbb {G}_T$ as the compilation of f on group elements such as: $\hat{f}(A_1,B_1\ldots ,A_n,B_n)= \prod _{i,j}^{n}{{\zeta _{i,j}} \cdot e(A_i,B_j)}\cdot \sum _{k=1}^{n}{\zeta _k \cdot e(A_k,h)}$

We will show the realization for the PRF with amortized closed-form efficiency For $\mathbf{Comp} (R_1,S_1,V_1\ldots , R_n,S_n,V_n,f)=\hat{f}(R_1,S_1,V_1\ldots , R_n,S_n,V_n)$. That is the adaptation of the scheme of Bakes et al. in [1] to work with the asymmetric bilinear group.

• F.KG( $\lambda$ ) $\rightarrow K=(K_1,K_2):$

  First generate $bgpp=(q,g,h,e)$ some bilinear group parameter, where $\mathbb {G}_1=<g>, \mathbb {G}_2=<h>\ and\ q=order(\mathbb {G}_i)\ for\ i=1,2$ and $e:\mathbb {G}_1\times \mathbb {G}_2 \rightarrow \mathbb {G}_T$ non−degenerate($\mathbb {G}_T=<e(g,h)>$) bilinear map. Choose two seeds $K_1,K_2$ for a family of PRFs $\mathbf{F} '_{K_{1,2}}:\{0,1\}^* \rightarrow \mathbb {F}_q^2$. Output $K_1,K_2$. The parameters define $F:\chi =\{0,1\}^*\times \{0,1\}^*\rightarrow \mathcal {R}^3$.

• F$_K(\varDelta ,\tau )\rightarrow (R,S,V)$:

  It generates $(u,v)\leftarrow F'_{K_1}(\tau )$ and $(a,b)\leftarrow F'_{K_2}(\varDelta )$ Finally it calculates $(R,S)=( g^{ua+vb},h^{ua+vb})$.

• CFEval $_{{\tau }}^{off}(K,f)\rightarrow w_f=\rho :$

  For $i=1 \; to \; t:$ calculate $(u_i,v_i)=F'_{K_1}(\tau _i)$ and construct a linear map $\rho _i$ using $(u_i,v_i)$ as $\rho _i(x_1,x_2)=u_i \cdot x_1 + v_i \cdot x_2$ Run $\rho \leftarrow f(\rho _1,\ldots ,\rho _t)$, i.e., $\forall z_1,z_2 \in \mathbb {F}_q$: $\rho (z_1,z_2)=f(\rho _1(z_1,z_2),\ldots ,\rho _t(z_1,z_2)).$

• CFEval $_{\varDelta }^{on}(K,w_f)\rightarrow W:$

  It generates $(a,b)\leftarrow F'_{k_{2}}(\varDelta )$ and computes $W= e(g,h)^{w_f(a,b)}$. for the proof of this scheme follow theorem 4 of [13].

C Realizations of Homomorphic Hash [13]

In the construction of , we use the $bgpp=(q,g,h,e)$ where q be a prime of $\lambda$ bit and let $\mathbb {F}_q =\mathbb {Z}/q\mathbb {Z}$. Let us define a function $H_{\alpha ,\beta }(\mu )$ as follow: for $\mu \in \mathcal {D}=\{\mu \in \mathbb {Z}_q[x][y]: deg_x(\mu )=N, deg_y(\mu )=c\}\subset R_q[y]$, $H_{\alpha ,\beta }(\mu )$ first evaluates $\mu$ at $y=\alpha$ and then evaluates $\mu (\alpha )$ at $\beta$ i.e. $H_{\alpha ,\beta }(\mu )=ev_\beta \circ ev_\alpha (\mu )$.

The family of hash functions $(\tilde{H}.KeyGen,\tilde{H},\tilde{H}.Eval)$ with domain $\mathcal {D}$ and range $\mathbb {G}_1\times \mathbb {G}_2$ is defined as below:

• $\rightarrow (K,\kappa =(\alpha ,\beta ))$:

  First at all, generate $bgpp=(g,h,q)$ Next, sample a random $(\alpha ,\beta )\leftarrow (\mathbb {F}_q)^2$ Afterwords, for $i=0,\ldots ,c$ and,$j=1,\ldots ,N$ and we calculate $g^{\alpha ^i\beta ^j},$ and $h^{\alpha _i\alpha _j}$ and include them to K. Output K and $\kappa =(\alpha ,\beta )$.

• :

  For $\mu \in \mathcal {D}$, in function of its degree $deg_y(\mu )$, $\tilde{H}_\kappa (\mu )$ is computed differently. If $deg_y(\mu )\le 1$ then $\tilde{H}_\kappa (\mu )=(T,U)=(g^{H_\kappa (\mu )},h^{H_\kappa (\mu )})\in \mathbb {G}_1\times \mathbb {G}_2$. If $deg_y(\mu )=2$, then $e(g,h)^{H_{\kappa }(\mu )}$

• $(f_g,\nu _1,\nu _2)$: It computes in a homomorphic way a function of degree 2 on the outputs of $\tilde{H}$. For $\nu _1=(T_1,U_1)$, $\nu _2=(T_2,U_2)$ and (respectively, $\tilde{\text {T}}_1, \tilde{\text {T}}_2 \in \mathbb {G}_T$).