PINFER: Privacy-Preserving Inference

Abstract

The foreseen growing role of outsourced machine learning services is raising concerns about the privacy of user data. This paper proposes a variety of protocols for privacy-preserving regression and classification that (i) only require additively homomorphic encryption algorithms, (ii) limit interactions to a mere request and response, and (iii) that can be used directly for important machine-learning algorithms such as logistic regression and SVM classification. The basic protocols are then extended and applied to simple feed-forward neural networks.

A A More Private Protocols

1.1 A.1 Linear Regression

As seen in Sect. 2.1, linear regression produces estimates using the identity map for g: . Since is linear, given an encryption of ${{\varvec{x}}}$, the value of can be homomorphically evaluated, in a provably secure way [11].

Therefore, the client encrypts its feature vector ${{\varvec{x}}}$ under its public key and sends to the server. Using ${\varvec{\theta }}$, the server then computes and returns it the client. Finally, the client decrypts and gets the output . This is straightforward and only requires one round of communication.

1.2 A.2 SVM Classification

A Naïve Protocol. A client holding a private feature vector ${{\varvec{x}}}$ wishes to evaluate where ${\varvec{\theta }}$ parametrises an SVM classification model. In the primal approach, the client can encrypt ${{\varvec{x}}}$ and send to the server. Next, the server computes for some random mask $\mu$ and sends  to the client. The client decrypts  and recovers $\eta$. Finally, the client and the server engage in a private comparison protocol with respective inputs $\eta$ and $\mu$, and the client deduces the sign of from the resulting comparison bit $[\mu \leqslant \eta ]$.

There are two issues. If we use the DGK+ protocol for the private comparison, at least one extra exchange from the server to the client is needed for the client to get $[\mu \leqslant \eta ]$. This can be fixed by considering the dual approach. A second, more problematic, issue is that the decryption of yields $\eta$ as an element of $\mathcal {M}\cong \mathbb {Z}/M\mathbb {Z}$, which is not necessarily equivalent to the integer . Note that if the inner product can take any value in $\mathcal {M}$, selecting a smaller value for $\mu \in \mathcal {M}$ to prevent the modular reduction does not solve the issue because the value of $\eta$ may then leak information on .

A Heuristic Protocol (Dual Approach). The bandwidth usage with the heuristic comparison protocol (cf. Fig. 5) could be even reduced to one ciphertext and a single bit with the dual approach. From the published encrypted model , the client could homomorphically compute and send to the server for random $\lambda , \mu \in \mathcal {B}$ with . The server would then decrypt $\mathfrak {t}^*$, obtain $t^*$, compute ${\smash {\delta }_{\scriptscriptstyle S}}= \frac{1}{2}(1-\mathrm{sign}(t^*))$, and return ${\smash {\delta }_{\scriptscriptstyle S}}$ to the client. Analogously to the primal approach, the output class is obtained by the client as $\hat{y}= (-1)^{\smash {\delta }_{\scriptscriptstyle S}}\cdot \mathrm{sign}(\lambda )$. However, and contrarily to the primal approach, the potential information leakage resulting from $t^*$—in this case on ${{\varvec{x}}}$—is now on the server’s side, which is in contradiction with our Requirement 1 (input confidentiality). We do not further discuss this variant.