Through the Looking-Glass: Benchmarking Secure Multi-party Computation Comparisons for ReLU ’s

Abstract

Comparisons or Inequality Tests are an essential building block of Rectified Linear Unit functions (ReLU ’s), ever more present in Machine Learning, specifically in Neural Networks. Motivated by the increasing interest in privacy-preserving Artificial Intelligence, we explore the current state of the art of privacy preserving comparisons over Multi-Party Computation (MPC). We then introduce constant round variations and combinations, which are compatible with customary fixed point arithmetic over MPC. Our main focus is implementation and benchmarking; hence, we showcase our contributions via an open source library, compatible with current MPC software tools. Furthermore, we include a comprehensive comparative analysis on various adversarial settings. Our results improve running times in practical scenarios. Finally, we offer conclusions about the viability of these protocols when adopted for privacy-preserving Machine Learning.

Appendices

Appendix

A Rabbit Definitions

1.1 A.1 The Problem

The majority of previous works have proposed mechanisms for comparisons based on bit decomposition. Such constructions tend to be expensive in terms of multiplicative depth. In general terms, they are compatible with both MPC and Homomorphic Encryption (HE) schemes. They also tend to have sublinear cost on the inputs size with some associated non-negligible constant. An example of this, is the seminal work from Damgård et al. [7] previously mentioned by this work.

To solve this issue, the literature has proposed to relax the security model, as a trade-off for efficiency e.g. [2, 18]. The principle is to reduce the impact of bit decomposition in exchange of security under statistical constraints, i.e. statistical security. This statistical security should not be confused with cryptographic security, and it rather relates to the probability distribution over masked inputs.

1.2 A.2 The Rabbit Principle

The method itself is based on the commutative properties of addition. It formulates 2 different but equivalent equations. Generally speaking, the authors derive a simplified algebraic construction comprised of 3 INQ ’s, one of which relies exclusively on public inputs. The other 2 depend on public inputs and secret shared precomputed randomness. Furthermore, they can be all executed in parallel. This construction is equivalent to an LTEQZ on secret inputs.

To generate randomness, Rabbit authors rely on $\textsf {edaBits}$ [8] (which is exclusively present on MP-SPDZ). For the purpose of this section, we abstract $\textsf {edaBits}$, as a construction that allows us to generate (together with its bit expansion) some bounded randomness in $\mathbb {Z}_{2^k}$.

Let $\mathbb {Z}_{M}$ be some commutative ring bounded by $M \in \mathbb {Z}$, $[\![ x ]\!]$ and $[\![ r ]\!]$ be some secret shared input and randomness in $\mathbb {Z}_{M}$ and, R be some public element of $\mathbb {Z}_{M}$. Then we can establish the following:

$$\begin{aligned} B&= M-R,\end{aligned}$$

(5)

$$\begin{aligned}{}[\![ a ]\!]&= [\![ x ]\!] + [\![ r ]\!],\end{aligned}$$

(6)

$$\begin{aligned}{}[\![ b ]\!]&= [\![ x ]\!] + [\![ r ]\!] + B,\end{aligned}$$

(7)

$$\begin{aligned}{}[\![ c ]\!]&= [\![ x ]\!] + B. \end{aligned}$$

(8)

If we observe carefully the constructions above, we can appreciate that B is simply the complement of R. Prior discussing the algebraic elements of Rabbit, let us now consider the following statement:

(9)

Which is always true for any element of $\mathbb {Z}_{M}$. Given the equations above, we can establish the following relations, let us start with $([\![ a ]\!] + B)$:

$$\begin{aligned}{}[\![ b ]\!]&= [\![ a ]\!] + B \\&= [\![ a ]\!] + B - M \cdot ([\![ a+B ]\!]< B)\\&= [\![ x ]\!] + [\![ r ]\!] - M\cdot ([\![ x+r ]\!]< [\![ r ]\!]) + B - M\cdot ([\![ a+B ]\!] < B). \end{aligned}$$

When we expand $[\![ c+r ]\!]$ in the same fashion we can derive the following:

$$\begin{aligned}{}[\![ b ]\!]&= [\![ c+r ]\!]\\&= [\![ c ]\!] + r - M\cdot ([\![ c+r ]\!]< [\![ r ]\!]) \\&= [\![ x ]\!] + [\![ b ]\!] - M\cdot ([\![ x+B ]\!]<[\![ r ]\!]) + [\![ r ]\!] - M\cdot ([\![ c+r ]\!] < [\![ r ]\!]). \end{aligned}$$

If we equate both expansions of $[\![ b ]\!]$, using $[\![ a+B ]\!]$ and $[\![ c ]\!]+ [\![ r ]\!]$, we can obtain the following (after simplifications):

$$\begin{aligned}{}[\![ a<r ]\!] + [\![ b<B ]\!]&= [\![ c<B ]\!] + [\![ b<r ]\!]. \end{aligned}$$

Let us now replace $[\![ a ]\!]$, $[\![ b ]\!]$ and $[\![ c ]\!]$, and express the equation above, in terms of $[\![ x ]\!]$ and $[\![ r ]\!]$:

$$\begin{aligned}{}[\![ x+r ]\!]<[\![ r ]\!] + [\![ c+r ]\!]< B&= [\![ x+B ]\!]< B + [\![ x+r+B ]\!]< [\![ r ]\!], \\ [\![ x+B ]\!]< B&= [\![ x+r ]\!]<[\![ r ]\!] +[\![ x+B+r ]\!]< B - [\![ x+ r+ B ]\!] < [\![ r ]\!]. \end{aligned}$$

The INQ that we have conveniently now placed on the left of the equation, expresses the relation between x and the complement of R. In this case the $\textsf {INQ}$ would be true, only if x is greater than R. Note that we are still working on $\mathbb {Z}_{M}$, meaning that any excess over R would force an overflow and a subsequent wraparound. Now, let us abuse the notation and consider R to be some public element on $\mathbb {Z}_{M}$ e.g. 0.

Given that we can freely disclose the masked secret $[\![ x ]\!] +[\![ r ]\!]$ without compromising security, the equation above would finally look like:

$$\begin{aligned}{}[\![ x+B ]\!]< B&= (x+r)<[\![ r ]\!] +(x+B+r)<B - (x+ r+ B) < [\![ r ]\!]. \end{aligned}$$

As previously stated both inequalities can be calculated in parallel. The inequality tests themselves are executed bitwise, using for instance, $\textsf {edaBits}$ or any other mean to obtain the bit decomposition offline.

B Constant Round ReLU Protocol

Our proposed ReLU construction, follows the same line of thought from the contribution sections of this work. Indeed, Catrina and Saxena’s fixed point representation is heavily interlinked with the protocol. In fact, Protocol 6, optimizes the fixed point multiplication needed by the ReLU, extracting the mantissa from $\langle x \rangle$.

In line with the definitions, introduced in Sect. 2.3. Our constant round ReLU can be trivially implemented as follows:

Complexity: The protocol has constant round complexity ($\mathcal {O}(1)$). It consists of 1 round (from the multiplication on line 3), plus what is added by any selected comparison mechanism introduced by this work i.e. 4 rounds.

Discussion: Our ReLU itself does not require to invoke PRTrunc, ever present in fixed point multiplications. Note that ReLU ’s are typically surrounded by more complex fixed point operations, that do require conventional fixed point multiplications e.g. [21] (hence the presence of slack and an invocation of PRTrunc per multiplication gate).