$\mathsf {FLOD}$: Oblivious Defender for Private Byzantine-Robust Federated Learning with Dishonest-Majority

Abstract

Privacy and Byzantine-robustness are two major concerns of federated learning (FL), but mitigating both threats simultaneously is highly challenging: privacy-preserving strategies prohibit access to individual model updates to avoid leakage, while Byzantine-robust methods require access for comprehensive mathematical analysis. Besides, most Byzantine-robust methods only work in the honest-majority setting.

We present $\mathsf {FLOD}$, a novel oblivious defender for private Byzantine-robust FL in dishonest-majority setting. Basically, we propose a novel Hamming distance-based aggregation method to resist $>1/2$ Byzantine attacks using a small root-dataset and server-model for bootstrapping trust. Furthermore, we employ two non-colluding servers and use additive homomorphic encryption ($\mathsf {AHE}$) and secure two-party computation (2PC) primitives to construct efficient privacy-preserving building blocks for secure aggregation, in which we propose two novel in-depth variants of Beaver Multiplication triples (MT) to reduce the overhead of Bit to Arithmetic ($\mathsf {Bit2A}$) conversion and vector weighted sum aggregation ($\mathsf {VSWA}$) significantly. Experiments on real-world and synthetic datasets demonstrate our effectiveness and efficiency: (i) $\mathsf {FLOD}$ defeats known Byzantine attacks with a negligible effect on accuracy and convergence, (ii) achieves a reduction of $\approx$2$\times$ for offline (resp. online) overhead of $\mathsf {Bit2A}$ and $\mathsf {VSWA}$ compared to $\mathsf {ABY}$-$\mathsf {AHE}$ (resp. $\mathsf {ABY}$-$\mathsf {MT}$) based methods (NDSS’15), (iii) and reduces total online communication and run-time by 167–1416$\times$ and 3.1–7.4$\times$ compared to $\mathsf {FLGUARD}$ (Crypto Eprint 2021/025).

Appendices

A Byzantine-Robustness Analysis

Cosine similarity is one of the best metrics to measure the similarity of two vectors. Recall the cosine similarity of two $\mathsf {sgn}$ $\widetilde{\mathbf {w}_i}$ and $\widetilde{\mathbf {w}_s}$ is $c_i = \frac{\langle \widetilde{\mathbf {w}_i}, \widetilde{\mathbf {w}_s}\rangle }{\Vert \widetilde{\mathbf {w}_i}\Vert \cdot \Vert \widetilde{\mathbf {w}_s}\Vert }$, and $\mathsf {FLTrust}$ clips $c_i$ using $\mathrm ReLU$ function to remove the poisoned model updates with negative $c_i$ [11]. Based on $\widetilde{\mathbf {w}_i}$, $\widetilde{\mathbf {w}_s}$ in $\{-1,1\}^d$ and Eq. (2, 3), we have

$$\begin{aligned} \begin{aligned} c_i&=\frac{\sum _{j=1}^d \widetilde{w_{ij}}\cdot \widetilde{w_{sj}}}{\sqrt{d}\cdot \sqrt{d}}=\frac{1}{d}\cdot (\sum _{j=1}^d (1-2\mathcal {E}(\widetilde{w_{ij}}))\cdot (1-2\mathcal {E}(\widetilde{w_{sj}})))\\&= 1 -\frac{2}{d}\cdot (\sum _{j=1}^d(\mathcal {E}(\widetilde{w_{ij}})+\mathcal {E}(\widetilde{w_{sj}})-2\mathcal {E}(\widetilde{w_{ij}})\mathcal {E}(\widetilde{w_{sj}})))\\&= 1 - \frac{2}{d}\cdot (\sum _{j=1}^d\mathcal {E}(\widetilde{w_{ij}})\oplus \mathcal {E}(\widetilde{w_{sj}})) = 1-2\frac{hd_i}{d}. \end{aligned} \end{aligned}$$

Thus, we have $c_i>0 \Leftrightarrow 1-2\cdot \frac{hd_i}{d}>0\Leftrightarrow hd_i<\frac{d}{2}$. Therefore, with $\tau = \frac{d}{2}$ we have $\nu _i>0 \Leftrightarrow c_i>0$, which means $\tau$-clipping Hamming distance-based method is capable to exclude the poisoned $\mathsf {sgn}$ model updates equivalent to that the cosine similarity-based method achieved. What is more, our $\tau$-clipping Hamming distance-based method is more flexible than the cosine similarity-based one since we can alter $\tau$ for different tasks to achieve the best Byzantine-robustness.

B Proof of Theorem 1

Proof (of Theorem 1)

The universal composability framework [10] guarantees the security of arbitrary composition of different protocols. Therefore, we only need to prove the security of individual protocols. We give the proof of the security under the semi-honest model in the real-ideal paradigm [10].

Privacy of $\mathsf {CXOR}$. There is nothing to simulate as the protocol is non-interactive.

Privacy of $\mathsf {PCBit2A}$. In offline phase, $P_0$’s view in real-world is composed of $\{ \mathbf {x}_i, \mathbf {r}_i,\mathbf {x}_i', \mathbf {r}_i', \mathsf {AHE.Enc}_{\mathrm{pk}_0}(\mathbf {y}_{i})\}$. To simulate it in ideal-world, the $\mathsf {Sim}$ can simply return $\{\mathbf {\Delta }_i^x,\mathbf {\Delta }_i^r, \mathbf {\Delta }_i^{x'},\mathbf {\Delta }_i^{r'}, \mathsf {AHE.Enc}_{\mathrm{pk}^{'}_0}([0,0,...,0])\}$ where $\mathbf {\Delta }_i^x,\mathbf {\Delta }_i^r, \mathbf {\Delta }_i^{x'},\mathbf {\Delta }_i^{r'}$ are chosen from $\mathcal {R}^d$ at random and $\mathrm{pk}'_0$ is generated by $\mathsf {Sim}$. Due to the semantic security of $\mathsf {AHE}$, these two views are computationally indistinguishable from each other. And $P_1$’s view in real execution can also be simulated by $\mathsf {Sim}$ which outputs two random vectors in $\mathcal {R}^d$ since the real-world view $\{\boldsymbol{\xi }_{i}, \boldsymbol{\xi }'_{i}\}$ are masked by random vectors $\mathbf {r}_{i}$ and $\mathbf {r}'_{i}$. In online, the output of $\mathsf {Sim}$ for corrupted $P_t$ is one share which is uniformly chosen from $\mathcal {R}^d$, and thus $P_t$’s view in the real-world is also indistinguishable from that in ideal-world.

Privacy of Private $\tau$-$\mathsf {Clipping}$. As the underlying garbled circuits are secure, $P_t$’s view composed of labels in real-world is indistinguishable from the ideal-world view, which comprises of simulated labels.

Privacy of $\mathsf {CSWA}$. In the offline, the view of $P_t$ in the real-world is computationally indistinguishable from the ideal-world view because of the semantic security of $\mathsf {AHE}$. Moreover, in the online, the real-world view of $P_t$ is also masked random values. $\mathsf {Sim}$ can simulate it with random values of the same size.

Therefore, we guarantee that the adversary $\mathcal {A}^s$ (when corrupts $P_0$) learns nothing beyond what can be inferred from the aggregated results ($\sum _{i=1}^K \langle \nu _i \widetilde{\mathbf {w}_i}\rangle ^\mathsf {A}_t$, $\sum _{i=1}^K \langle \nu _i\rangle ^\mathsf {A}_t$) with an overwhelming probability. Completing the proof.

C MA of ResNet-18 on CIFAR10 with Altering $\delta$

Fig. 8.

MA of ResNet-18 on CIFAR10 with $\delta =10\%$–$90\%$ for all Byzantine-robust aggregation methods, where 8(a) is for GA and 8(b) is for LF.

D Online Overhead of Free-HD and Private $\tau$-Clipping

Table 4. Comm. and Run-time of $\mathsf {Free-HD}$ and Private $\tau$-$\mathsf {Clipping}$.