Ranking Distance Metric for Privacy Budget in Distributed Learning of Finite Embedding Data

Abstract

Federated Learning (FL) is a collective of distributed learning paradigm that aims to preserve privacy in data. Recent studies have shown FL models to be vulnerable to reconstruction attacks that compromise data privacy by inverting gradients computed on confidential data. To address the challenge of defending against these attacks, it is common to employ methods that guarantee data confidentiality using the principles of Differential Privacy (DP). However, in many cases, especially for machine learning models trained on unstructured data such as text, evaluating privacy requires to consider also the finite space of embedding for client’s private data. In this study, we show how privacy in a distributed FL setup is sensitive to the underlying finite embeddings of the confidential data. We show that privacy can be quantified for a client batch that uses either noise, or a mixture of finite embeddings, by introducing a normalised rank distance ($d_{rank}$). This measure has the advantage of taking into account the size of a finite vocabulary embedding, and align the privacy budget to a partitioned space. We further explore the impact of noise and client batch size on the privacy budget and compare it to the standard $\epsilon$ derived from Local-DP.

Appendices

Appendix

Starting with LHS of 3 and substituting probability density function of Laplace distribution we get

$$\begin{aligned} \begin{aligned} \frac{\Pr (\textbf{M}(B, f, b))=y)}{\Pr (\textbf{M}(B', f, b))=y)} =\varPi _i \frac{\exp \big (-\frac{|y_i - \frac{\sum _{t}f({\textbf {x}}_t)_i }{|B|} |}{br}\big )}{\exp \big (-\frac{|y_i - \frac{\sum _{t'}f({\textbf {x}}_{t'})_i }{|B|} |}{br}\big )} \nonumber \\ =\varPi _i \frac{\exp \big (-\frac{ \frac{| |B| y_i -\sum _{t}f({\textbf {x}}_t)_i }{|B|} |}{br}\big )}{\exp \big (-\frac{ \frac{| |B| y_i -\sum _{t'}f({\textbf {x}}_{t'})_i }{|B|} |}{br}\big )} \nonumber \\ =\varPi _i \frac{\exp \big (- \frac{| |B| y_i -\sum _{t}f({\textbf {x}}_t)_i |}{|B|br}\big )}{\exp \big (- \frac{| |B| y_i -\sum _{t'}f({\textbf {x}}_{t'})_i | }{|B|br} \big )} \nonumber \\ = \varPi _i {\exp \big (\frac{||B|y_i - \sum _{t'}f({\textbf {x}}_{t'})_i| - ||B|y_i - \sum _{t}f({\textbf {x}}_{t})_i|}{|B|br}\big )} \nonumber \\ \le \varPi _i\exp {\big (\frac{| \sum _{t'}f({\textbf {x}}_{t'})_i - \sum _{t}f({\textbf {x}}_{t})_i |}{br|B|}\big ) } \nonumber \\ = \exp (\sum _i \big (\frac{| \sum _{t'}f({\textbf {x}}_{t'})_i - \sum _{t}f({\textbf {x}}_{t})_i |}{br|B|}\big ) \nonumber \\ = \exp (\epsilon ) \end{aligned} \end{aligned}$$

The first inequality follows from triangle inequality and in the last step $\epsilon$ is substituted for its definition.

Fig. 3.

From Fig. 1 The figure shows how the original text changes as we increase the Laplace noise we add to the embeddings and it maps the texts to the epsilon and d-rank. For the example we use a vocabulary of 1,000 tokens.

Fig. 4.

From Fig. 2 The figure shows how the original text changes as we increase the size of the batches (k) we add to the embeddings and it maps the texts to the epsilon and d-rank. For the example we use a vocabulary of 1,000 tokens.

Disclaimer

This paper was prepared for informational purposes by the Global Technology Applied Research center of JPMorgan Chase & Co. This paper is not a product of the Research Department of JPMorgan Chase & Co. or its affiliates. Neither JPMorgan Chase & Co. nor any of its affiliates makes any explicit or implied representation or warranty and none of them accept any liability in connection with this paper, including, without limitation, with respect to the completeness, accuracy, or reliability of the information contained herein and the potential legal, compliance, tax, or accounting effects thereof. This document is not intended as investment research or investment advice, or as a recommendation, offer, or solicitation for the purchase or sale of any security, financial instrument, financial product or service, or to be used in any way for evaluating the merits of participating in any transaction.