Skip to main content
SpringerLink
Log in

Differential Privacy for Free? Harnessing the Noise in Approximate Homomorphic Encryption

  • Conference paper
  • First Online:
Topics in Cryptology – CT-RSA 2024 (CT-RSA 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14643))

Included in the following conference series:

  • 53 Accesses

Abstract

Homomorphic Encryption (HE) is a type of cryptography that allows computing on encrypted data, enabling computation on sensitive data to be outsourced securely. Many popular HE schemes rely on noise for their security. On the other hand, Differential Privacy (DP) seeks to guarantee the privacy of data subjects by obscuring any one individual’s contribution to an output. Many mechanisms for achieving DP involve adding appropriate noise. In this work, we investigate the extent to which the noise native to Homomorphic Encryption can provide Differential Privacy “for free”.

We identify the dependence of HE noise on the underlying data as a critical barrier to privacy, and derive new results on the Differential Privacy under this constraint. We apply these ideas to a proof of concept HE application, ridge regression training using gradient descent, and are able to achieve privacy budgets of \(\varepsilon \approx 2\) after 50 iterations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    More practical use cases might include [40, 41], which use degree 3, 5, or 7 approximations to the sigmoid function.

  2. 2.

    For TFHE and related schemes [16, 17], a so-called “average case”, or variance tracking, approach is more common – see for example [15, 18, 42].

  3. 3.

    In this work, for simplicity we do not train a constant weight \(\beta _0\).

  4. 4.

    This bound applies unconditionally to the minimum of the cost function – see [45]. However, in our case study we will only evaluate a fixed number of iterations of gradient descent, and so cannot assume we converge to the minimum.

  5. 5.

    Indeed, high precision constants are used, requiring an additional rescale, as well as multiplying by 1-hot masks to compensate for the feature by feature encoding. By contrast, our method uses 1 level in precomputation of \(M_{jk},Y_j\), and then one multiplication per iteration.

  6. 6.

    We use the formula from [1] as we bound noise in the canonical embedding.

References

  1. Openfhe noise flooding. https://github.com/openfheorg/openfhe-development/blob/main/src/pke/examples/CKKS_NOISE_FLOODING.md. Accessed 21 Jan 2024

  2. Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318 (2016)

    Google Scholar 

  3. Akavia, A., Leibovich, M., Resheff, Y.S., Ron, R., Shahar, M., Vald, M.: Privacy-preserving decision trees training and prediction. ACM Trans. Priv. Secur. 25(3), 1–30 (2022)

    Article  Google Scholar 

  4. Boemer, F., Cammarota, R., Demmler, D., Schneider, T., Yalame, H.: MP2ML: a mixed-protocol machine learning framework for private inference. In: Proceedings of the 15th International Conference on Availability, Reliability and Security, pp. 1–10 (2020)

    Google Scholar 

  5. Bossuat, J.P., Troncoso-Pastoriza, J., Hubaux, J.P.: Bootstrapping for approximate homomorphic encryption with negligible failure-probability by using sparse-secret encapsulation. In: Ateniese, G., Venturi, D. (eds.) ACNS 2022. LNCS, vol. 13269, pp. 521–541. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-09234-3_26

    Chapter  Google Scholar 

  6. Brakerski, Z., Gentry, C., Vaikuntanathan, V.: (Leveled) fully homomorphic encryption without bootstrapping. ACM Trans. Comput. Theory (TOCT) 6(3), 1–36 (2014)

    Article  MathSciNet  Google Scholar 

  7. Brakerski, Z., Vaikuntanathan, V.: Efficient fully homomorphic encryption from (standard) LWE. SIAM J. Comput. 43(2), 831–871 (2014)

    Article  MathSciNet  Google Scholar 

  8. Castryck, W., Iliashenko, I., Vercauteren, F.: On error distributions in ring-based LWE. LMS J. Comput. Math. 19(A), 130–145 (2016). https://doi.org/10.1112/S1461157016000280

    Article  MathSciNet  Google Scholar 

  9. Chaudhuri, K., Monteleoni, C., Sarwate, A.D.: Differentially private empirical risk minimization. J. Mach. Learn. Res. 12(29), 1069–1109 (2011). http://jmlr.org/papers/v12/chaudhuri11a.html

  10. Chen, H., Dai, W., Kim, M., Song, Y.: Efficient multi-key homomorphic encryption with packed ciphertexts with application to oblivious neural network inference. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 395–412 (2019)

    Google Scholar 

  11. Cheon, J.H., Han, K., Kim, A., Kim, M., Song, Y.: Bootstrapping for approximate homomorphic encryption. In: Nielsen, J.B., Rijmen, V. (eds.) EUROCRYPT 2018. LNCS, vol. 10820, pp. 360–384. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78381-9_14

    Chapter  Google Scholar 

  12. Cheon, J.H., Han, K., Kim, A., Kim, M., Song, Y.: A full RNS variant of approximate homomorphic encryption. In: Cid, C., Jacobson, M., Jr. (eds.) SAC 2018. LNCS, vol. 11349, pp. 347–368. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-10970-7_16

    Chapter  Google Scholar 

  13. Cheon, J.H., Hong, S., Kim, D.: Remark on the security of CKKS scheme in practice. Cryptology ePrint Archive (2020)

    Google Scholar 

  14. Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 409–437. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70694-8_15

    Chapter  Google Scholar 

  15. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Faster packed homomorphic operations and efficient circuit bootstrapping for TFHE. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 377–408. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70694-8_14

    Chapter  Google Scholar 

  16. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: TFHE: fast fully homomorphic encryption over the torus. J. Cryptol. 33(1), 34–91 (2020)

    Article  MathSciNet  Google Scholar 

  17. Chillotti, I., Joye, M., Ligier, D., Orfila, J.B., Tap, S.: CONCRETE: concrete operates on ciphertexts rapidly by extending TFHE. In: WAHC 2020-8th Workshop on Encrypted Computing & Applied Homomorphic Cryptography (2020)

    Google Scholar 

  18. Chillotti, I., Ligier, D., Orfila, J.-B., Tap, S.: Improved programmable bootstrapping with larger precision and efficient arithmetic circuits for TFHE. In: Tibouchi, M., Wang, H. (eds.) ASIACRYPT 2021. LNCS, vol. 13092, pp. 670–699. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92078-4_23

    Chapter  Google Scholar 

  19. Costache, A., Curtis, B.R., Hales, E., Murphy, S., Ogilvie, T., Player, R.: On the precision loss in approximate homomorphic encryption. In: Carlet, C., Kalikinkar Mandal, V.R. (eds.) SAC 2023. LNCS, vol. 14201, pp. 325–345. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-53368-6_16

    Chapter  Google Scholar 

  20. Costache, A., Laine, K., Player, R.: Evaluating the effectiveness of heuristic worst-case noise analysis in FHE. In: Chen, L., Li, N., Liang, K., Schneider, S. (eds.) ESORICS 2020. LNCS, vol. 12309, pp. 546–565. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59013-0_27

    Chapter  Google Scholar 

  21. Costache, A., Nürnberger, L., Player, R.: Optimisations and tradeoffs for HElib. In: Rosulek, M. (ed.) CT-RSA 2023. LNCS, vol. 13871, pp. 29–53. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-30872-7_2

    Chapter  Google Scholar 

  22. Ding, J., Zhang, X., Li, X., Wang, J., Yu, R., Pan, M.: Differentially private and fair classification via calibrated functional mechanism. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 622–629 (2020)

    Google Scholar 

  23. Ducas, L., Micciancio, D.: FHEW: bootstrapping homomorphic encryption in less than a second. In: Oswald, E., Fischlin, M. (eds.) EUROCRYPT 2015. LNCS, vol. 9056, pp. 617–640. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46800-5_24

    Chapter  Google Scholar 

  24. Dwork, C., Roth, A., et al.: The algorithmic foundations of differential privacy. Found. Trends® Theor. Comput. Sci. 9(3–4), 211–407 (2014)

    MathSciNet  Google Scholar 

  25. Dwork, C., Rothblum, G.N., Vadhan, S.: Boosting and differential privacy. In: 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, pp. 51–60. IEEE (2010)

    Google Scholar 

  26. Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. Cryptology ePrint Archive (2012)

    Google Scholar 

  27. Fukuchi, K., Tran, Q.K., Sakuma, J.: Differentially private empirical risk minimization with input perturbation. In: Yamamoto, A., Kida, T., Uno, T., Kuboyama, T. (eds.) DS 2017. LNCS (LNAI), vol. 10558, pp. 82–90. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67786-6_6

    Chapter  Google Scholar 

  28. Gaboardi, M., Lim, H., Rogers, R., Vadhan, S.: Differentially private chi-squared hypothesis testing: goodness of fit and independence testing. In: International Conference on Machine Learning, pp. 2111–2120. PMLR (2016)

    Google Scholar 

  29. Gentry, C.: Fully homomorphic encryption using ideal lattices. In: Proceedings of the Forty-First Annual ACM Symposium on Theory of Computing, STOC 2009, pp. 169–178. Association for Computing Machinery, New York (2009). https://doi.org/10.1145/1536414.1536440

  30. Gentry, C., Sahai, A., Waters, B.: Homomorphic encryption from learning with errors: conceptually-simpler, asymptotically-faster, attribute-based. In: Canetti, R., Garay, J.A. (eds.) CRYPTO 2013. LNCS, vol. 8042, pp. 75–92. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40041-4_5

    Chapter  Google Scholar 

  31. Hardt, M., Recht, B., Singer, Y.: Train faster, generalize better: stability of stochastic gradient descent. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning, ICML 2016, vol. 48, pp. 1225–1234. JMLR.org (2016)

    Google Scholar 

  32. Heaan v1.0. Online (2018). https://github.com/snucrypto/HEAAN/releases/tag/1.0

  33. Jain, P., Thakurta, A.: Differentially private learning with kernels. In: Dasgupta, S., McAllester, D. (eds.) Proceedings of the 30th International Conference on Machine Learning. Proceedings of Machine Learning Research, Atlanta, Georgia, USA, vol. 28, pp. 118–126. PMLR (2013). https://proceedings.mlr.press/v28/jain13.html

  34. Jayaraman, B., Evans, D.: Evaluating differentially private machine learning in practice. In: 28th USENIX Security Symposium (USENIX Security 2019), Santa Clara, CA, pp. 1895–1912. USENIX Association (2019). https://www.usenix.org/conference/usenixsecurity19/presentation/jayaraman

  35. Jayaraman, B., Wang, L., Evans, D., Gu, Q.: Distributed learning without distress: privacy-preserving empirical risk minimization. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc. (2018). https://proceedings.neurips.cc/paper/2018/file/7221e5c8ec6b08ef6d3f9ff3ce6eb1d1-Paper.pdf

  36. Jiang, X., Kim, M., Lauter, K., Song, Y.: Secure outsourced matrix computation and application to neural networks. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 1209–1222 (2018)

    Google Scholar 

  37. Kang, Y., Liu, Y., Niu, B., Tong, X., Zhang, L., Wang, W.: Input perturbation: a new paradigm between central and local differential privacy (2020). https://doi.org/10.48550/ARXIV.2002.08570. https://arxiv.org/abs/2002.08570

  38. Kifer, D., Smith, A., Thakurta, A.: Private convex empirical risk minimization and high-dimensional regression. In: Mannor, S., Srebro, N., Williamson, R.C. (eds.) Proceedings of the 25th Annual Conference on Learning Theory. Proceedings of Machine Learning Research, Edinburgh, Scotland, vol. 23, pp. 25.1–25.40. PMLR (2012). https://proceedings.mlr.press/v23/kifer12.html

  39. Kim, A., Papadimitriou, A., Polyakov, Y.: Approximate homomorphic encryption with reduced approximation error. In: Galbraith, S.D. (ed.) CT-RSA 2022. LNCS, vol. 13161, pp. 120–144. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-95312-6_6

    Chapter  Google Scholar 

  40. Kim, A., Song, Y., Kim, M., Lee, K., Cheon, J.H.: Logistic regression model training based on the approximate homomorphic encryption. BMC Med. Genomics 11(4), 23–31 (2018)

    Google Scholar 

  41. Kim, M., Song, Y., Wang, S., Xia, Y., Jiang, X., et al.: Secure logistic regression based on homomorphic encryption: design and evaluation. JMIR Med. Inform. 6(2), e8805 (2018)

    Article  Google Scholar 

  42. Klemsa, J.: Setting up efficient TFHE parameters for multivalue plaintexts and multiple additions. Cryptology ePrint Archive (2021)

    Google Scholar 

  43. Li, B., Micciancio, D.: On the security of homomorphic encryption on approximate numbers. In: Canteaut, A., Standaert, F.-X. (eds.) EUROCRYPT 2021. LNCS, vol. 12696, pp. 648–677. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77870-5_23

    Chapter  Google Scholar 

  44. Li, B., Micciancio, D., Schultz, M., Sorrell, J.: Securing approximate homomorphic encryption using differential privacy. In: Dodis, Y., Shrimpton, T. (eds.) CRYPTO 2022. LNCS, vol. 13507, pp. 560–589. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-15802-5_20

    Chapter  Google Scholar 

  45. Ligett, K., Neel, S., Roth, A., Waggoner, B., Wu, Z.S.: Accuracy first: selecting a differential privacy level for accuracy-constrained ERM. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, Red Hook, NY, USA, pp. 2563–2573. Curran Associates Inc. (2017)

    Google Scholar 

  46. Lyubashevsky, V., Peikert, C., Regev, O.: On ideal lattices and learning with errors over rings. J. ACM (JACM) 60(6), 1–35 (2013)

    Article  MathSciNet  Google Scholar 

  47. Lyubashevsky, V., Peikert, C., Regev, O.: A toolkit for ring-LWE cryptography. In: Johansson, T., Nguyen, P.Q. (eds.) EUROCRYPT 2013. LNCS, vol. 7881, pp. 35–54. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38348-9_3

    Chapter  Google Scholar 

  48. Ma, J., Naas, S.A., Sigg, S., Lyu, X.: Privacy-preserving federated learning based on multi-key homomorphic encryption. Int. J. Intell. Syst. 37(9), 5880–5901 (2022)

    Article  Google Scholar 

  49. Murphy, S., Player, R.: A central limit framework for ring-LWE decryption. Cryptology ePrint Archive (2019)

    Google Scholar 

  50. Ogilvie, T., Player, R., Rowell, J.: Improved privacy-preserving training using fixed-hessian minimisation. In: Brenner, M., Lepoint, T. (eds.) Proceedings of the 8th Workshop on Encrypted Computing and Applied Homomorphic Cryptography (WAHC 2020) (2020). https://doi.org/10.25835/0072999

  51. Papernot, N., Abadi, M., Erlingsson, U., Goodfellow, I., Talwar, K.: Semi-supervised knowledge transfer for deep learning from private training data. arXiv preprint arXiv:1610.05755 (2016)

  52. Phong, L.T., Aono, Y., Hayashi, T., Wang, L., Moriai, S.: Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13(5), 1333–1345 (2018). https://doi.org/10.1109/TIFS.2017.2787987

    Article  Google Scholar 

  53. Polyakov, Y., Rohloff, K., Ryan, G.W.: Palisade lattice cryptography library user manual (2017)

    Google Scholar 

  54. Raisaro, J.L., et al.: Protecting privacy and security of genomic data in i2b2 with homomorphic encryption and differential privacy. IEEE/ACM Trans. Comput. Biol. Bioinf. 15(5), 1413–1426 (2018). https://doi.org/10.1109/TCBB.2018.2854782

    Article  Google Scholar 

  55. Regev, O.: On lattices, learning with errors, random linear codes, and cryptography. J. ACM 56(6), 1–40 (2009). https://doi.org/10.1145/1568318.1568324

    Article  MathSciNet  Google Scholar 

  56. Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1310–1321 (2015)

    Google Scholar 

  57. Song, S., Chaudhuri, K., Sarwate, A.D.: Stochastic gradient descent with differentially private updates. In: 2013 IEEE Global Conference on Signal and Information Processing, pp. 245–248. IEEE (2013)

    Google Scholar 

  58. Tang, P., Wang, W., Gu, X., Lou, J., Xiong, L., Li, M.: Two birds, one stone: achieving both differential privacy and certified robustness for pre-trained classifiers via input perturbation (2021)

    Google Scholar 

  59. Tang, X., Zhu, L., Shen, M., Du, X.: When homomorphic cryptosystem meets differential privacy: training machine learning classifier with privacy protection. arXiv preprint arXiv:1812.02292 (2018)

  60. Triastcyn, A., Faltings, B.: Federated learning with Bayesian differential privacy. In: 2019 IEEE International Conference on Big Data (Big Data), pp. 2587–2596. IEEE (2019)

    Google Scholar 

  61. Wu, X., Li, F., Kumar, A., Chaudhuri, K., Jha, S., Naughton, J.: Bolt-on differential privacy for scalable stochastic gradient descent-based analytics. In: Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD 2017, pp. 1307–1322. Association for Computing Machinery, New York (2017). https://doi.org/10.1145/3035918.3064047

  62. Zhang, J., Zheng, K., Mou, W., Wang, L.: Efficient private ERM for smooth objectives. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI 2017, pp. 3922–3928. AAAI Press (2017)

    Google Scholar 

  63. Zhang, J., Zhang, Z., Xiao, X., Yang, Y., Winslett, M.: Functional mechanism: regression analysis under differential privacy. Proc. VLDB Endow. 5(11), 1364–1375 (2012). https://doi.org/10.14778/2350229.2350253

    Article  Google Scholar 

  64. Zhang, T., Zhu, T., Gao, K., Zhou, W., Philip, S.Y.: Balancing learning model privacy, fairness, and accuracy with early stopping criteria. IEEE Trans. Neural Netw. Learn. Syst. 34(9), 5557–5569 (2021)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We thank Fernando Virdia for his invaluable suggestions and discussions in the development of this work, including detailed comments on an early draft of this paper.

Author information

Authors and Affiliations

  1. Intel Labs, London, UK

    Tabitha Ogilvie

Authors
  1. Tabitha Ogilvie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tabitha Ogilvie .

Editor information

Editors and Affiliations

  1. University of Klagenfurt, Klagenfurt, Austria

    Elisabeth Oswald

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ogilvie, T. (2024). Differential Privacy for Free? Harnessing the Noise in Approximate Homomorphic Encryption. In: Oswald, E. (eds) Topics in Cryptology – CT-RSA 2024. CT-RSA 2024. Lecture Notes in Computer Science, vol 14643. Springer, Cham. https://doi.org/10.1007/978-3-031-58868-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-58868-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-58867-9

  • Online ISBN: 978-3-031-58868-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation