Skip to main content
SpringerLink
Log in

Force: Highly Efficient Four-Party Privacy-Preserving Machine Learning on GPU

  • Conference paper
  • First Online:
Secure IT Systems (NordSec 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14324))

Included in the following conference series:

  • 284 Accesses

Abstract

Tremendous efforts have been made to improve the efficiency of secure Multi-Party Computation (MPC), which allows \(n \ge 2\) parties to jointly evaluate a target function without leaking their own private inputs. It has been confirmed by previous research that Three-Party Computation (3PC) and outsourcing computations to GPUs can lead to huge performance improvement of MPC in computationally intensive tasks such as Privacy-Preserving Machine Learning (PPML). A natural question to ask is whether super-linear performance gain is possible for a linear increase in resources. In this paper, we give an affirmative answer to this question. We propose \(\textsf{Force}\), an extremely efficient Four-Party Computation (4PC) system for PPML. To the best of our knowledge, each party in \(\textsf{Force}\) enjoys the least number of local computations, smallest graphic memory consumption and lowest data exchanges between parties. This is achieved by introducing a new sharing type \(\mathcal {X}\text {-}\textsf{share} \) along with MPC protocols in privacy-preserving training and inference that are semi-honest secure in the honest-majority setting. By comparing the results with state-of-the-art research, we showcase that \(\textsf{Force}\) is sound and extremely efficient, as it can improve the PPML performance by a factor of 2 to 38 compared with other latest GPU-based semi-honest secure systems, such as \(\textsf{Piranha}\) (including \(\textsf{SecureML}\), \(\textsf{Falcon}\), \(\textsf{FantasticFour}\)), \(\textsf{CryptGPU}\) and \(\textsf{CrypTen}\).

Y. Jiang—Main contributor.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Recall that generating zero sharing does not require parties to interact.

  2. 2.

    https://github.com/ucbrise/piranha/.

  3. 3.

    https://man7.org/linux/man-pages/man8/tc.8.html.

  4. 4.

    https://github.com/jeffreysijuntan/CryptGPU.

  5. 5.

    https://github.com/facebookresearch/CrypTen.

References

  1. Araki, T., Furukawa, J., Lindell, Y., Nof, A., Ohara, K.: High-throughput semi-honest secure three-party computation with an honest majority. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 805–817 (2016)

    Google Scholar 

  2. Bogdanov, D., Laur, S., Willemson, J.: Sharemind: a framework for fast privacy-preserving computations. In: Jajodia, S., Lopez, J. (eds.) ESORICS 2008. LNCS, vol. 5283, pp. 192–206. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88313-5_13

    Chapter  Google Scholar 

  3. Bogdanov, D., Niitsoo, M., Toft, T., Willemson, J.: High-performance secure multi-party computation for data mining applications. Int. J. Inf. Secur. 11(6), 403–418 (2012)

    Article  Google Scholar 

  4. Bonawitz, K.A., et al.: Practical secure aggregation for federated learning on user-held data. CoRR abs/1611.04482 (2016)

    Google Scholar 

  5. Byali, M., Chaudhari, H., Patra, A., Suresh, A.: Flash: fast and robust framework for privacy-preserving machine learning. Proc. Priv. Enh. Technol. 2, 459–480 (2020)

    Google Scholar 

  6. Cabrero-Holgueras, J., Pastrana, S.: SoK: privacy-preserving computation techniques for deep learning. Proc. Priv. Enh. Technol. 2021(4), 139–162 (2021)

    Google Scholar 

  7. Canetti, R.: Universally composable security: a new paradigm for cryptographic protocols. In: Proceedings 42nd IEEE Symposium on Foundations of Computer Science, pp. 136–145. IEEE (2001)

    Google Scholar 

  8. Catrina, O., de Hoogh, S.: Improved primitives for secure multiparty integer computation. In: Garay, J.A., De Prisco, R. (eds.) SCN 2010. LNCS, vol. 6280, pp. 182–199. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15317-4_13

    Chapter  MATH  Google Scholar 

  9. Chaudhari, H., Choudhury, A., Patra, A., Suresh, A.: Astra: high throughput 3PC over rings with application to secure prediction. In: Proceedings of the 2019 ACM SIGSAC Conference on Cloud Computing Security Workshop, pp. 81–92 (2019)

    Google Scholar 

  10. Chaudhari, H., Rachuri, R., Suresh, A.: Trident: efficient 4PC framework for privacy preserving machine learning. In: 27th Annual Network and Distributed System Security Symposium, NDSS 2020, San Diego, California, USA, 23–26 February 2020. The Internet Society (2020)

    Google Scholar 

  11. Dai, W., Sunar, B.: cuHE: a homomorphic encryption accelerator library. In: Pasalic, E., Knudsen, L.R. (eds.) BalkanCryptSec 2015. LNCS, vol. 9540, pp. 169–186. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-29172-7_11

    Chapter  Google Scholar 

  12. Dalskov, A., Escudero, D., Keller, M.: Fantastic four: honest-majority four-party secure computation with malicious security. In: 30th USENIX Security Symposium (USENIX Security 2021), pp. 2183–2200 (2021)

    Google Scholar 

  13. Demmler, D., Schneider, T., Zohner, M.: ABY-a framework for efficient mixed-protocol secure two-party computation. In: NDSS (2015)

    Google Scholar 

  14. Dwork, C., Roth, A., et al.: The algorithmic foundations of differential privacy. Found. Trends® Theor. Comput. Sci. 9(3–4), 211–407 (2014)

    Google Scholar 

  15. Escudero, D., Ghosh, S., Keller, M., Rachuri, R., Scholl, P.: Improved primitives for MPC over mixed arithmetic-binary circuits. In: Micciancio, D., Ristenpart, T. (eds.) CRYPTO 2020. LNCS, vol. 12171, pp. 823–852. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-56880-1_29

    Chapter  Google Scholar 

  16. Evans, D., Kolesnikov, V., Rosulek, M., et al.: A pragmatic introduction to secure multi-party computation. Found. Trends® Priv. Secur. 2(2–3), 70–246 (2018)

    Google Scholar 

  17. Frederiksen, T.K., Nielsen, J.B.: Fast and maliciously secure two-party computation using the GPU. In: Jacobson, M., Locasto, M., Mohassel, P., Safavi-Naini, R. (eds.) ACNS 2013. LNCS, vol. 7954, pp. 339–356. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38980-1_21

    Chapter  Google Scholar 

  18. Furukawa, J., Lindell, Y., Nof, A., Weinstein, O.: High-throughput secure three-party computation for malicious adversaries and an honest majority. In: Coron, J.-S., Nielsen, J.B. (eds.) EUROCRYPT 2017. LNCS, vol. 10211, pp. 225–255. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56614-6_8

    Chapter  Google Scholar 

  19. Google LLC: Tensorflow privacy (2019). https://github.com/tensorflow/privacy

  20. Hastings, M., Hemenway, B., Noble, D., Zdancewic, S.: SoK: general purpose compilers for secure multi-party computation. In: 2019 IEEE Symposium on Security and Privacy (SP), pp. 1220–1237. IEEE (2019)

    Google Scholar 

  21. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  22. Hesamifard, E., Takabi, H., Ghasemi, M., Wright, R.N.: Privacy-preserving machine learning as a service. Proc. Priv. Enh. Technol. 2018(3), 123–142 (2018)

    Google Scholar 

  23. Huang, Z., Lu, W.j., Hong, C., Ding, J.: Cheetah: lean and fast secure two-party deep neural network inference. In: 31st USENIX Security Symposium (USENIX Security 2022), pp. 809–826 (2022)

    Google Scholar 

  24. Husted, N., Myers, S., Shelat, A., Grubbs, P.: GPU and CPU parallelization of honest-but-curious secure two-party computation. In: Proceedings of the 29th Annual Computer Security Applications Conference, pp. 169–178 (2013)

    Google Scholar 

  25. Juvekar, C., Vaikuntanathan, V., Chandrakasan, A.: Gazelle: a low latency framework for secure neural network inference. In: 27th USENIX Security Symposium (USENIX Security 2018), pp. 1651–1669 (2018)

    Google Scholar 

  26. Kanagavelu, R., et al.: Two-phase multi-party computation enabled privacy-preserving federated learning. In: 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), pp. 410–419. IEEE (2020)

    Google Scholar 

  27. Keller, M., Pastro, V., Rotaru, D.: Overdrive: making SPDZ great again. In: Nielsen, J.B., Rijmen, V. (eds.) EUROCRYPT 2018. LNCS, vol. 10822, pp. 158–189. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78372-7_6

    Chapter  Google Scholar 

  28. Knott, B., Venkataraman, S., Hannun, A., Sengupta, S., Ibrahim, M., van der Maaten, L.: Crypten: secure multi-party computation meets machine learning. Adv. Neural. Inf. Process. Syst. 34, 4961–4973 (2021)

    Google Scholar 

  29. Koti, N., Pancholi, M., Patra, A., Suresh, A.: SWIFT: super-fast and robust privacy-preserving machine learning. In: 30th USENIX Security Symposium (USENIX Security 2021), pp. 2651–2668. USENIX Association (2021)

    Google Scholar 

  30. Koti, N., Patra, A., Rachuri, R., Suresh, A.: Tetrad: actively secure 4PC for secure training and inference. In: 29th Annual Network and Distributed System Security Symposium, NDSS 2022, San Diego, California, USA, 24–28 April 2022. The Internet Society (2022)

    Google Scholar 

  31. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    Google Scholar 

  32. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25. Curran Associates, Inc. (2012)

    Google Scholar 

  33. Le, Y., Yang, X.: Tiny imagenet visual recognition challenge. CS 231N 7(7), 3 (2015)

    Google Scholar 

  34. Lehmkuhl, R., Mishra, P., Srinivasan, A., Popa, R.A.: Muse: secure inference resilient to malicious clients. In: 30th USENIX Security Symposium (USENIX Security 2021), pp. 2201–2218 (2021)

    Google Scholar 

  35. Li, Y., Xu, W.: Privpy: general and scalable privacy-preserving data mining. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1299–1307 (2019)

    Google Scholar 

  36. Lindell, Y.: How to simulate it-a tutorial on the simulation proof technique. In: Tutorials on the Foundations of Cryptography, pp. 277–346 (2017)

    Google Scholar 

  37. Liu, J., Juuti, M., Lu, Y., Asokan, N.: Oblivious neural network predictions via minionn transformations. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 619–631 (2017)

    Google Scholar 

  38. Mishra, P., Lehmkuhl, R., Srinivasan, A., Zheng, W., Popa, R.A.: Delphi: a cryptographic inference service for neural networks. In: 29th USENIX Security Symposium (USENIX Security 2020), pp. 2505–2522 (2020)

    Google Scholar 

  39. Mohassel, P., Rindal, P.: ABY3: a mixed protocol framework for machine learning. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 35–52 (2018)

    Google Scholar 

  40. Mohassel, P., Zhang, Y.: Secureml: a system for scalable privacy-preserving machine learning. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 19–38. IEEE (2017)

    Google Scholar 

  41. Papernot, N., McDaniel, P., Sinha, A., Wellman, M.P.: SoK: security and privacy in machine learning. In: 2018 IEEE European Symposium on Security and Privacy (EuroS &P), pp. 399–414. IEEE (2018)

    Google Scholar 

  42. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  43. Patra, A., Schneider, T., Suresh, A., Yalame, H.: ABY2.0: improved mixed-protocol secure two-party computation. In: 30th USENIX Security Symposium (USENIX Security 2021), pp. 2165–2182 (2021)

    Google Scholar 

  44. Patra, A., Suresh, A.: BLAZE: blazing fast privacy-preserving machine learning. In: 27th Annual Network and Distributed System Security Symposium, NDSS 2020, San Diego, California, USA, 23–26 February 2020. The Internet Society (2020)

    Google Scholar 

  45. Pu, S., Duan, P., Liu, J.: Fastplay-a parallelization model and implementation of SMC on CUDA based GPU cluster architecture. IACR Cryptology ePrint Archive, p. 97 (2011)

    Google Scholar 

  46. Rathee, D., et al.: Cryptflow2: practical 2-party secure inference. In: Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 325–342 (2020)

    Google Scholar 

  47. Rotaru, D., Wood, T.: MArBled circuits: mixing arithmetic and boolean circuits with active security. In: Hao, F., Ruj, S., Sen Gupta, S. (eds.) INDOCRYPT 2019. LNCS, vol. 11898, pp. 227–249. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35423-7_12

    Chapter  Google Scholar 

  48. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  49. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)

    Google Scholar 

  50. Tan, S., Knott, B., Tian, Y., Wu, D.J.: Cryptgpu: fast privacy-preserving machine learning on the GPU. In: 2021 IEEE Symposium on Security and Privacy (SP), pp. 1021–1038. IEEE (2021)

    Google Scholar 

  51. Truex, S., et al.: A hybrid approach to privacy-preserving federated learning. In: Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, pp. 1–11 (2019)

    Google Scholar 

  52. Wagh, S., Gupta, D., Chandran, N.: Securenn: 3-party secure computation for neural network training. Proc. Priv. Enh. Technol. 2019(3), 26–49 (2019)

    Google Scholar 

  53. Wagh, S., Tople, S., Benhamouda, F., Kushilevitz, E., Mittal, P., Rabin, T.: Falcon: honest-majority maliciously secure framework for private deep learning. Proc. Priv. Enh. Technol. 1, 188–208 (2021)

    Google Scholar 

  54. Watson, J.L., Wagh, S., Popa, R.A.: Piranha: a GPU platform for secure computation. In: 31st USENIX Security Symposium (USENIX Security 2022), pp. 827–844 (2022)

    Google Scholar 

  55. Zhang, Q., et al.: MORSE-STF: a privacy preserving computation system. CoRR abs/2109.11726 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Huawei European Research Center, Munich, Germany

    Tianxiang Dai, Li Duan, Yufan Jiang, Yong Li, Fei Mei & Yulian Sun

  2. Karlsruhe Institute of Technology, Karlsruhe, Germany

    Yufan Jiang

Authors
  1. Tianxiang Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yufan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fei Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yulian Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Li .

Editor information

Editors and Affiliations

  1. OsloMet – Oslo Metropolitan University, Oslo, Norway

    Lothar Fritsch

  2. OsloMet – Oslo Metropolitan University, Oslo, Norway

    Ismail Hassan

  3. OsloMet - Oslo Metropolitan University, Oslo, Norway

    Ebenezer Paintsil

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dai, T., Duan, L., Jiang, Y., Li, Y., Mei, F., Sun, Y. (2024). Force: Highly Efficient Four-Party Privacy-Preserving Machine Learning on GPU. In: Fritsch, L., Hassan, I., Paintsil, E. (eds) Secure IT Systems. NordSec 2023. Lecture Notes in Computer Science, vol 14324. Springer, Cham. https://doi.org/10.1007/978-3-031-47748-5_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47748-5_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47747-8

  • Online ISBN: 978-3-031-47748-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation