Abstract
We propose a novel multi-party computation protocol for evaluating continuous real-valued functions with high numerical precision. Our method is based on approximations with Fourier series and uses at most two rounds of communication during the online phase. For the offline phase, we propose a trusted-dealer and honest-but-curious aided solution, respectively. We apply our algorithm to train a logistic regression classifier via a variant of Newton’s method (known as IRLS) to compute unbalanced classification problems that detect rare events and cannot be solved using previously proposed privacy-preserving optimization algorithms (e.g., based on piecewise-linear approximations of the sigmoid function). Our protocol is efficient as it can be implemented using standard quadruple-precision floating point arithmetic. We report multiple experiments and provide a demo application that implements our algorithm for training a logistic regression model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: Deep learning with differential privacy. CoRR, abs/1607.00133 (2016)
Aono, Y., Hayashi, T., Trieu Phong, L., Wang, L.: Privacy-preserving logistic regression with distributed data sources via homomorphic encryption. IEICE Trans. 99-D(8):2079–2089 (2016)
Araki, T., Furukawa, J., Lindell, Y., Nof, A., Ohara, K.: High-throughput semi-honest secure three-party computation with an honest majority. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, 24–28 October 2016, pp. 805–817 (2016)
Beaver, D.: Efficient multiparty protocols using circuit randomization. In: Feigenbaum, J. (ed.) CRYPTO 1991. LNCS, vol. 576, pp. 420–432. Springer, Heidelberg (1992). https://doi.org/10.1007/3-540-46766-1_34
Björck, A.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
Bogdanov, D., Laur, S., Willemson, J.: Sharemind: a framework for fast privacy-preserving computations. In: Jajodia, S., Lopez, J. (eds.) ESORICS 2008. LNCS, vol. 5283, pp. 192–206. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88313-5_13
Boura, C., Chillotti, I., Gama, N., Jetchev, D., Peceny, S., Petric, A.: High-precision privacy-preserving real-valued function evaluation. Cryptology ePrint Archive, Report 2017/1234 (2017). https://eprint.iacr.org/2017/1234
Boyd, J.: A comparison of numerical algorithms for Fourier extension of the first, second, and third kinds. J. Comput. Phys. 178(1), 118–160 (2002)
Boyd, J.: Fourier embedded domain methods: extending a function defined on an irregular region to a rectangle so that the extension is spatially periodic and \(c^{\infty }\). Appl. Math. Comput. 161(2), 591–597 (2005)
Boyd, J.: Asymptotic fourier coefficients for a C infinity bell (smoothed-“top-hat”) & the fourier extension problem. J. Sci. Comput. 29(1), 1–24 (2006)
Chaudhuri, K., Monteleoni, C.: Privacy-preserving logistic regression. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, 8–11 December 2008, vol. 21, pp. 289–296. Curran Associates Inc. (2008)
Cramer, R., Damgård, I., Nielsen, J.B.: Secure Multiparty Computation and Secret Sharing. Cambridge University Press, Cambridge (2015)
Damgård, I., Pastro, V., Smart, N., Zakarias, S.: Multiparty computation from somewhat homomorphic encryption. In: Safavi-Naini, R., Canetti, R. (eds.) CRYPTO 2012. LNCS, vol. 7417, pp. 643–662. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32009-5_38
Damgård, I., Pastro, V., Smart, N.P., Zakarias, S.: SPDZ Software. https://www.cs.bris.ac.uk/Research/CryptographySecurity/SPDZ/
Dataset, Arcene Data Set. https://archive.ics.uci.edu/ml/datasets/Arcene
Dataset, MNIST Database. http://yann.lecun.com/exdb/mnist/
Fefferman, C.: Interpolation and extrapolation of smooth functions by linear operators. Rev. Mat. Iberoamericana 21(1), 313–348 (2005)
Gascón, A., et al.: Privacy-preserving distributed linear regression on high-dimensional data. Proc. Priv. Enhancing Technol. 4, 248–267 (2017)
Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K.E., Naehrig, M., Wernsing, J.: CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In: Proceedings of the 33rd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, 19–24 June 2016, pp. 201–210 (2016)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.org
Hestenes, M.R.: Extension of the range of a differentiable function. Duke Math. J. 8, 183–192 (1941)
Huybrechs, D.: On the Fourier extension of nonperiodic functions. SIAM J. Numer. Anal. 47(6), 4326–4355 (2010)
Jäschke, A., Armknecht, F.: Accelerating homomorphic computations on rational numbers. In: Manulis, M., Sadeghi, A.-R., Schneider, S. (eds.) ACNS 2016. LNCS, vol. 9696, pp. 405–423. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39555-5_22
Lindell, Y., Pinkas, B.: Privacy preserving data mining. In: Bellare, M. (ed.) CRYPTO 2000. LNCS, vol. 1880, pp. 36–54. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-44598-6_3
Livni, R., Shalev-Shwartz, S., Shamir, O.: On the computational efficiency of training neural networks. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems: Annual Conference on Neural Information Processing Systems 2014, Montreal, Quebec, Canada, 8–13 December 2014, vol. 27, pp. 855–863 (2014)
Mohassel, P., Zhang, Y.: SecureML: a system for scalable privacy-preserving machine learning. In: 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, 22–26 May 2017, pp. 19–38. IEEE Computer Society (2017)
Nikolaenko, V., Weinsberg, U., Ioannidis, S., Joye, M., Boneh, D., Taft, N.: Privacy-preserving ridge regression on hundreds of millions of records. In: 2013 IEEE Symposium on Security and Privacy, SP 2013, Berkeley, CA, USA, 19–22 May 2013, pp. 334–348. IEEE Computer Society (2013)
Phong, L.T., Aono, Y., Hayashi, T., Wang, L., Moriai, S.: Privacy-preserving deep learning: revisited and enhanced. In: Batten, L., Kim, D.S., Zhang, X., Li, G. (eds.) ATIS 2017. CCIS, vol. 719, pp. 100–110. Springer, Singapore (2017). https://doi.org/10.1007/978-981-10-5421-1_9
Whitney, H.: Analytic extensions of differentiable functions defined in closed sets. Trans. Am. Math. Soc. 36(1), 63–89 (1934)
Wu, S., Teruya, T., Kawamoto, J., Sakuma, J., Kikuchi, H.: Privacy-preservation for stochastic gradient descent application to secure logistic regression. In: The 27th Annual Conference of the Japanese Society for Artificial Intelligence, vol. 27, pp. 1–4 (2013)
Acknowledgements
We thank Hunter Brooks, Daniel Kressner and Marco Picasso for useful conversations on data-independent iterative optimization algorithms. We are grateful to Jordan Brandt, Alexandre Duc and Morten Dahl for various useful discussions regarding multi-party computations and privacy-preserving machine learning.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A Timings for \(n=3\) players
A Timings for \(n=3\) players
We present in this section a table (Table 1) summarizing the different measures we obtained during our experiments for \(n = 3\) players. For this we considered datasets containing from 10000 to 1500000 points having 8, 12 or 20 features each.
Figure 1 shows the evolution of the cost function during the logistic regression as a function of the number of iterations, on a test dataset of 150000 samples, with 8 features and an acceptance rate of 0.5%. In yellow is the standard gradient descent with optimal learning rate, in red, the gradient descent using the piecewise linear approximation of the sigmoid function (as in [26]), and in green, our MPC model (based on the IRLS method). The MPC IRLS method (as well as the plaintext IRLS) method converge in less than 8 iterations, against 500 iterations for the standard gradient method. As expected, the approx method does not reach the minimal cost.
Evolution of the cost function depending on the method
Evolution of the F-score depending on the method
Figure 2 shows the evolution of the F-score during the same logistic regression as a function of the number of iterations. The standard gradient descent and our MPC produce the same model, with a limit F-score of 0.64. However, no positive samples are detected by the piecewise linear approximation, leading to a null F-score. However, in the three cases, the accuracy (purple) is nearly 100% from the first iteration.
Rights and permissions
Copyright information
© 2018 International Financial Cryptography Association
About this paper
Cite this paper
Boura, C., Chillotti, I., Gama, N., Jetchev, D., Peceny, S., Petric, A. (2018). High-Precision Privacy-Preserving Real-Valued Function Evaluation. In: Meiklejohn, S., Sako, K. (eds) Financial Cryptography and Data Security. FC 2018. Lecture Notes in Computer Science(), vol 10957. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-58387-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-662-58387-6_10
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-58386-9
Online ISBN: 978-3-662-58387-6
eBook Packages: Computer ScienceComputer Science (R0)