Abstract
Incredible capacity of machine learning models to mine the underlying information has led to concerns of privacy disclosure. This makes privacy-preserving learning algorithms become a hot spot. In this paper, we focus on Gaussian processes classification (GPC) with a provable secure and feasible privacy model, differential privacy (DP). First we apply a functional mechanism to design a basic privacy-preserving GP classifier. This involves finding the sensitivity of the outputs, and adding a Gaussian process noise proportional to the sensitivity to the trained classifier. Then we propose a variant-noise mechanism to perturb the classifier with different scaled noise based on the density of dataset. We show that this method can significantly reduce the added noise, whilst sufficiently maintaining the accuracy of the classifier both in theory and experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
When we say a datapoint is of high density, that means in its neighbourhood of a given radius, there are a relatively larger amount of data points than that of a datapoint with low density.
- 2.
References
Abadi, M., et al.: Deep learning with differential privacy. In: ACM Conference on Computer and Communications Security (CCS) (2016)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Quiñonero Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6, 1939–1959 (2005)
Chaudhuri, K., Monteleoni, C.: Privacy-preserving logistic regression. In: Advances in Neural Information Processing Systems (NIPS) (2008)
Chaudhuri, K., Monteleoni, C., Sarwate, A.D.: Differentially private empirical risk minimization. J. Mach. Learn. Res. 12, 1069–1109 (2011)
Damianou, A., Lawrence, N.: Deep Gaussian processes. In: International Conference on Artificial Intelligence and Statistics (AISTATS) (2013)
Dwork, C.: Differential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006). https://doi.org/10.1007/11787006_1
Dwork, C., Roth, A.: The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9(3–4), 211–407 (2014)
Ganta, S.R., Kasiviswanathan, S.P., Smith, A.: Composition attacks and auxiliary information in data privacy. In: ACM Conference on Knowledge Discovery and Data Mining (SIGKDD) (2008)
Hall, R., Rinaldo, A., Wasserman, L.A.: Differential privacy for functions and functional data. J. Mach. Learn. Res. 14(1), 703–727 (2013)
Lawrence, N.D.: Gaussian process latent variable models for visualisation of high dimensional data. In: Advances in Neural Information Processing Systems (NIPS) (2004)
Lawrence, N.D., Seeger, M., Herbrich, R.: Fast sparse Gaussian process methods: the informative vector machine. In: Advances in Neural Information Processing Systems (NIPS) (2002)
Lee, J., Sohl-dickstein, J., Pennington, J., Novak, R., Schoenholz, S., Bahri, Y.: Deep neural networks as Gaussian processes. In: International Conference on Learning Representations (ICLR) (2018)
Park, M., Foulds, J.R., Chaudhuri, K., Welling, M.: DP-EM: differentially private expectation maximization. In: International Conference on Artificial Intelligence and Statistics (AISTATS) (2017)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2005)
Smith, M.T., Álvarez, M., Zwiessele, M., Lawrence, N.D.: Differentially private regression with Gaussian processes. In: International Conference on Artificial Intelligence and Statistics (AISTATS) (2018)
Steffan, J., Schumacher, M.: Collaborative attack modeling. In: ACM Symposium on Applied Computing (SAC) (2002)
Acknowledgements
This work is supported by National Key Research and Development Program of China (No. 2018YFC0830400) and Shanghai Electric Vehicle Public Data Center.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Xiong, Z., Li, L., Yan, J., Wang, H., He, H., Jin, Y. (2019). Differential Privacy with Variant-Noise for Gaussian Processes Classification. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11672. Springer, Cham. https://doi.org/10.1007/978-3-030-29894-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-29894-4_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29893-7
Online ISBN: 978-3-030-29894-4
eBook Packages: Computer ScienceComputer Science (R0)