Abstract
We present \(\epsilon \)-differentially private functional mechanisms for variants of regularised linear regression, LASSO, Ridge, and elastic net. We empirically and comparatively analyse their effectiveness. We quantify the error incurred by these \(\epsilon \)-differentially private functional mechanisms with respect to the non-private linear regression. We show that the functional mechanism is more effective than the state-of-art differentially private mechanism using input perturbation for the three main regularised linear regression models. We also discuss caveats in the functional mechanism, such as non-convexity of the noisy loss function, which causes instability in the results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Minnesota Population Center. Integrated public use microdata series - international: version 5.0 (2009). https://international.ipums.org
Dandekar, A., Basu, D., Bressan, S.: Differential privacy for regularised linear regression. Technical report TRA6/18, National University of Singapore, June 2018. https://dl.comp.nus.edu.sg/handle/1900.100/7051
Diamond, S., Boyd, S.: CVXPY: a Python-embedded modeling language for convex optimization. J. Mach. Learn. Res. 17(83), 1–5 (2016)
Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006). https://doi.org/10.1007/11681878_14
Dwork, C., Roth, A., et al.: The algorithmic foundations of differential privacy. Found. Trends\({\textregistered }\) Theoret. Comput. Sci. 9(3–4), 211–407 (2014)
Hall, R., Rinaldo, A., Wasserman, L.: Differential privacy for functions and functional data. J. Mach. Learn. Res. (JMLR) 14(Feb), 703–727 (2013)
Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics: The Approach Based on Influence Functions, vol. 196. Wiley, London (2011)
Hardt, M., Talwar, K.: On the geometry of differential privacy. In: Proceedings of the Forty-Second ACM Symposium on Theory of Computing (STOC). ACM (2010)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)
Lei, J.: Differentially private M-estimators. In: Advances in Neural Information Processing Systems, pp. 361–369 (2011)
Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press, Cambridge (2012)
O’Donoghue, B., Chu, E., Parikh, N., Boyd, S.: Conic optimization via operator splitting and homogeneous self-dual embedding. J. Optim. Theory Appl. 169, 1042–1068 (2016)
Talwar, K., Thakurta, A.G., Zhang, L.: Nearly optimal private LASSO. In: Advances in Neural Information Processing Systems (NIPS), pp. 3025–3033 (2015)
Thomas, G.B., Weir, M.D., Hass, J.: Thomas Calculus. Addison-Wesley, Reading (2016)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soci. Ser. B (Methodol.) 58, 267–288 (1996)
Yu, F., Rybar, M., Uhler, C., Fienberg, S.E.: Differentially-private logistic regression for detecting multiple-SNP association in GWAS databases. In: Domingo-Ferrer, J. (ed.) PSD 2014. LNCS, vol. 8744, pp. 170–184. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11257-2_14
Zhang, J., Zhang, Z., Xiao, X., Yang, Y., Winslett, M.: Functional mechanism: regression analysis under differential privacy. Proceed. VLDB Endow. 5(11), 1364–1375 (2012)
Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soci.: Ser. B (Stat. Methodol.) 67(2), 301–320 (2005)
Acknowledgement
This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore, under its Corporate Laboratory@University Scheme, National University of Singapore, and Singapore Telecommunications Ltd.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Dandekar, A., Basu, D., Bressan, S. (2018). Differential Privacy for Regularised Linear Regression. In: Hartmann, S., Ma, H., Hameurlain, A., Pernul, G., Wagner, R. (eds) Database and Expert Systems Applications. DEXA 2018. Lecture Notes in Computer Science(), vol 11030. Springer, Cham. https://doi.org/10.1007/978-3-319-98812-2_44
Download citation
DOI: https://doi.org/10.1007/978-3-319-98812-2_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98811-5
Online ISBN: 978-3-319-98812-2
eBook Packages: Computer ScienceComputer Science (R0)