Abstract
For nowadays real-world data mining task, multiple data holders usually maintain the different feature part of the common data which is called as vertically partitioned data. Accompanying by the emerging demand of privacy persevering, it is hard to do data mining over this kind of vertically partitioned data by legacy machine learning methods. In consideration of less literature for non-linear learning over kernels, in this chapter, we propose a vertical federated kernel learning (VFKL) method to train over the vertically partitioned data. Specifically, we first approximate the kernel function by the random feature technique, and then federatedly update the predict function by the special designed doubly stochastic gradient without leaking privacy in both data and model. Theoretically, our VFKL could provide a sublinear convergence rate, and guarantee the security of data under the common semi-honest assumption. We conduct numerous experiments on various datasets to demonstrate the effectiveness and superiority of the proposed VFKL method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The framework code is available at https://github.com/FederatedAI/FATE.
- 3.
- 4.
Datasets can be found at https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/.
- 5.
- 6.
References
Badshah, N.: Facebook to contact 87 million users affected by data breach. The Guardian) (2018). https://www.theguardian.com/technology/2018/apr/08/facebook-to-contact-the-87-million-users-affected-by-data-breach
Berlinet, A., Thomas-Agnan, C.: Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer, New York (2011)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 1–27 (2011). Software. http://www.csie.ntu.edu.tw/~cjlin/libsvm
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Yang, Q.: SecureBoost: a lossless federated learning framework. arXiv preprint arXiv:1901.08755 (2019)
Dai, B., Xie, B., He, N., Liang, Y., Raj, A., Balcan, M.F.F., Song, L.: Scalable kernel methods via doubly stochastic gradients. In: Advances in Neural Information Processing Systems, pp. 3041–3049 (2014)
Dalcin, L.D., Paz, R.R., Kler, P.A., Cosimo, A.: Parallel distributed computing using Python. Adv. Water Resour. 34(9), 1124–1139 (2011)
Du, W., Atallah, M.J.: Privacy-preserving cooperative statistical analysis. In: Seventeenth Annual Computer Security Applications Conference, pp. 102–110. IEEE (2001)
EU: Regulation (EU) 2016/679 of the European parliament and of the council on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/EC (general data protection regulation) (2016). https://eur-lex.europa.eu/legal-content/EN/TXT
Gascón, A., et al.: Secure linear regression on vertically partitioned datasets. IACR Cryptology ePrint Archive 2016, 892 (2016)
Gascón, A., et al.: Privacy-preserving distributed linear regression on high-dimensional data. Proc. Priv. Enhancing Technol. 2017(4), 345–364 (2017)
Gu, B., Dang, Z., Li, X., Huang, H.: Federated doubly stochastic kernel learning for vertically partitioned data. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining on Applied Data Science Track. ACM (2020)
Gu, B., Huo, Z., Huang, H.: Asynchronous doubly stochastic group regularized learning. In: International Conference on Artificial Intelligence and Statistics, pp. 1791–1800 (2018)
Hardy, S., et al.: Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. arXiv preprint arXiv:1711.10677 (2017)
Karr, A.F., Lin, X., Sanil, A.P., Reiter, J.P.: Privacy-preserving analysis of vertically partitioned data using secure matrix products. J. Off. Stat. 25(1), 125 (2009)
Liu, Y., Liu, Y., Liu, Z., Zhang, J., Meng, C., Zheng, Y.: Federated forest. arXiv preprint arXiv:1905.10053 (2019)
Nock, R., et al.: Entity resolution and federated learning get a federated resolution. arXiv preprint arXiv:1803.04035 (2018)
Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: Advances in Neural Information Processing Systems, pp. 1177–1184 (2008)
Rahimi, A., Recht, B.: Weighted sums of random kitchen sinks: replacing minimization with randomization in learning. In: Advances in Neural Information Processing Systems, pp. 1313–1320 (2009)
Rudin, W.: Fourier Analysis on Groups, vol. 121967. Wiley, Hoboken (1962)
Sanil, A.P., Karr, A.F., Lin, X., Reiter, J.P.: Privacy preserving regression modelling via distributed computation. In: Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 677–682. ACM (2004)
Skillicorn, D.B., Mcconnell, S.M.: Distributed prediction from vertically partitioned data. J. Parallel Distrib. Comput. 68(1), 16–36 (2008)
Vaidya, J., Clifton, C.: Privacy preserving association rule mining in vertically partitioned data. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery And Data Mining, pp. 639–644. ACM (2002)
Vaidya, J., Clifton, C.: Privacy-preserving k-means clustering over vertically partitioned data. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 206–215 (2003)
Wan, L., Ng, W.K., Han, S., Lee, V.: Privacy-preservation for gradient descent methods. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 775–783. ACM (2007)
Xie, B., Liang, Y., Song, L.: Scale up nonlinear component analysis with doubly stochastic gradients. In: Advances in Neural Information Processing Systems, pp. 2341–2349 (2015)
Yang, J., Sindhwani, V., Fan, Q., Avron, H., Mahoney, M.W.: Random laplace feature maps for semigroup kernels on histograms. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 971–978 (2014)
Yu, H., Vaidya, J., Jiang, X.: Privacy-preserving SVM classification on vertically partitioned data. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS (LNAI), vol. 3918, pp. 647–656. Springer, Heidelberg (2006). https://doi.org/10.1007/11731139_74
Zhang, G.D., Zhao, S.Y., Gao, H., Li, W.J.: Feature-distributed SVRG for high-dimensional linear classification. arXiv preprint arXiv:1802.03604 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Dang, Z., Gu, B., Huang, H. (2020). Large-Scale Kernel Method for Vertical Federated Learning. In: Yang, Q., Fan, L., Yu, H. (eds) Federated Learning. Lecture Notes in Computer Science(), vol 12500. Springer, Cham. https://doi.org/10.1007/978-3-030-63076-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-63076-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63075-1
Online ISBN: 978-3-030-63076-8
eBook Packages: Computer ScienceComputer Science (R0)