Abstract
Generative Adversary Network (GAN) is a promising field with many practical applications. By using GANs, generated data can replace real sensitive data to be released for outside productive research. However, sometimes sensitive data is distributed among multiple parties, in which global generators are needed. Additionally, generated samples could remember or reflect sensitive features of real data. In this paper, we propose a scheme to aggregate a global generator from distributed local parties without access to local parties’ sensitive datasets, and the global generator will not reveal sensitive information of local parties’ training data. In our scheme, we separate GAN into two parts: discriminators played by local parties, a global generator played by the global party. Our scheme allows local parties to train different types of discriminators. To prevent generators from stealing sensitive information of real training datasets, we propose noised discriminator loss aggregation, add Gaussian noise to discriminators’ loss, then use the average of noised loss to compute global generator’s gradients and update its parameters. Our scheme is easy to implement by modifying plain GAN structures. We test our scheme on real-world MNIST and Fashion MNIST datasets, experimental results show that our scheme can achieve high-quality global generators without breaching local parties’ training data privacy.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318. ACM (2016)
Beaulieu-Jones, B.K., Wu, Z.S., Williams, C., Greene, C.S.: Privacy-preserving generative deep neural networks support clinical data sharing. Biorxiv 10, 159756 (2017)
Bonawitz, K., et al.: Practical secure aggregation for privacy-preserving machine learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1175–1191. ACM (2017)
Dwork, C., Lei, J.: Differential privacy and robust statistics. Stoc 9, 371–380 (2009)
Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006). https://doi.org/10.1007/11681878_14
Dwork, C., Roth, A., et al.: The algorithmic foundations of differential privacy. Found. Trends® Theor. Comput. Sci. 9(3–4), 211–407 (2014)
Dwork, C., Rothblum, G.N.: Concentrated differential privacy (2016). arXiv preprint arXiv:1603.01887
Dwork, C., Rothblum, G.N., Vadhan, S.: Boosting and differential privacy. In: 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, pp. 51–60. IEEE (2010)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in neural information processing systems, pp. 2672–2680 (2014)
Hardy, C., Le Merrer, E., Sericola, B.: Md-gan: Multi-discriminator generative adversarial networks for distributed datasets. In: 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 866–877. IEEE (2019)
Jordon, J., Yoon, J., van der Schaar, M.: Pate-gan: Generating synthetic data with differential privacy guarantees (2018)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Long, Y., Lin, S., Yang, Z., Gunter, C.A., Liu, H., Li, B.: Scalable differentially private data generation via private aggregation of teacher ensembles (2020). https://openreview.net/forum?id=Hkl6i0EFPH
Pathak, M., Rane, S., Raj, B.: Multiparty differential privacy via aggregation of locally trained classifiers. In: Advances in Neural Information Processing Systems, pp. 1876–1884 (2010)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks (2015). arXiv preprint arXiv:1511.06434
Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pp. 1310–1321. ACM (2015)
Torkzadehmahani, R., Kairouz, P., Paten, B.: Dp-cgan: Differentially private synthetic data and label generation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms (2017). arXiv preprint arXiv:1708.07747
Xie, L., Lin, K., Wang, S., Wang, F., Zhou, J.: Differentially private generative adversarial network (2018). arXiv preprint arXiv:1802.06739
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Han, C., Xue, R. (2020). Private Global Generator Aggregation from Different Types of Local Models. In: Park, N., Sun, K., Foresti, S., Butler, K., Saxena, N. (eds) Security and Privacy in Communication Networks. SecureComm 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 336. Springer, Cham. https://doi.org/10.1007/978-3-030-63095-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-63095-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63094-2
Online ISBN: 978-3-030-63095-9
eBook Packages: Computer ScienceComputer Science (R0)