Abstract
Federated learning is a machine learning framework where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g. service provider), while keeping the training data decentralized. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the clients. Thus, federated learning could raise “unfairness” according to some fairness metrics. Differential privacy is a privacy model used to protect privacy in federated learning with bounded leakage about the presence of a specific point in the training data. Previous work showed that a reduction in accuracy induced by deep private models disproportionately impacts underrepresented groups. This motivates us to analyze the impact of differential privacy on model fairness in federated learning. In this work, we conduct extensive experiments to evaluate the impact of differential privacy on model fairness in federated learning. Experiments show that, with a proper choice of parameters, differential privacy might improve fairness with an ignoble reduction on accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318 (2016)
Avent, B., Korolova, A., Zeber, D., Hovden, T., Livshits, B.: Blender: enabling local search with a hybrid differential privacy model. J. Priv. Confidentiality 9(2), 747–764 (2017)
Bagdasaryan, E., Shmatikov, V.: Differential privacy has disparate impact on model accuracy. In: Advances in Neural Information Processing Systems, pp. 15479–15488 (2019)
Bilal Zafar, M., Valera, I., Gomez Rodriguez, M., Gummadi, K.P.: Fairness constraints: mechanisms for fair classification. In: Artificial Intelligence and Statistics, pp. 962–970 (2015)
Caldas, S., et al.: Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097 (2018)
Cummings, R., Gupta, V., Kimpara, D., Morgenstern, J.: On the compatibility of privacy and fairness. In: Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, pp. 309–315 (2019)
Dwork, C.: A firm foundation for private data analysis. Commun. ACM 54(1), 86–95 (2011)
Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006). https://doi.org/10.1007/11681878_14
Dwork, C., Roth, A.: The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9(3–4), 211–407 (2014)
Eckhouse, L., Lum, K., Conti-Cook, C., Ciccolini, J.: Layers of bias: a unified approach for understanding problems with risk assessment. Crim. Justice Behav. 46(2), 185–209 (2018)
Geyer, R.C., Klein, T., Nabi, M.: Differentially private federated learning: a client level perspective. arXiv preprint arXiv:1712.07557 (2017)
Gupta, O., Raskar, R.: Distributed learning of deep neural network over multiple agents. J. Netw. Comput. Appl. 116, 1–8 (2018)
Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. Adv. Neural Inf. Process. Syst. 29, 3315–3323 (2016)
Kairouz, P., et al.: Advances and open problems in federated learning. arXiv: Learning (2019)
Kasiviswanathan, S.P., Lee, H.K., Nissim, K., Raskhodnikova, S., Smith, A.: What can we learn privately? In: Proceedings of the 54th Annual Symposium on Foundations of Computer Science, pp. 531–540 (2008)
Li, T., Sanjabi, M., Smith, V.: Fair resource allocation in federated learning. arXiv: Learning (2019)
Madden, M., Rainie, L., Project, P.I.: Americans’ attitudes about privacy, security and surveillance
Madras, D., Creager, E., Pitassi, T., Zemel, R.: Learning adversarially fair and transferable representations. arXiv preprint arXiv:1802.06309 (2018)
Mcmahan, H.B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.Y.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282 (2016)
Mcmahan, H.B., Ramage, D., Talwar, K., Zhang, L.: Learning differentially private recurrent language models. arXiv preprint arXiv:1710.06963 (2017)
Mitchell, S., Potash, E., Barocas, S., D’Amour, A., Lum, K.: Prediction-based decisions and fairness: a catalogue of choices, assumptions, and definitions. arXiv preprint arXiv:1811.07867 (2018)
Tran, B., Li, J., Madry, A.: Spectral signatures in backdoor attacks. In: Advances in Neural Information Processing Systems, pp. 8000–8010 (2018)
Warner, S.L.: Randomized response: a survey technique for eliminating evasive answer bias. J. Am. Stat. Assoc. 60(309), 63–69 (1965)
Acknowledgement
The research was financially supported by National Natural Science Foundation of China (No.61972366), the Foundation of Key Laboratory of Network Assessment Technology, Chinese Academy of Sciences (No. KFKT2019-003), the Foundation of Guangxi Key Laboratory of Cryptography and Information Security (No. GCIS201913), and the Foundation of Guizhou Provincial Key Laboratory of Public Big Data (No. 2018BDKFJJ009, No. 2019BDKFJJ003, No. 2019BDKFJJ011).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gu, X., Zhu, T., Li, J., Zhang, T., Ren, W. (2020). The Impact of Differential Privacy on Model Fairness in Federated Learning. In: Kutyłowski, M., Zhang, J., Chen, C. (eds) Network and System Security. NSS 2020. Lecture Notes in Computer Science(), vol 12570. Springer, Cham. https://doi.org/10.1007/978-3-030-65745-1_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-65745-1_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65744-4
Online ISBN: 978-3-030-65745-1
eBook Packages: Computer ScienceComputer Science (R0)