Abstract
Federated Learning (FL) enables multiple distributed local clients to coordinate with a central server to train a global model without sharing their private data. However, the data owned by different clients, even with the same label, may induce conflicts in the latent feature maps, especially under the non-IID FL scenarios. This would fatally impair the performance of the global model. To this end, we propose a novel approach, DAFL, for Dual Adversarial Federated Learning, to mitigate the divergence on latent feature maps among different clients on non-IID data. In particular, a local dual adversarial training is designed to identify the origins of latent feature maps, and then transforms the conflicting latent feature maps to reach a consensus between global and local models in each client. Besides, the latent feature maps of the two models become closer to each other adaptively by reducing their Kullback Leibler divergence. Extensive experiments on benchmark datasets validate the effectiveness of DAFL and also demonstrate that DAFL outperforms the state-of-the-art approaches in terms of test accuracy under different non-IID settings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arora, S., Ge, R., Liang, Y., Ma, T., Zhang, Y.: Generalization and equilibrium in generative adversarial nets (GANs). In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning, ICML, vol. 70, pp. 224–232. PMLR (2017)
Bonawitz, K.A., et al.: Practical secure aggregation for privacy-preserving machine learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS, pp. 1175–1191. ACM (2017)
Fallah, A., Mokhtari, A., Ozdaglar, A.E.: Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach (2020)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning, ICML, vol. 70, pp. 1126–1135. PMLR (2017)
Goodfellow, I.J., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680 (2014)
Guo, X., Liu, Z., Li, J., et al.: VeriFL: communication-efficient and fast verifiable aggregation for federated learning. IEEE Trans. Inf. Forensics Secur. 16, 1736–1751 (2021)
Kaissis, G., Makowski, M.R., Rueckert, D., Braren, R.: Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2(6), 305–311 (2020)
Karimireddy, S.P., et al.: SCAFFOLD: stochastic controlled averaging for federated learning. In: Proceedings of the 37th International Conference on Machine Learning, ICML, vol. 119, pp. 5132–5143. PMLR (2020)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Handbook Syst. Autoimmune Dis. 1(4), 7 (2009)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. In: Dhillon, I.S., Papailiopoulos, D.S., Sze, V. (eds.) Proceedings of Machine Learning and Systems 2020, MLSys. mlsys.org (2020)
Li, X., Huang, K., Yang, W., Wang, S., Zhang, Z.: On the convergence of FedAvg on Non-IID data. In: 8th International Conference on Learning Representations, ICLR. OpenReview.net (2020)
Liu, C., Shum, H.: Kullback-Leibler boosting. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 587–594. IEEE Computer Society (2003)
Liu, Y., et al.: FedVision: an online visual object detection platform powered by federated learning. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI. pp. 13172–13179. AAAI Press (2020)
Mathiassen, J.R., Skavhaug, A., Bø, K.: Texture similarity measure using Kullback-Leibler divergence between gamma distributions. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 133–147. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-47977-5_9
McMahan, B., et al.: Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTAT, vol. 54, pp. 1273–1282. PMLR (2017)
Qiu, H., Qiu, M., Lu, Z.: Selective encryption on ECG data in body sensor network based on supervised machine learning. Inf. Fusion 55, 59–67 (2020)
Qiu, M., Gai, K., Xiong, Z.: Privacy-preserving wireless communications using bipartite matching in social big data. Future Gener. Comput. Syst. 87, 772–781 (2018)
Qiu, M., Zhang, L., Ming, Z., Chen, Z., Qin, X., Yang, L.T.: Security-aware optimization for ubiquitous computing systems with SEAT graph approach. J. Comput. Syst. Sci. 79(5), 518–529 (2013)
Smith, V., et al.: Federated multi-task learning. In: Advances in Neural Information Processing Systems, vol. 30, pp. 4424–4434 (2017)
Wu, X., Liu, S., Zhou, Z.: Heterogeneous model reuse via optimizing multiparty multiclass margin. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, ICML, vol. 97, pp. 6840–6849. PMLR (2019)
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. CoRR abs/1708.07747 (2017)
Yang, Q., et al.: FLOP: federated learning on medical datasets using partial networks. In: KDD 2021: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, pp. 3845–3853. ACM (2021)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. 10(2), 12:1–12:19 (2019)
Zeng, D., Liang, S., Hu, X., Xu, Z.: FedLab: a flexible federated learning framework. CoRR abs/2107.11621 (2021)
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., Chandra, V.: Federated learning with Non-IID data. CoRR abs/1806.00582 (2018)
Acknowledgements
This work was supported in part by the National Key R &D Program of China (Grant No. 2018YFE0207600), the Key Research and Development Program of Shaanxi (Grant No. 2021ZDLGY07-05, 2019ZDLGY13-03-01), the Innovation Capability Support Program of Shaanxi(Grant No. 2020CGXNG-002), and the Fundamental Research Funds for the Central Universities (Grant No. JB210306).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, T., Yang, S., Song, A., Li, G., Dong, X. (2022). Dual Adversarial Federated Learning on Non-IID Data. In: Memmi, G., Yang, B., Kong, L., Zhang, T., Qiu, M. (eds) Knowledge Science, Engineering and Management. KSEM 2022. Lecture Notes in Computer Science(), vol 13370. Springer, Cham. https://doi.org/10.1007/978-3-031-10989-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-10989-8_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10988-1
Online ISBN: 978-3-031-10989-8
eBook Packages: Computer ScienceComputer Science (R0)