Abstract
In large-scale machine learning, federated learning (FL) is considered as a promising paradigm to address the problem of data privacy breach. Previous works have focused on improving fairness in terms of fair resource allocation. However, this is not sufficient considering that federated learning is essentially distributed training with average aggregation. Because low-contributing nodes, even if they are assigned more computational resources, are diluted by large-weighted nodes in aggregation. In particular, fair resource allocation for sophisticated systems is not realistic for real scenarios. In this paper, we propose FedHF, a new hierarchical fair federated learning framework with robust convergence and high fairness. FedHF improves upon naive combinations of federated learning and fair resource allocation with a hierarchy-based optimization of client selection algorithm and a conflict elimination method for fairness and discriminatory incentives. Through extensive experimental validation of our approach, we show that FedHF outperforms previous state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
Sattler, F., Wiedemann, S., Müller, K.R., Samek, W.: Robust and communication-efficient federated learning from non-IID data. IEEE Trans. Neural Netw. Learn. Syst. 31(9), 3400–3413 (2019)
Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: concept and applications. ACM Trans. Intell. Syst. Technol. 10(2) (2019)
Li, T., Sanjabi, M., Beirami, A., Smith, V.: Fair resource allocation in federated learning. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020 (2020). OpenReview.net
Li, T., Hu, S., Beirami, A., Smith, V.: Ditto: fair and robust federated learning through personalization. In: International Conference on Machine Learning, pp. 6357–6368. PMLR (2021)
Rehman, M.H.U., Dirir, A.M., Salah, K., Damiani, E., Svetinovic, D.: TrustFed: a framework for fair and trustworthy cross-device federated learning in IIoT. IEEE Trans. Ind. Inf. 17(12), 8485–8494 (2021)
Hashimoto, T., Srivastava, M., Namkoong, H., Liang, P.: Fairness without demographics in repeated loss minimization. In: International Conference on Machine Learning, pp. 1929–1938. PMLR (2018)
Wang, Z., Fan, X., Qi, J., Wen, C., Wang, C., Yu, R.: Federated learning with fair averaging. In: Zhou, Z. (ed.) Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event/Montreal, Canada, 19–27 August 2021, pp. 1615–1623 (2021). ijcai.org
Wu, W., He, L., Lin, W., Mao, R., Maple, C., Jarvis, S.A.: SAFA: a semi-asynchronous protocol for fast federated learning with low overhead. IEEE Trans. Comput. 70, 655–668 (2020)
Chen, Y., Ning, Y., Slawski, M., Rangwala, H.: Asynchronous online federated learning for edge devices with non-IID data. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 15–24. IEEE (2020)
Huang, W., Li, T., Wang, D., Du, S., Zhang, J., Huang, T.: Fairness and accuracy in horizontal federated learning. Inf. Sci. 589, 170–185 (2022)
Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., Finn, C.: Gradient surgery for multi-task learning. Adv. Neural. Inf. Process. Syst. 33, 5824–5836 (2020)
Li, J., He, L., Ren, S., Mao, R.: Developing a loss prediction-based asynchronous stochastic gradient descent algorithm for distributed training of deep neural networks. In: 49th International Conference on Parallel Processing-ICPP, pp. 1–10 (2020)
Farajtabar, M., Azizan, N., Mott, A., Li, A.: Orthogonal gradient descent for continual learning. In: International Conference on Artificial Intelligence and Statistics, pp. 3762–3773. PMLR (2020)
Guo, Y., Liu, M., Yang, T., Rosing, T.: Learning with long-term remembering: following the lead of mixed stochastic gradient (2019)
Désidéri, J.A.: Multiple-gradient descent algorithm (MGDA) for multiobjective optimization. C.R. Math. 350(5–6), 313–318 (2012)
Wang, Q., Du, X., Gao, Z., Guizani, M.: An optimal channel occupation time adjustment method for LBE in unlicensed spectrum. IEEE Trans. Veh. Technol. 68(11), 10943–10955 (2019)
Krishna, K., Murty, M.N.: Genetic k-means algorithm. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 29(3), 433–439 (1999)
Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: NetVLAD: CNN architecture for weakly supervised place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5297–5307 (2016)
Kairouz, P., et al.: Advances and open problems in federated learning. Found. Trends® Mach. Learn. 14(1–2), 1–210 (2021)
Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. Proc. Mach. Learn. Syst. 2, 429–450 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, Z., Duan, Y., Yang, Y., Rui, L., Zhao, C. (2022). FedHF: A High Fairness Federated Learning Algorithm Based on Deconfliction in Heterogeneous Networks. In: Troya, J., Medjahed, B., Piattini, M., Yao, L., Fernández, P., Ruiz-Cortés, A. (eds) Service-Oriented Computing. ICSOC 2022. Lecture Notes in Computer Science, vol 13740. Springer, Cham. https://doi.org/10.1007/978-3-031-20984-0_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-20984-0_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20983-3
Online ISBN: 978-3-031-20984-0
eBook Packages: Computer ScienceComputer Science (R0)