Abstract
While fairness-aware machine learning algorithms have been receiving increasing attention, the focus has been on centralized machine learning, leaving decentralized methods underexplored. Federated Learning is a decentralized form of machine learning where clients train local models with a server aggregating them to obtain a shared global model. Data heterogeneity amongst clients is a common characteristic of Federated Learning, which may induce or exacerbate discrimination of unprivileged groups defined by sensitive attributes such as race or gender. In this work we propose FAIR-FATE: a novel FAIR FederATEd Learning algorithm that aims to achieve group fairness while maintaining high utility via a fairness-aware aggregation method that computes the global model by taking into account the fairness of the clients. To achieve that, the global model update is computed by estimating a fair model update using a Momentum term that helps to overcome the oscillations of non-fair gradients. To the best of our knowledge, this is the first approach in machine learning that aims to achieve fairness using a fair Momentum estimate. Experimental results on real-world datasets demonstrate that FAIR-FATE outperforms state-of-the-art fair Federated Learning algorithms under different levels of data heterogeneity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For the sake of simplicity, we chose to randomly select a portion of the dataset as the validation set. To ensure the results are based on a representative sample, we conducted the experiments 10 times with a different validation set randomly selected each time.
- 2.
Source code can be found at: https://github.com/teresalazar13/FAIR-FATE.
- 3.
Since there are variations in the results due to different hyperparameters, we choose the outcome that achieves a good balance between accuracy and fairness, but prioritizes fairness by allowing a small sacrifice in accuracy if necessary.
References
Abay, A., Zhou, Y., Baracaldo, N., Rajamoni, S., Chuba, E., Ludwig, H.: Mitigating bias in federated learning. arXiv preprint arXiv:2012.02447 (2020)
Angwin, J., Larson, J., Mattu, S., Kirchner, L.: Machine Bias (2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Asuncion, A., Newman, D.: Adult data set. In: UCI Machine Learning Repository (2017). https://archive.ics.uci.edu/ml/datasets/adult
Bhagoji, A.N., Chakraborty, S., Mittal, P., Calo, S.: Analyzing federated learning through an adversarial lens. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 634–643. PMLR (2019)
Caton, S., Haas, C.: Fairness in machine learning: a survey. arXiv preprint arXiv:2010.04053 (2020)
Chen, J., Wolfe, C., Li, Z., Kyrillidis, A.: Demon: improved neural network training with momentum decay. In: 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2022, pp. 3958–3962. IEEE (2022)
Ezzeldin, Y.H., Yan, S., He, C., Ferrara, E., Avestimehr, S.: FairFed: enabling group fairness in federated learning. arXiv preprint arXiv:2110.00857 (2021)
Fernandes, M., Silva, C., Arrais, J.P., Cardoso, A., Ribeiro, B.: Decay momentum for improving federated learning. In: ESANN 2021, pp. 17–22 (2021). https://doi.org/10.14428/esann/2021.ES2021-106
Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Heo, B., et al.: AdamP: slowing down the slowdown for momentum optimizers on scale-invariant weights. arXiv preprint arXiv:2006.08217 (2020)
Hsu, T.M.H., Qi, H., Brown, M.: Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335 (2019)
Kanaparthy, S., Padala, M., Damle, S., Gujar, S.: Fair federated learning for heterogeneous face data. arXiv preprint arXiv:2109.02351 (2021)
Van der Laan, P.: The 2001 census in the Netherlands. In: Conference the Census of Population (2000)
Liu, W., Chen, L., Chen, Y., Zhang, W.: Accelerating federated learning via momentum gradient descent. IEEE Trans. Parallel Distrib. Syst. 1 (2020). https://doi.org/10.1109/TPDS.2020.2975189
McMahan, H., Moore, E., Ramage, D., Hampson, S., Arcass, B.A.: Communication-efficient learning of deep networks from decentralized data. In: AAAI Fall Symposium. Google Inc. (2017)
Mehrabi, N., de Lichy, C., McKay, J., He, C., Campbell, W.: Towards multi-objective statistically fair federated learning. arXiv preprint arXiv:2201.09917 (2022)
Qian, N.: On the momentum term in gradient descent learning algorithms. Neural Netw. 12(1), 145–151 (1999)
Salazar, T., Santos, M.S., Araujo, H., Abreu, P.H.: FAWOS: fairness-aware oversampling algorithm based on distributions of sensitive attributes. IEEE Access 9, 81370–81379 (2021). https://doi.org/10.1109/ACCESS.2021.3084121
Wightman, L.: LSAC national longitudinal bar passage study. In: LSAC Research Report Series (1998)
Zafar, M.B., Valera, I., Rogriguez, M.G., Gummadi, K.P.: Fairness constraints: mechanisms for fair classification. In: Artificial Intelligence and Statistics, pp. 962–970. PMLR (2017)
Acknowledgments
This work is funded by the FCT - Foundation for Science and Technology, I.P./MCTES through national funds (PIDDAC), within the scope of CISUC R &D Unit - UIDB/00326/2020 or project code UIDP/00326/2020. This work was supported in part by the Portuguese Foundation for Science and Technology (FCT) Research Grants 2021.05763.BD.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Salazar, T., Fernandes, M., Araújo, H., Abreu, P.H. (2023). FAIR-FATE: Fair Federated Learning with Momentum. In: Mikyška, J., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M. (eds) Computational Science – ICCS 2023. ICCS 2023. Lecture Notes in Computer Science, vol 14073. Springer, Cham. https://doi.org/10.1007/978-3-031-35995-8_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-35995-8_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35994-1
Online ISBN: 978-3-031-35995-8
eBook Packages: Computer ScienceComputer Science (R0)