Tong Liu
ABSTRACT
Performance degradation under heterogeneous data distribution is one of the biggest challenges of federated learning. Among the optimization methods for federated learning with heterogeneous data, client selection strategies based on client grouping have received much attention due to their effectiveness and generality. However, current strategies are crude and only achieve client-level class balancing. In this paper, we propose FedCBA, which enables sample-level class balancing. First, we propose a privacy-preserving sample grouping approach, which consists of feature extraction, locality-sensitive hashing, and unsupervised clustering, to obtain the class distribution of each client. Then in each communication round, based on the clients' class distribution, the good scaling aggregation parameters which can make the class balanced are learnt. Comprehensive experimental results on widely used and publicly accessible datasets show that compared to existing methods, FedCBA can effectively improve the prediction accuracy of federated learning models under heterogeneous data.
- McMahan, B., Moore, E., Ramage, D., , “Communication efficient learning of deep networks from decentralized data,” Artificial Intelligence and Statistics (AISTATS), Microtome Publishing, Brookline, 1273-1282. 2017.Google Scholar
- Zhao, Y., Li, M., Lai, L., , “Federated learning with non-IID data,” arXiv: 1806.00582. 2018.Google Scholar
- Jeong, E., Oh, S., Kim, H., , “Communication-efficient on-device machine learning: federated distillation and augmentation under non-IID private data,” arXiv.1811.11479. 2018.Google Scholar
- Li, T, Sahu, A. K., Zaheer, M., , “Federated optimization in heterogeneous networks,” Proceedings of Machine Learning and Systems, 2: 429-450. 2020.Google Scholar
- Karimireddy, S. P., Kale, S., Mohri, M., , “Scaffold: Stochastic controlled averaging for federated learning,” International Conference on Machine Learning, PMLR, 5132-5143. 2020.Google Scholar
- Wolfrath, J., Sreekumar, N., Kumar, D., , “HACCS: Heterogeneity-aware clustered client selection for accelerated federated learning,” 2022 IEEE International Parallel and Distributed Processing Symposium, 985-995. 2022.Google ScholarCross Ref
- Liu, T., Ding, J., Wang, T., , “Towards fast and accurate federated learning with non-IID data for cloud-based IoT applications,” Journal of Circuits Systems and Computers, vol. 31, no. 13. 2022.Google ScholarCross Ref
- Chen, T., Kornblith, S., Norouzi, M., , “A simple framework for contrastive learning of visual representations,” International conference on machine learning, PMLR, 1597-1607. 2020.Google Scholar
- Datar, M., Immorlica, N., Indyk, P., , “Locality-sensitive hashing scheme based on p-stable distributions” Proceedings of the twentieth annual symposium on Computational geometry, 253-262. 2004.Google ScholarDigital Library
- MacQueen, J., “Some methods for classification and analysis of multivariate observations,” Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 281-297. 1967.Google Scholar
- Tolstikhin, I. O., Houlsby, N., Kolesnikov, A., , “Mlp-mixer: An all-mlp architecture for vision”, Advances in Neural Information Processing Systems, 34: 24261-24272. 2021.Google Scholar
- LeCun, Y., Bottou, L., Bengio, Y., , “Gradient-based learning applied to document recognition”, Proceedings of the IEEE, 86(11): 2278-2324. 1998.Google ScholarCross Ref
- Rong, D., Ye, S., Zhao, R., , “FedRecAttack: model poisoning attack to federated recommendation,” 2022 IEEE 38th International Conference on Data Engineering (ICDE), IEEE, 2643-2655. 2022.Google ScholarCross Ref
- Rong, D., He, Q., Chen, J., “Poisoning deep learning based recommender model in federated learning scenarios,” International Joint Conference on Artificial Intelligence (IJCAI), 2204-2210. 2022.Google ScholarCross Ref
- Hu, H., Rong, D., Chen, J., , “CoMeta: Enhancing meta embeddings with collaborative information in cold-start problem of recommendation,” Knowledge Science, Engineering and Management (KSEM), vol. 14119. 2023.Google Scholar
Index Terms
- Federated Learning with Sample-Level Class Balancing Aggregation
Recommendations
Fast-convergent federated learning with class-weighted aggregation
AbstractRecently, federated learning has attracted great attention due to its advantage of enabling model training in a distributed manner. Instead of uploading data for centralized training, it allows devices to keep local data private and ...
Training Aggregation in Federated Learning
Cyberspace Safety and SecurityAbstractFederated learning is a new machine learning paradigm for distributed data. It enables multi-party cooperation to train global models without sharing their private data. In the classic federated learning protocol, the model parameters are the ...
Verifiable Secure Aggregation Protocol Under Federated Learning
Artificial Intelligence Security and PrivacyAbstractFederated learning is a new machine learning paradigm used for collaborative training models among multiple devices. In federated learning, multiple clients participate in model training locally and use decentralized learning methods to ensure the ...
Comments