skip to main content
10.1145/3579654.3579732acmotherconferencesArticle/Chapter ViewAbstractPublication PagesacaiConference Proceedingsconference-collections
research-article

FedDDB: Clustered Federated Learning based on Data Distribution Difference

Published: 14 March 2023 Publication History

Abstract

Clustered federated learning is a federated learning method based on multi-task learning. It groups similar clients into the same clusters and shares model parameters to solve the problem that the joint model is trapped in local optima on Non-IID data. Most of the existing clustered federated learning methods are based on the difference of model parameters for clients clustering. During the client model training process, the model parameters are biased and the clustering result is affected due to insufficient samples and missing eigenvalues in the dataset. In this paper, we develop a clustered federated learning method based on data distribution difference (FedDDB) in the dataset level. The method in this paper focuses on the distribution of label probability and eigenvalues, analyzes the difference of data distribution difference between clients and measures the distance between datasets which is used for client clustering. Every cluster will be trained independently and in parallel on the cluster center model. At the beginning of each round of training, the client clustering process needs to be repeated. We conduct relevant experiments and demonstrate the effectiveness of our method.

References

[1]
Kairouz P, McMahan H B, Avent B, Advances and open problems in federated learning[J]. Foundations and Trends® in Machine Learning, 2021, 14(1–2): 1-210.
[2]
Zhang C, Xie Y, Bai H, A survey on federated learning[J]. Knowledge-Based Systems, 2021, 216: 106775.
[3]
Y. Zhao, M. Li, L. Lai, Chandra. Federated Learning with Non-IID Data. ArXiv:1806.00582, 2018.
[4]
McMahan B, Moore E, Ramage D, Communication-efficient learning of deep networks from decentralized data[C]//Artificial intelligence and statistics. PMLR, 2017: 1273-1282.
[5]
Hsieh K, Phanishayee A, Mutlu O, The non-iid data quagmire of decentralized machine learning[C]//International Conference on Machine Learning. PMLR, 2020: 4387-4398.
[6]
Li T, Sahu A K, Zaheer M, Federated optimization in heterogeneous networks[J]. Proceedings of Machine Learning and Systems, 2020, 2: 429-450.
[7]
Chen C, Chen Z, Zhou Y, Fedcluster: Boosting the convergence of federated learning via cluster-cycling[C]. International Conference on Big Data (Big Data). IEEE, 2020: 5017-5026.
[8]
Sattler F, Müller K R, Samek W. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints[J]. IEEE transactions on neural networks and learning systems, 2020, 32(8): 3710-3722.
[9]
Li C, Li G, Varshney P K. Federated Learning With Soft Clustering[J]. IEEE Internet of Things Journal, 2021, 9(10): 7773-7782.
[10]
Ruan Y, Joe-Wong C. Fedsoft: Soft clustered federated learning with proximal local updating[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(7): 8124-8131.
[11]
Briggs C, Fan Z, Andras P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data[C]//2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 2020: 1-9.
[12]
Jamali-Rad H, Abdizadeh M, Singh A. Federated learning with taskonomy for non-iid data[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[13]
Feng C, Yang H H, Hu D, Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networks[J]. IEEE Transactions on Wireless Communications, 2022.
[14]
Cover T M, Thomas J A. Entropy, relative entropy and mutual information[J]. Elements of information theory, 1991, 2(1): 12-13.
[15]
Sattler F, Müller K R, Samek W. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints[J]. IEEE transactions on neural networks and learning systems, 2020, 32(8): 3710-3722.
[16]
LeCun Y, Bottou L, Bengio Y, Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[17]
Zargar S T, Joshi J, Tipper D, A Survey of defense mechanisms against distributed denial of service (DDoS) flooding attacks[J]. IEEE Communications Surveys and Tutorials, 2013, 15(4): 2046-2069.
[18]
Barratt S, Sharma R. A note on the inception score. ArXiv:1801.01973, 2018.

Cited By

View all
  • (2025)FedMG: Vehicular Edge Federated Learning for Mobile Scenarios With Geo-Dispersed DataIEEE Transactions on Vehicular Technology10.1109/TVT.2024.345533374:1(1520-1533)Online publication date: Jan-2025
  • (2024)Adaptive Single-layer Aggregation Framework for Energy-efficient and Privacy-preserving Load Forecasting in Heterogeneous Federated Smart GridsInternet of Things10.1016/j.iot.2024.10137628(101376)Online publication date: Dec-2024

Index Terms

  1. FedDDB: Clustered Federated Learning based on Data Distribution Difference
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ACAI '22: Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence
    December 2022
    770 pages
    ISBN:9781450398336
    DOI:10.1145/3579654
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 March 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Clustering method
    2. Feature extraction
    3. Federated learning
    4. Label distribution
    5. Non-independent and identically distributed data

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • the Shanghai Science and Technology Innovation Action Plan Project
    • the Strategic Research and Consulting Project of the Chinese Academy of Engineering

    Conference

    ACAI 2022

    Acceptance Rates

    Overall Acceptance Rate 173 of 395 submissions, 44%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)27
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 23 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)FedMG: Vehicular Edge Federated Learning for Mobile Scenarios With Geo-Dispersed DataIEEE Transactions on Vehicular Technology10.1109/TVT.2024.345533374:1(1520-1533)Online publication date: Jan-2025
    • (2024)Adaptive Single-layer Aggregation Framework for Energy-efficient and Privacy-preserving Load Forecasting in Heterogeneous Federated Smart GridsInternet of Things10.1016/j.iot.2024.10137628(101376)Online publication date: Dec-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media