Subjective Logic-based Decentralized Federated Learning for Non-IID Data
ARES '24: Proceedings of the 19th International Conference on Availability, Reliability and Security
Article No.: 12, Pages 1 - 11
Abstract
Existing Federated Learning (FL) methods are highly influenced by the training data distribution. In the single global model FL systems, users with highly non-IID data do not improve the global model, and neither does the global model work well on their local data distribution. Even with the clustering-based FL approaches, not all participants get clustered adequately enough for the models to fulfill their local demands. In this work, we design a modified subjective logic-based FL system utilizing the distribution-based similarity among users. Each participant has complete control over their own aggregated model, with handpicked contributions from other participants. The existing clustered model only satisfies a subset of clients, while our individual aggregated models satisfy all the clients. We design a decentralized FL approach, which functions without a trusted central server; the communication and computation overhead is distributed among the clients. We also develop a layer-wise secret-sharing scheme to amplify privacy. We experimentally show that our approach improves the performance of each participant’s aggregated model on their local distribution over the existing single global model and clustering-based approach.
References
[1]
Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. 2002. Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 2 (2002), 235–256.
[2]
Christopher Briggs, Zhong Fan, and Peter Andras. 2020. Federated learning with hierarchical clustering of local updates to improve training on non-IID data. In 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–9.
[3]
Haokun Chen, Ahmed Frikha, Denis Krompass, Jindong Gu, and Volker Tresp. 2023. FRAug: Tackling federated learning with Non-IID features via representation augmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4849–4859.
[4]
Yujing Chen, Yue Ning, Martin Slawski, and Huzefa Rangwala. 2020. Asynchronous online federated learning for edge devices with non-iid data. In 2020 IEEE International Conference on Big Data (Big Data). IEEE, 15–24.
[5]
Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter. 2017. A downsampled variant of imagenet as an alternative to the cifar datasets. arXiv preprint arXiv:1707.08819 (2017).
[6]
Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. 2020. The limitations of federated learning in sybil settings. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020). 301–316.
[7]
Liang Gao, Huazhu Fu, Li Li, Yingwen Chen, Ming Xu, and Cheng-Zhong Xu. 2022. Feddc: Federated learning with non-iid data via local drift decoupling and correction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10112–10121.
[8]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
[9]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
[10]
István Hegedűs, Gábor Danner, and Márk Jelasity. 2019. Gossip learning as a decentralized alternative to federated learning. In IFIP International Conference on Distributed Applications and Interoperable Systems. Springer, 74–90.
[11]
Chenghao Hu, Jingyan Jiang, and Zhi Wang. 2019. Decentralized federated learning: A segmented gossip approach. arXiv preprint arXiv:1908.07782 (2019).
[12]
Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. 2022. Membership inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR) 54, 11s (2022), 1–37.
[13]
Md Sirajul Islam, Simin Javaherian, Fei Xu, Xu Yuan, Li Chen, and Nian-Feng Tzeng. 2024. FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clustering. arXiv preprint arXiv:2403.04144 (2024).
[14]
Omid Jafari, Preeti Maurya, Parth Nagarkar, Khandker Mushfiqul Islam, and Chidambaram Crushev. 2021. A survey on locality sensitive hashing algorithms and their applications. arXiv preprint arXiv:2102.08942 (2021).
[15]
Audun Jøsang. 1997. Artificial reasoning with subjective logic. In Proceedings of the second Australian workshop on commonsense reasoning, Vol. 48. Citeseer, 34.
[16]
Audun Jøsang. 2001. A logic for uncertain probabilities. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 9, 03 (2001), 279–311.
[17]
Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. 2021. Fedbn: Federated learning on non-iid features via local batch normalization. arXiv preprint arXiv:2102.07623 (2021).
[18]
Yuzheng Li, Chuan Chen, Nan Liu, Huawei Huang, Zibin Zheng, and Qiang Yan. 2020. A blockchain-based decentralized federated learning framework with committee consensus. IEEE Network 35, 1 (2020), 234–241.
[19]
Yi Liu, Jialiang Peng, JQ James, and Yi Wu. 2019. PPGAN: Privacy-preserving generative adversarial network. In 2019 IEEE 25Th international conference on parallel and distributed systems (ICPADS). IEEE, 985–989.
[20]
Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, and Richard Vidal. 2021. Federated multi-task learning under a mixture of distributions. Advances in Neural Information Processing Systems 34 (2021), 15434–15447.
[21]
Leland McInnes, John Healy, and Steve Astels. 2017. hdbscan: Hierarchical density based clustering.J. Open Source Softw. 2, 11 (2017), 205.
[22]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR, 1273–1282.
[23]
Mahdi Morafah, Saeed Vahidian, Weijia Wang, and Bill Lin. 2023. Flis: Clustered federated learning via inference similarity for non-iid data distribution. IEEE Open Journal of the Computer Society 4 (2023), 109–120.
[24]
Xutong Mu, Yulong Shen, Ke Cheng, Xueli Geng, Jiaxuan Fu, Tao Zhang, and Zhiwei Zhang. 2023. Fedproc: Prototypical contrastive federated learning on non-iid data. Future Generation Computer Systems 143 (2023), 93–104.
[25]
Thien Duc Nguyen, Phillip Rieger, Huili Chen, Hossein Yalame, Helen Möllering, Hossein Fereidooni, Samuel Marchal, Markus Miettinen, Azalia Mirhoseini, Shaza Zeitouni, 2022. FLAME: Taming Backdoors in Federated Learning. (2022).
[26]
Christodoulos Pappas, Dimitris Chatzopoulos, Spyros Lalis, and Manolis Vavalis. 2021. Ipls: A framework for decentralized federated learning. In 2021 IFIP Networking Conference (IFIP Networking). IEEE, 1–6.
[27]
Phillip Rieger, Thien Duc Nguyen, Markus Miettinen, and Ahmad-Reza Sadeghi. 2022. DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection. arXiv preprint arXiv:2201.00763 (2022).
[28]
Abhijit Guha Roy, Shayan Siddiqui, Sebastian Pölsterl, Nassir Navab, and Christian Wachinger. 2019. Braintorrent: A peer-to-peer environment for decentralized federated learning. arXiv preprint arXiv:1905.06731 (2019).
[29]
Anit Kumar Sahu, Tian Li, Maziar Sanjabi, Manzil Zaheer, Ameet Talwalkar, and Virginia Smith. 2018. On the convergence of federated optimization in heterogeneous networks. arXiv preprint arXiv:1812.06127 3 (2018), 3.
[30]
Felix Sattler, Klaus-Robert Müller, and Wojciech Samek. 2021. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints. IEEE Transactions on Neural Networks and Learning Systems 32, 8 (2021), 3710–3722. https://doi.org/10.1109/TNNLS.2020.3015958
[31]
Adi Shamir. 1979. How to share a secret. Commun. ACM 22, 11 (1979), 612–613.
[32]
Warit Sirichotedumrong and Hitoshi Kiya. 2021. A gan-based image transformation scheme for privacy-preserving deep neural networks. In 2020 28th European Signal Processing Conference (EUSIPCO). IEEE, 745–749.
[33]
Yue Tan, Yixin Liu, Guodong Long, Jing Jiang, Qinghua Lu, and Chengqi Zhang. 2023. Federated learning on non-iid graphs via structural knowledge sharing. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37. 9953–9961.
[34]
Hao Wang, Zakhary Kaplan, Di Niu, and Baochun Li. 2020. Optimizing federated learning on non-iid data with reinforcement learning. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 1698–1707.
[35]
Yanmeng Wang, Qingjiang Shi, and Tsung-Hui Chang. 2023. Why batch normalization damage federated learning on non-iid data?arXiv preprint arXiv:2301.02982 (2023).
[36]
Ming Xie, Guodong Long, Tao Shen, Tianyi Zhou, Xianzhi Wang, Jing Jiang, and Chengqi Zhang. 2020. Multi-center federated learning. arXiv preprint arXiv:2005.01026 (2020).
[37]
Lei Yang, Jiaming Huang, Wanyu Lin, and Jiannong Cao. 2023. Personalized federated learning on non-IID data via group-based meta-learning. ACM Transactions on Knowledge Discovery from Data 17, 4 (2023), 1–20.
[38]
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 (2018).
Index Terms
- Subjective Logic-based Decentralized Federated Learning for Non-IID Data
Recommendations
Federated learning using model projection for multi-center disease diagnosis with non-IID data
AbstractMulti-center disease diagnosis aims to build a global model for all involved medical centers. Due to privacy concerns, it is infeasible to collect data from multiple centers for training (i.e., centralized learning). Federated Learning (FL) is a ...
Highlights- The forgetting at clients and invalid aggregation at the server are both alleviated.
- MPC avoids an increase in loss of local models on global data after local training.
- MPS avoids an increase in the loss of global model on local ...
Fast-convergent federated learning with class-weighted aggregation
AbstractRecently, federated learning has attracted great attention due to its advantage of enabling model training in a distributed manner. Instead of uploading data for centralized training, it allows devices to keep local data private and ...
Non-IID always Bad? Semi-Supervised Heterogeneous Federated Learning with Local Knowledge Enhancement
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementFederated learning (FL) is important for privacy-preserving services by training models without collecting raw user data. Most FL algorithms assume all data is annotated, which is impractical due to the high cost of labeling data in real applications. To ...
Comments
Information & Contributors
Information
Published In
July 2024
2032 pages
ISBN:9798400717185
DOI:10.1145/3664476
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ARES 2024
ARES 2024: The 19th International Conference on Availability, Reliability and Security
July 30 - August 2, 2024
Vienna, Austria
Acceptance Rates
Overall Acceptance Rate 228 of 451 submissions, 51%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 66Total Downloads
- Downloads (Last 12 months)66
- Downloads (Last 6 weeks)24
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format