Federated Learning with Class Balanced Loss Optimized by Implicit Stochastic Gradient Descent

Abstract

Federated learning is a paradigm for distributed machine learning in which a central server interacts with a large number of remote devices to create the optimal global model. System and data heterogeneity are now the two largest impediments to federated learning. This work suggests a federated learning strategy based on stochastic gradient descent optimization as a solution to the problem of heterogeneity-induced slow convergence, or even non-convergence, of the global model. This work estimates the average global gradient using locally uploaded model parameters without computing the first derivative or updating global model parameters through gradient descent. Allowing the global model to be used with fewer communication rounds. Obtain faster and more reliable convergence results. In experiments simulating varying degrees of heterogeneous settings, the strategy proposed in this work delivered faster and more stable convergence than FedProx and FedAvg. This work offers a strategy that decreases the number of communication cycles on highly heterogeneous synthetic datasets by around 50% compared to FedProx, therefore considerably enhancing the stability and durability of federated learning.