Our research addresses the pressing challenges of privacy and data heterogeneity in FL, a domain gaining increasing attention in the quest for secure and collaborative machine learning models. While existing FL works often incorporate DP for privacy concerns, none have effectively tackled the dual challenges of data heterogeneity and privacy preservation in a unified framework. In response, our study introduces the DP-FedCVR algorithm, a pioneering solution that adeptly navigates the complexities of non-i.i.d. data while upholding robust privacy safeguards. The significance of our research lies in its real-world applicability, particularly in sectors like healthcare, finance, and telecommunications, where diverse data sources and stringent privacy requirements are paramount. The DP-FedCVR algorithm not only addresses a critical gap in current FL methodologies but also provides a theoretical foundation, fortified by rigorous analysis, establishing it as a guiding principle for future FL...

Federated learning (FL) is a popular distributed paradigm where enormous clients collaboratively train a machine learning (ML) model under the orchestration of a central server without knowing the clients’ private raw data. The development of effective FL algorithms faces multiple practical challenges including data heterogeneity and clients’ privacy protection. Despite that numerous attempts have been made to deal with data heterogeneity or rigorous privacy protection, none have effectively tackled both issues simultaneously. In this article, we propose a differentially private and heterogeneity-robust FL algorithm, named DP-FedCVR to mitigate the data heterogeneity by following the client-variance-reduction strategy. Besides, it adopts a sophisticated differential privacy (DP) mechanism where the privacy-amplified strategy is applied, to achieve a rigorous privacy protection guarantee. We show that the proposed DP-FedCVR algorithm maintains its heterogeneity-robustness though DP nois...