Loading [a11y]/accessibility-menu.js
Adaptive Distributed Learning with Byzantine Robustness: A Gradient-Projection-Based Method | IEEE Conference Publication | IEEE Xplore

Adaptive Distributed Learning with Byzantine Robustness: A Gradient-Projection-Based Method

Publisher: IEEE

Abstract:

In this paper, we propose an adaptive distributed learning algorithm that not only resists three types of Byzantine attacks (i.e., gradient negative direction attacks, gr...View more

Abstract:

In this paper, we propose an adaptive distributed learning algorithm that not only resists three types of Byzantine attacks (i.e., gradient negative direction attacks, gradient partial dimension zeroing attacks, gradient scaling attacks) but also ensures high model accuracy. The proposed algorithm is built on a fully distributed model: clients share their local model updates with a group of dynamic committee clients, who cooperatively and iteratively train a global model. Specifically, to counter gradient negative direction attacks, we design a method based on gradient projection that maps clients' local gradients into small subspaces. The design allows committee clients to efficiently and precisely filter out adversarial clients by comparing angles between these subspaces. Moreover, considering that data heterogeneity among clients may cause misdetections of gradient partial dimension zeroing and scaling attacks, thereby reducing model accuracy, we introduce an adaptive multi-dimensional scoring method, which is applied after the gradient-projection-based filtering. The method assists committee clients in scoring and selecting most suitable clients for model aggregation using three hyperparameters, and thus achieves a balance between model accuracy and security. Finally, we conduct extensive experiments on real-world datasets to show the proposed algorithm's effectiveness: it can achieve Byzantine robustness and simultaneously maintain high model accuracy.
Date of Conference: 04-08 December 2023
Date Added to IEEE Xplore: 26 February 2024
ISBN Information:

ISSN Information:

Publisher: IEEE
Conference Location: Kuala Lumpur, Malaysia

References

References is not available for this document.