Skip to main content

Advertisement

SpringerLink
Log in

DFedSN: Decentralized federated learning based on heterogeneous data in social networks

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Users talk with each other and share their lives online, which forms a huge social network. However, a series of potential problems such as privacy security and data leakage will occur in the process of data transmission and sharing. Therefore, we came up with the idea of putting social networks into the framework of decentralized federated learning. Federated learning is a model of distributed training model, which can limit users’ own data in and out of the local area during transmission. Moreover, the decentralized federated learning eliminates the hidden threat from third-party servers. However, the data generated by a large user group is also messy and heterogeneous, and there are different distribution changes among them, which will greatly reduce the learning performance of the model. This paper aims to solve the problem of user data heterogeneity in decentralized federated learning, so as to reduce the loss of model performance. To be detailed, we set up the affine distribution of user data structure to capture the users who have the heterogeneity between data, and then put forward an approach called DFedSN training to reduce the independent identically distributed data loss. This method has excellent robustness even in the case of different data distribution, and it can train and learn the model stably without losing efficiency. Through several experiments on DFedSN algorithm in different neural networks, the results show that our algorithm can still achieve excellent training results even when there is a huge gap in the distribution of user data, and it will have no negative effects due to different distributions, which is the first step in decentralized federated learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12

Similar content being viewed by others

Data Availability

Some or all data, models, or code generated or used during the study are available from the corresponding author by request.

References

  1. Cheng, S.: Research on data privacy protection technology of social network users based on differential disturbance. Ain Shams Eng. J. 13(5), 101745 (2022)

    Article  MathSciNet  Google Scholar 

  2. Galli, F., Biswas, S., Jung, K., et al.: Group privacy for personalized federated learning. arXiv:2206.03396 (2022)

  3. Shaheen, M., Farooq, M.S., Umer, T., et al.: Applications of federated learning; Taxonomy, challenges, and research trends. Electronics 11(4), 670 (2022)

    Article  Google Scholar 

  4. Qin, Z., Deng, S., Yan, X., et al.: Secure and efficient decentralized federated learning with data representation protection. arXiv:2205.10568 (2022)

  5. Li, T., Sahu, A.K., Talwalkar, A., et al.: Federated learning: Challenges, methods, and future directions. IEEE Signal Process. Mag. 37(3), 50–60 (2020)

    Article  Google Scholar 

  6. Kairouz, P., McMahan, H.B., Avent, B., et al.: Advances and open problems in federated learning. Found. Trends Mach. Learn. 14(1–2), 1–210 (2021)

    Article  MATH  Google Scholar 

  7. Reisizadeh, A., Farnia, F., Pedarsani, R., et al.: Robust federated learning: The case of affine distribution shifts. Adv. Neural Inf. Process. Syst. 33, 21554–21565 (2020)

    Google Scholar 

  8. Pei, K., Cao, Y., Yang, J., et al.: Deepxplore: Automated whitebox testing of deep learning systems. In: Proceedings of the 26th Symposium on Operating Systems Principles, pp 1–18 (2017)

  9. Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. arXiv:1903.12261 (2019)

  10. Madry, A., Makelov, A., Schmidt, L., et al.: Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083 (2017)

  11. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv:1412.6572 (2014)

  12. Shafahi, A., Najibi, M., Xu, Z., et al.: Universal adversarial training. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp 5636–5643 (2020)

  13. McMahan, B., Moore, E., Ramage, D., et al.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp 1273–1282. PMLR (2017)

  14. Karimireddy, S.P., Kale, S., Mohri, M., et al.: SCAFFOLD: stochastic controlled averaging for on-device federated learning (2019)

  15. Onoszko, N., Karlsson, G., Mogren, O., et al.: Decentralized federated learning of deep neural networks on non-iid data. arXiv:2107.08517(2021)

  16. Liu, R., Wu, F., Wu, C., et al.: No One left behind: inclusive federated learning over heterogeneous devices. arXiv:2202.08036 (2022)

  17. Achituve, I., Shamsian, A., Navon, A., et al.: Personalized Federated Learning with Gaussian Processes. Adv. Neural Inf. Process. Syst. 34, 8392–8406 (2021)

    Google Scholar 

  18. Bassily, R., Smith, A., Thakurta, A.: Private empirical risk minimization: Efficient algorithms and tight error bounds. In: 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, pp 464–473. IEEE (2014)

  19. Nedic, A., Ozdaglar, A.: Subgradient methods for saddle-point problems. J. Optim. Theory Appl. 142(1), 205–228 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  20. Lin, T., Jin, C., Jordan, M.: On gradient descent ascent for nonconvex-concave minimax problems. In: International Conference on Machine Learning, pp 6083–6093. PMLR (2020)

  21. Yu, H., Jin, R., Yang, S.: On the linear speedup analysis of communication efficient momentum SGD for distributed non-convex optimization. In: International Conference on Machine Learning, pp 7184–7193. PMLR (2019)

  22. Yang, J., Kiyavash, N., He, N.: Global convergence and variance-reduced optimization for a class of nonconvex-nonconcave minimax problems. arXiv:2002.09621 (2020)

  23. Abadi, M., Agarwal, A., Barham, P., et al.: Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467(2016)

  24. LeCun, Y.: The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/ (1998)

  25. Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)

  26. Krizhevsky, A., Sutskever, I., Hinton, G. E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25 (2012)

  27. Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9 (2015)

  28. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778 (2016)

  29. Zachary, W. W.: An information flow model for conflict and fission in small groups. J. Anthropol. Res. 33(4), 452–473 (1977)

    Article  Google Scholar 

  30. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. PMLR, pp 448–456 (2015)

  31. Kingma, D.P., Ba, J., Adam, A.: Method for stochastic optimization. arXiv:1412.6980 (2014)

Download references

Acknowledgements

The authors thank the reviewers for their constructive comments in improving the quality of this paper.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Yunnan Normal University, Kunming, China

    Yikuan Chen, Li Liang & Wei Gao

Authors
  1. Yikuan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yikuan Chen wrote the manuscript, Wei Gao and Li Liang revised and made improvements on the manuscript. The authors have worked equally when writing this paper. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Li Liang or Wei Gao.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for Publication

Not applicable.

Human and animal ethics

Not applicable.

Conflict of Interests

We declare that none of the authors have any competing interests in the manuscript.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Privacy and Security in Machine Learning Guest Editors: Jin Li, Francesco Palmieri and Changyu Dong

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Liang, L. & Gao, W. DFedSN: Decentralized federated learning based on heterogeneous data in social networks. World Wide Web 26, 2545–2568 (2023). https://doi.org/10.1007/s11280-023-01152-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-023-01152-4

Keywords

Search

Navigation