Abstract
Federated learning is a distributed machine learning framework that allows participants to keep their privacy data locally. Traditional federated learning coordinates participants collaboratively train a powerful global model. However, this process has several problems: it cannot meet the heterogeneous model’s requirements, and it cannot resist poisoning attacks and single-point-of-failure. In order to resolve these issues, we proposed a heterogeneous federated learning algorithm based on blockchain and model distillation. The problem of fully heterogeneous models that are hard to aggregate in the central server can be solved by leveraging model distillation technology. Moreover, blockchain replaces the central server in federated learning to solve the single-point-of-failure problem. The validation algorithm is combined with cross-validation, which helps federated learning to resist poison attacks. The extensive experimental results demonstrate that HBMD-FL can resist poisoning attacks while losing less than 3\(\%\) of model accuracy, and the communication consumption significantly outperformed the comparison algorithm.
This work is partially supported by Natural Science Foundation of China (62206238), Natural Science Foundation of Jiangsu Province (Grant No. BK20220562), Natural Science Foundation of Jiangsu Higher Education Institutions of China (Grant No. 22KJB520010), Future Network Scientific Research Fund Project (FNSRFP-2021-YB-47), Yangzhou City-Yangzhou University Science and Technology Cooperation Fund Project (YZ2021158).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
McMahan, B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
Smith, V., Chiang, C.K., Sanjabi, M., Talwalkar, A.S.: Federated multi-task learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Corinzia, L., Beuret, A., Buhmann, J.M.: Variational federated multi-task learning. arXiv preprint arXiv:1906.06268 (2019)
Jiang, Y., Konečnỳ, J., Rush, K., Kannan, S.: Improving federated learning personalization via model agnostic meta learning. arXiv preprint arXiv:1909.12488 (2019)
Khodak, M., Balcan, M.F.F., Talwalkar, A.S.: Adaptive gradient-based meta-learning methods. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
He, C., Annavaram, M., Avestimehr, S.: Towards non-IID and invisible data with FedNAS: federated deep learning via neural architecture search. arXiv preprint arXiv:2004.08546 (2020)
Li, D., Wang, J.: FedMD: heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019)
Jeong, E., Oh, S., Kim, H., Park, J., Bennis, M., Kim, S.L.: Communication-efficient on-device machine learning: federated distillation and augmentation under non-IID private data. arXiv preprint arXiv:1811.11479 (2018)
Shen, T., et al.: Federated mutual learning. arXiv preprint arXiv:2006.16765 (2020)
Mo, Z., Gao, Z., Zhao, C., Lin, Y.: FedDQ: a communication-efficient federated learning approach for Internet of Vehicles. J. Syst. Archit. 131, 102690 (2022). https://www.sciencedirect.com/science/article/pii/S1383762122001928
Sattler, F., Marban, A., Rischke, R., Samek, W.: CFD: communication-efficient federated distillation via soft-label quantization and delta coding. IEEE Trans. Netw. Sci. Eng. 9(4), 2025–2038 (2022)
Kim, H., Park, J., Bennis, M., Kim, S.L.: Blockchained on-device federated learning. IEEE Commun. Lett. 24(6), 1279–1283 (2019)
Nakamoto, S.: Bitcoin: a peer-to-peer electronic cash system. Decentralized Bus. Rev., 21260 (2008)
Samarakoon, S., Bennis, M., Saad, W., Debbah, M.: Distributed federated learning for ultra-reliable low-latency vehicular communications. IEEE Trans. Commun. 68(2), 1146–1159 (2019)
Nishio, T., Yonetani, R.: Client selection for federated learning with heterogeneous resources in mobile edge. In: ICC 2019 IEEE International Conference on Communications (ICC), pp. 1–7. IEEE (2019)
Kairouz, P., et al.: Advances and open problems in federated learning. Found. Trends® Mach. Learn. 14(1–2), 1–210 (2021)
He, C., et al.: FedML: a research library and benchmark for federated machine learning. arXiv preprint arXiv:2007.13518 (2020)
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., Khazaeni, Y.: Federated learning with matched averaging. arXiv preprint arXiv:2002.06440 (2020)
Li, Q., He, B., Song, D.: Model-contrastive federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10713–10722 (2021)
Wu, Q., He, K., Chen, X.: Personalized federated learning for intelligent IoT applications: a cloud-edge based framework. IEEE Open J. Comput. Soc. 1, 35–44 (2020)
Fallah, A., Mokhtari, A., Ozdaglar, A.: Personalized federated learning: a meta-learning approach. CoRR, vol. abs/2002.07948 (2020). https://arxiv.org/abs/2002.07948
Arivazhagan, M.G., Aggarwal, V., Singh, A.K., Choudhary, S.: Federated learning with personalization layers. CoRR, vol. abs/1912.00818 (2019). http://arxiv.org/abs/1912.00818
Hinton, G., Vinyals, O., Dean, J., et al.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 2(7) (2015)
Wang, C., Yang, G., Papanastasiou, G., Zhang, H., Rodrigues, J.J., de Albuquerque, V.H.C.: Industrial cyber-physical systems-based cloud IoT edge for federated heterogeneous distillation. IEEE Trans. Industr. Inf. 17(8), 5511–5521 (2020)
Peng, Z., et al.: VFChain: enabling verifiable and auditable federated learning via blockchain systems. IEEE Trans. Netw. Sci. Eng. 9(1), 173–186 (2021)
Lu, Y., Huang, X., Dai, Y., Maharjan, S., Zhang, Y.: Blockchain and federated learning for privacy-preserved data sharing in industrial IoT. IEEE Trans. Industr. Inf. 16(6), 4177–4186 (2019)
Zhang, W., et al.: Blockchain-based federated learning for device failure detection in industrial IoT. IEEE Internet Things J. 8(7), 5926–5937 (2020)
Nguyen, D.C., et al.: Federated learning meets blockchain in edge computing: opportunities and challenges. IEEE Internet Things J. 8(16), 12806–12825 (2021)
Awan, S., Li, F., Luo, B., Liu, M.: Poster: a reliable and accountable privacy-preserving federated learning framework using the blockchain. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 2561–2563 (2019)
Chen, H., Asif, S.A., Park, J., Shen, C.C., Bennis, M.: Robust blockchained federated learning with model validation and proof-of-stake inspired consensus. arXiv preprint arXiv:2101.03300 (2021)
Majeed, U., Hong, C.S.: FLchain: Federated learning via MEC-enabled blockchain network. In: 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS), pp. 1–4. IEEE (2019)
Li, Y., Chen, C., Liu, N., Huang, H., Zheng, Z., Yan, Q.: A blockchain-based decentralized federated learning framework with committee consensus. IEEE Netw. 35(1), 234–241 (2020)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Y., Zhang, J., Zhu, J., Li, W. (2022). HBMD-FL: Heterogeneous Federated Learning Algorithm Based on Blockchain and Model Distillation. In: Chen, J., He, D., Lu, R. (eds) Emerging Information Security and Applications. EISA 2022. Communications in Computer and Information Science, vol 1641. Springer, Cham. https://doi.org/10.1007/978-3-031-23098-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-23098-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23097-4
Online ISBN: 978-3-031-23098-1
eBook Packages: Computer ScienceComputer Science (R0)