Abstract
To promote the application of federated learning in resource-constraint unmanned aerial vehicle swarm, we propose a novel efficient federated learning framework CALIM-FL, short for Cross-All-Layers Importance Measure pruning-based Federated Learning. In CALIM-FL, an efficient one-shot filter pruning mechanism is intertwined with the standard FL procedure. The model size is adapted during FL to reduce both communication and computation overhead at the cost of a slight accuracy loss. The novelties of this work come from the following two aspects: 1) a more accurate importance measure on filters from the perspective of the whole neural networks; and 2) a communication-efficient one-shot pruning mechanism without data transmission from the devices. Comprehensive experiment results show that CALIM-FL is effective in a variety of scenarios, with a resource overhead saving of 88.4% at the cost of \(1\%\) accuracy loss.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Device and UAV are interchangeably used in this article.
References
Gong, Y., Liu, L., Yang, M., Bourdev, L.: Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.J., Han, S.: AMC: automl for model compression and acceleration on mobile devices. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 784–800 (2018)
Jiang, Y., et al.: Model pruning enables efficient federated learning on edge devices. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S., Stich, S., Suresh, A.T.: Scaffold: stochastic controlled averaging for federated learning. In: International Conference on Machine Learning, pp. 5132–5143 (2020)
Lee, N.T.A., Torr, P.: SNIP: single-shot network pruning based on connection sensitivity. In: ICLR (2019)
Lee, W.: Federated reinforcement learning-based UAV swarm system for aerial remote sensing. Wirel. Commun. Mob. Comput. 2022 (2022)
Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., Smith, V.: Federated optimization in heterogeneous networks. Proc. Mach. Learn. Syst. 2, 429–450 (2020)
Lin, M., et al.: Hrank: filter pruning using high-rank feature map. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1529–1538 (2020)
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)
Ruan, X., Liu, Y., Li, B., Yuan, C., Hu, W.: DPFPS: dynamic and progressive filter pruning for compressing convolutional neural networks from scratch. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 2495–2503 (2021)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Vogels, T., Karimireddy, S.P., Jaggi, M.: PowerSGD: practical low-rank gradient compression for distributed optimization. Adv. Neural Inf. Process. Syst. 32 (2019)
Voigt, P., Von dem Bussche, A.: The EU general data protection regulation (GDPR). In: A Practical Guide, 1st edn, vol. 10, no. 3152676, p. 105555 . Springer, Cham (2017)
Wang, H., Qin, C., Bai, Y., Zhang, Y., Fu, Y.: Recent advances on neural network pruning at initialization. In: Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI, Vienna, pp. 23–29 (2022)
Wang, J., Liu, Q., Liang, H., Joshi, G., Poor, H.V.: Tackling the objective inconsistency problem in heterogeneous federated optimization. Adv. Neural. Inf. Process. Syst. 33, 7611–7623 (2020)
Yoshida, N., Nishio, T., Morikura, M., Yamamoto, K., Yonetani, R.: Hybrid-FL for wireless networks: cooperative learning mechanism using non-IID data. In: ICC 2020–2020 IEEE International Conference On Communications (ICC), pp. 1–7. IEEE (2020)
Zhang, H., Liu, J., Jia, J., Zhou, Y., Dai, H., Dou, D.: FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server. arXiv preprint arXiv:2204.11536 (2022)
Acknowledgments
This work was supported in part by the National Key Research and Development Program of China (2021YFB3101304), in part by the Natural Science Basic Research Program of Shaanxi (2022JQ-621, 2022JQ-658), in part by the National Natural Science Foundation of China (62001359, 62002278), in part by the Fundamental Research Funds for the Central Universities (ZYTS23163).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Song, Q., Cao, J., Li, Y., Gao, X., Shangguan, C., Liang, L. (2024). On Efficient Federated Learning for Aerial Remote Sensing Image Classification: A Filter Pruning Approach. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14450. Springer, Singapore. https://doi.org/10.1007/978-981-99-8070-3_15
Download citation
DOI: https://doi.org/10.1007/978-981-99-8070-3_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8069-7
Online ISBN: 978-981-99-8070-3
eBook Packages: Computer ScienceComputer Science (R0)