Abstract
The Federated Learning (FL) paradigm is emerging as a way to train machine learning (ML) models in distributed systems. A large population of interconnected devices (i.e. Internet of Things (IoT)) acting as local learners optimize the model parameters collectively (e.g., neural networks’ weights), rather than sharing and disclosing the training data set with the server. FL approaches assume each participant has enough training data for the tasks of interest. Realistically, data collected by IoT devices may be insufficient and often unlabeled. In particular, each IoT device may only contain one or a few samples of every relevant data category, and may not have the time or interest to label them. In realistic applications, this severely limits FL’s practicality and usability. In this paper, we propose a One-Shot Federated Learning (OSFL) framework considering a FL scenario wherein the local training is carried out on IoT devices and the global aggregation is done at the level of an edge server. Moreover, we combine model-free reinforcement learning with OSFL to design a more intelligent IoT device to infer whether to label a sample automatically or request the true label for the one-shot learning set-up. We validate our system on the SODA10M dataset. Experiments show that our solution achieves better performance than DQN and RS benchmark approaches.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
References
Amiri, M.M., Gündüz, D., Kulkarni, S.R., Poor, H.V.: Convergence of update aware device scheduling for federated learning at the wireless edge. IEEE Trans. Wirel. Commun. 20(6), 3643–3658 (2021)
Bataineh, A.S., Bentahar, J., Abdel Wahab, O., Mizouni, R., Rjoub, G.: A game-based secure trading of big data and IoT services: blockchain as a two-sided market. In: Kafeza, E., Benatallah, B., Martinelli, F., Hacid, H., Bouguettaya, A., Motahari, H. (eds.) ICSOC 2020. LNCS, vol. 12571, pp. 85–100. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65310-1_7
Chen, H., Huang, S., Zhang, D., Xiao, M., Skoglund, M., Poor, H.V.: Federated learning over wireless IoT networks with optimized communication and resources. IEEE Internet Things J. (2022). https://doi.org/10.1109/JIOT.2022.3151193
Drawel, N., Bentahar, J., Laarej, A., Rjoub, G.: Formalizing group and propagated trust in multi-agent systems. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp. 60–66 (2021)
Drawel, N., Bentahar, J., Laarej, A., Rjoub, G.: Formal verification of group and propagated trust in multi-agent systems. Auton. Agent. Multi-Agent Syst. 36(1), 1–31 (2022)
Fei-Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 594–611 (2006)
Gronauer, S., Diepold, K.: Multi-agent deep reinforcement learning: a survey. Artif. Intell. Rev. 55(2), 895–943 (2022)
Han, J., et al.: Soda10m: a large-scale 2d self/semi-supervised object detection dataset for autonomous driving (2021)
Kasturi, A., Ellore, A.R., Hota, C.: Fusion learning: a one shot federated learning. In: Krzhizhanovskaya, V.V., et al. (eds.) ICCS 2020. LNCS, vol. 12139, pp. 424–436. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50420-5_31
Li, Q., He, B., Song, D.: Practical one-shot federated learning for cross-silo setting. arXiv preprint arXiv:2010.01017 (2020)
Mehdi, M., Bouguila, N., Bentahar, J.: Probabilistic approach for QoS-aware recommender system for trustworthy web service selection. Appl. Intell. 41(2), 503–524 (2014). https://doi.org/10.1007/s10489-014-0537-x
Mnih, V., Kavukcuoglu, et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Nguyen, D.C., Ding, M., Pathirana, P.N., Seneviratne, A., Li, J., Poor, H.V.: Federated learning for internet of things: a comprehensive survey. IEEE Commun. Surv. Tutorials 23(3), 1622–1658 (2021)
Rjoub, G.: Artificial Intelligence Models for Scheduling Big Data Services on the Cloud. Ph.D. thesis, Concordia University, September 2021. https://spectrum.library.concordia.ca/id/eprint/989143/
Rjoub, G., Abdel Wahab, O., Bentahar, J., Bataineh, A.: A trust and energy-aware double deep reinforcement learning scheduling strategy for federated learning on IoT devices. In: Kafeza, E., Benatallah, B., Martinelli, F., Hacid, H., Bouguettaya, A., Motahari, H. (eds.) ICSOC 2020. LNCS, vol. 12571, pp. 319–333. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65310-1_23
Rjoub, G., Bentahar, J.: Cloud task scheduling based on swarm intelligence and machine learning. In: 2017 IEEE 5th International Conference on Future Internet of Things and Cloud (FiCloud), pp. 272–279. IEEE (2017)
Rjoub, G., Bentahar, J., Abdel Wahab, O., Saleh Bataineh, A.: Deep and reinforcement learning for automated task scheduling in large-scale cloud computing systems. Concurrency Comput. Pract. Experience 33(23), e5919 (2021)
Rjoub, G., Bentahar, J., Wahab, O.A.: Bigtrustscheduling: trust-aware big data task scheduling approach in cloud computing environments. Future Gener. Comput. Syst. 110, 1079–1097 (2020)
Rjoub, G., Bentahar, J., Wahab, O.A., Bataineh, A.: Deep smart scheduling: a deep learning approach for automated big data scheduling over the cloud. In: 2019 7th International Conference on Future Internet of Things and Cloud (FiCloud), pp. 189–196. IEEE (2019)
Rjoub, G., Wahab, O.A., Bentahar, J., Bataineh, A.: Trust-driven reinforcement selection strategy for federated learning on IoT devices. Computing 1–23 (2022). https://doi.org/10.1007/s00607-022-01078-1
Rjoub, G., Wahab, O.A., Bentahar, J., Bataineh, A.S.: Improving autonomous vehicles safety in snow weather using federated YOLO CNN learning. In: Bentahar, J., Awan, I., Younas, M., Grønli, T.-M. (eds.) MobiWIS 2021. LNCS, vol. 12814, pp. 121–134. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-83164-6_10
Sami, H., Bentahar, J., Mourad, A., Otrok, H., Damiani, E.: Graph convolutional recurrent networks for reward shaping in reinforcement learning. Inf. Sci. 608, 63–80 (2022)
Sami, H., Otrok, H., Bentahar, J., Mourad, A.: AI-based resource provisioning of IoE services in 6g: a deep reinforcement learning approach. IEEE Trans. Netw. Serv. Manag. 18(3), 3527–3540 (2021)
Shi, W., Zhou, S., Niu, Z., Jiang, M., Geng, L.: Joint device scheduling and resource allocation for latency constrained wireless federated learning. IEEE Trans. Wirel. Commun. 20(1), 453–467 (2020)
Shin, M., Hwang, C., Kim, J., Park, J., Bennis, M., Kim, S.L.: Xor mixup: privacy-preserving data augmentation for one-shot federated learning. arXiv preprint arXiv:2006.05148 (2020)
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Lee, D. D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 5–10 December 2016, Barcelona, Spain, pp. 3630–3638 (2016)
Wahab, O.A.: Intrusion detection in the IoT under data and concept drifts: online deep learning approach. IEEE Internet Things J. (2022). https://doi.org/10.1109/JIOT.2022.3167005
Wahab, O.A., Cohen, R., Bentahar, J., Otrok, H., Mourad, A., Rjoub, G.: An endorsement-based trust bootstrapping approach for newcomer cloud services. Inf. Sci. 527, 159–175 (2020)
Wahab, O.A., Mourad, A., Otrok, H., Taleb, T.: Federated machine learning: survey, multi-level classification, desirable criteria and future directions in communication and networking systems. IEEE Commun. Surv. Tutorials 23(2), 1342–1397 (2021)
Wahab, O.A., Rjoub, G., Bentahar, J., Cohen, R.: Federated against the cold: a trust-based federated learning approach to counter the cold start problem in recommendation systems. Inf. Sci. 601, 189–206 (2022)
Xia, W., Quek, T.Q., Guo, K., Wen, W., Yang, H.H., Zhu, H.: Multi-armed bandit-based client scheduling for federated learning. IEEE Trans. Wirel. Commun. 19(11), 7108–7123 (2020)
Yang, H., Zhao, J., Xiong, Z., Lam, K.Y., Sun, S., Xiao, L.: Privacy-preserving federated learning for UAV-enabled networks: learning-based joint scheduling and resource management. IEEE J. Sel. Areas Commun. 39(10), 3144–3159 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rjoub, G., Bentahar, J., Wahab, O.A., Drawel, N. (2023). One-Shot Federated Learning-based Model-Free Reinforcement Learning. In: Awan, I., Younas, M., Bentahar, J., Benbernou, S. (eds) The International Conference on Deep Learning, Big Data and Blockchain (DBB 2022). DBB 2022. Lecture Notes in Networks and Systems, vol 541. Springer, Cham. https://doi.org/10.1007/978-3-031-16035-6_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-16035-6_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16034-9
Online ISBN: 978-3-031-16035-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)