Abstract
Federated learning is a revolutionary machine learning approach whose main idea is to train the machine learning model in a distributed fashion over a large number of edge/end devices without having to share the raw data. We consider in this work a federated learning scenario wherein the local training is carried out on IoT devices and the global aggregation is done at the level of an edge server. One essential challenge in this emerging approach is scheduling, i.e., how to select the IoT devices to participate in the distributed training process. The existing approaches suggest to base the scheduling decision on the resource characteristics of the devices to guarantee that the selected devices would have enough resources to carry out the training. In this work, we argue that trust should be an integral part of the decision-making process and therefore design a trust establishment mechanism between the edge server and IoT devices. The trust mechanism aims to detect those IoT devices that over-utilize or under-utilize their resources during the local training. Thereafter, we design a Double Deep Q Learning (DDQN)-based scheduling algorithm that takes into account the trust scores and energy levels of the IoT devices to make appropriate scheduling decisions. Experiments conducted using a real-world dataset (https://www.cs.toronto.edu/~kriz/cifar.html) show that our DDQN solution always achieves better performance compared to the DQN and random scheduling algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anh, T.T., Luong, N.C., Niyato, D., Kim, D.I., Wang, L.C.: Efficient training management for mobile crowd-machine learning: a deep reinforcement learning approach. IEEE Wirel. Commun. Lett. 8(5), 1345–1348 (2019)
Bataineh, A.S., Mizouni, R., Bentahar, J., Barachi, M.E.: Toward monetizing personal data: a two-sided market analysis. Future Gener. Comput. Syst. 111, 435–459 (2020)
Bataineh, A.S., Mizounib, R., El Barachic, M., Bentahara, J.: Monetizing personal data: a two-sided market approach. Procedia Comput. Sci. 83, 472–479 (2016)
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H.V., Cui, S.: A joint learning and communications framework for federated learning over wireless networks. arXiv preprint arXiv:1909.07972 (2019)
Dai, H., Zeng, X., Yu, Z., Wang, T.: A scheduling algorithm for autonomous driving tasks on mobile edge computing servers. Syst. Arch. 94, 14–23 (2019)
Ding, D., Fan, X., Zhao, Y., Kang, K., Yin, Q., Zeng, J.: Q-learning based dynamic task scheduling for energy-efficient cloud computing. Future Gener. Comput. Syst. 108, 361–371 (2020)
Drawel, N., Bentahar, J., Laarej, A., Rjoub, G.: Formalizing group and propagated trust in multi-agent systems. In: Bessiere, C. (ed.) Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pp. 60–66 (2020)
Hu, C., Jiang, J., Wang, Z.: Decentralized federated learning: a segmented gossip approach. arXiv preprint arXiv:1908.07782 (2019)
Huang, J., Li, S., Chen, Y.: Revenue-optimal task scheduling and resource management for IoT batch jobs in mobile edge computing. Peer-to-Peer Netw. Appl., 1–12 (2020)
Iglewicz, B., Hoaglin, D.C.: How to Detect and Handle Outliers, vol. 16. ASQ Press (1993)
Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., Shen, X.: Deep reinforcement learning for autonomous internet of things: model, applications and challenges. IEEE Commun. Surv. Tutor. 22(3), 1722–1760 (2020)
Lin, C.C., Deng, D.J., Chih, Y.L., Chiu, H.T.: Smart manufacturing scheduling with edge computing using multiclass deep Q network. IEEE Trans. Ind. Inform. 15(7), 4276–4284 (2019)
Luo, S.: Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Appl. Soft Comput., 106208 (2020)
Mao, H., Alizadeh, M., Menache, I., Kandula, S.: Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM Workshop on Hot Topics in Networks, pp. 50–56 (2016)
Nguyen, H.T., Luong, N.C., Zhao, J., Yuen, C., Niyato, D.: Resource allocation in mobility-aware federated learning networks: a deep reinforcement learning approach. arXiv preprint arXiv:1910.09172 (2019)
Nishio, T., Yonetani, R.: Client selection for federated learning with heterogeneous resources in mobile edge. In: IEEE International Conference on Communications (ICC), pp. 1–7 (2019)
Rjoub, G., Bentahar, J., Abdel Wahab, O., Saleh Bataineh, A.: Deep and reinforcement learning for automated task scheduling in large-scale cloud computing systems. Concurr. Comput. Pract. Exp. (2020)
Rjoub, G., Bentahar, J., Wahab, O.A.: BigTrustScheduling: trust-aware big data task scheduling approach in cloud computing environments. Future Gener. Comput. Syst. 110, 1079–1097 (2020)
Rjoub, G., Bentahar, J., Wahab, O.A., Bataineh, A.: Deep smart scheduling: a deep learning approach for automated big data scheduling over the cloud. In: 7th International Conference on Future Internet of Things and Cloud (FiCloud), pp. 189–196 (2019)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: Thirtieth AAAI Conference on Artificial Intelligence (2016)
Wahab, O.A., Bentahar, J., Otrok, H., Mourad, A.: Resource-aware detection and defense system against multi-type attacks in the cloud: repeated Bayesian stackelberg game. IEEE Trans. Dependable Secur. Comput. (2019)
Wahab, O.A., Cohen, R., Bentahar, J., Otrok, H., Mourad, A., Rjoub, G.: An endorsement-based trust bootstrapping approach for newcomer cloud services. Inf. Sci. 527, 159–175 (2020)
Wang, X., Han, Y., Wang, C., Zhao, Q., Chen, X., Chen, M.: In-edge AI: intelligentizing mobile edge computing, caching and communication by federated learning. IEEE Netw. 33(5), 156–165 (2019)
Yang, H.H., Liu, Z., Quek, T.Q.S., Poor, H.V.: Scheduling policies for federated learning in wireless networks. IEEE Trans. Commun. 68(1), 317–333 (2020)
Zhou, Z., Yang, S., Pu, L., Yu, S.: CEFL: online admission control, data scheduling, and accuracy tuning for cost-efficient federated learning across edge nodes. IEEE Internet Things J. 7(10), 9341–9356 (2020)
Zhu, G., Liu, D., Du, Y., You, C., Zhang, J., Huang, K.: Toward an intelligent edge: wireless communication meets machine learning. IEEE Commun. Mag. 58(1), 19–25 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Rjoub, G., Abdel Wahab, O., Bentahar, J., Bataineh, A. (2020). A Trust and Energy-Aware Double Deep Reinforcement Learning Scheduling Strategy for Federated Learning on IoT Devices. In: Kafeza, E., Benatallah, B., Martinelli, F., Hacid, H., Bouguettaya, A., Motahari, H. (eds) Service-Oriented Computing. ICSOC 2020. Lecture Notes in Computer Science(), vol 12571. Springer, Cham. https://doi.org/10.1007/978-3-030-65310-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-65310-1_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65309-5
Online ISBN: 978-3-030-65310-1
eBook Packages: Computer ScienceComputer Science (R0)