Hansong Zhou
ABSTRACT
Future mobile edge computing (MEC) is envisioned to provide federated intelligence to delay-sensitive learning tasks with multimodal data. Conventional horizontal federated learning (FL) suffers from high resource demand in response to complicated multi-modal models. Multi-modal FL (MFL), on the other hand, offers a more efficient approach for learning from multi-modal data. In MFL, the entire multi-modal model is split into several sub-models with each tailored to a specific data modality and trained on a designated edge. As sub-models are considerably smaller than the multi-modal model, MFL requires fewer computation resources and reduces communication time. Nevertheless, deploying MFL over MEC faces the challenges of device mobility and edge heterogeneity, which, if not addressed, could negatively impact MFL performance. In this paper, we investigate an Service Migration-assisted Mobile Multi-modal Federated Learning (SM3FL) framework, where the service migration for sub-models between edges is enabled. To effectively utilize both communication and computation resources without extravagance in SM3FL, we develop the optimal strategies of service migration and data sample collection to minimize the wall-clock time, defined as the required training time to reach the learning target. Our experiment results show that the proposed SM3FL framework demonstrates remarkable performance, surpassing other state-of-art FL frameworks via substantially reducing the computing demand by 17.5% and dramatically decreasing the wall-clock time by 25.3%.
- Alaa Awad Abdellatif, Amr Mohamed, Carla Fabiana Chiasserini, Mounira Tlili, and Aiman Erbad. 2019. Edge Computing for Smart Health: Context-Aware Approaches, Opportunities, and Challenges. IEEE Network 33, 3 (2019), 196--203.Google Scholar
Cross Ref
- Sergio Barbarossa, Stefania Sardellitti, and Paolo Di Lorenzo. 2014. Communicating While Computing: Distributed mobile cloud computing over 5G heterogeneous networks. IEEE Signal Processing Magazine 31, 6 (2014), 45--55.Google ScholarCross Ref
- Ilker Bozcan and Erdal Kayacan. 2020. AU-AIR: A Multi-modal Unmanned Aerial Vehicle Dataset for Low Altitude Traffic Surveillance. In 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, Virtual, 8504--8510.Google Scholar
- Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. 2020. nuScenes: A Multimodal Dataset for Autonomous Driving. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Virtual, 11618--11628.Google ScholarCross Ref
- Timothy Castiglia, Shiqiang Wang, and Stacy Patterson. 2022. Flexible Vertical Federated Learning with Heterogeneous Parties. https://arxiv.org/abs/2208.12672Google Scholar
- Tianyi Chen, Xiao Jin, Yuejiao Sun, and Wotao Yin. 2020. VAFL: a Method of Vertical Asynchronous Federated Learning. https://arxiv.org/abs/2007.06081Google Scholar
- Bart Cox, Lydia Y. Chen, and Jérémie Decouchant. 2022. Aergia: Leveraging Heterogeneity in Federated Learning Systems. In Proceedings of the 23rd ACM/IFIP International Middleware Conference (Quebec, QC, Canada) (Middleware '22). Association for Computing Machinery, New York, NY, USA, 107--120.Google ScholarDigital Library
- R.L Graham. 1969. Bounds on Multiprocessing Timing Anomalies. SIAM J. Appl. Math. 17, 2 (1969), 416--429.Google ScholarDigital Library
- Wei Han, Hui Chen, and Soujanya Poria. 2021. Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis. In Proceedings of the 2021 Conference on Empirical Methods in NLP. Association for Computational Linguistics, Punta Cana, Dominican Republic, 9180--9192.Google ScholarCross Ref
- Mounssif Krouka, Anis Elgabli, Chaouki Ben Issaid, and Mehdi Bennis. 2022. Communication-Efficient and Federated Multi-Agent Reinforcement Learning. IEEE Transactions on Cognitive Communications and Networking 8, 1 (2022), 311--320.Google ScholarCross Ref
- Dana Lahat, Tülay Adali, and Christian Jutten. 2015. Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects. Proc. IEEE 103, 9 (2015), 1449--1477.Google Scholar
- Liangzhi Li, Kaoru Ota, and Mianxiong Dong. 2018. Deep Learning for Smart Industry: Efficient Manufacture Inspection System With Fog Computing. IEEE Transactions on Industrial Informatics 14, 10 (2018), 4665--4673.Google ScholarCross Ref
- Shaoshan Liu, Liangkai Liu, Jie Tang, Bo Yu, Yifan Wang, and Weisong Shi. 2019. Edge Computing for Autonomous Driving: Opportunities and Challenges. Proc. IEEE 107, 8 (2019), 1697--1716.Google Scholar
- Yang Liu, Xinwei Zhang, Yan Kang, Liping Li, Tianjian Chen, Mingyi Hong, and Qiang Yang. 2022. FedBCD: A Communication-Efficient Collaborative Learning Framework for Distributed Features. IEEE Transactions on Signal Processing 70 (2022), 4277--4290.Google ScholarCross Ref
- Qianxia Ma, Yongfang Nie, Jingyan Song, and Tao Zhang. 2020. Multimodal Data Processing Framework for Smart City: A Positional-Attention Based Deep Learning Approach. IEEE Access 8 (2020), 215505--215515.Google ScholarCross Ref
- Pavel Mach and Zdenek Becvar. 2017. Mobile Edge Computing: A Survey on Architecture and Computation Offloading. IEEE Communications Surveys and Tutorials 19, 3 (2017), 1628--1656. Google ScholarDigital Library
- Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. JMLR, Fort Lauderdale, Florida, USA, 1273--1282.Google Scholar
- Stefan Nastic, Thomas Rausch, Ognjen Scekic, Schahram Dustdar, Marjan Gusev, Bojana Koteska, Magdalena Kostoska, Boro Jakimovski, Sasko Ristov, and Radu Prodan. 2017. A Serverless Real-Time Data Analytics Platform for Edge Computing. IEEE Internet Computing 21, 4 (2017), 64--71.Google ScholarDigital Library
- Solmaz Niknam, Harpreet S. Dhillon, and Jeffrey H. Reed. 2020. Federated Learning for Wireless Communications: Motivation, Opportunities, and Challenges. IEEE Communications Magazine 58, 6 (2020), 46--51.Google ScholarCross Ref
- Haixia Peng, Qiang Ye, and Xuemin Shen. 2020. Spectrum Management for Multi-Access Edge Computing in Autonomous Vehicular Networks. IEEE Transactions on Intelligent Transportation Systems 21, 7 (2020), 3001--3012.Google ScholarDigital Library
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1532--1543.Google ScholarCross Ref
- Nikos Piperigkos, Aris S. Lalos, and Kostas Berberidis. 2021. Multi-modal cooperative awareness of connected and automated vehicles in smart cities. In 2021 IEEE International Conference on Smart Internet of Things (SmartIoT). IEEE, Jeju, Korea, 377--382.Google ScholarCross Ref
- Dhanesh Ramachandram and Graham W. Taylor. 2017. Deep Multimodal Learning: A Survey on Recent Advances and Trends. IEEE Signal Processing Magazine 34, 6 (2017), 96--108.Google ScholarCross Ref
- Dian Shi, Liang Li, Maoqiang Wu, Minglei Shu, Rong Yu, Miao Pan, and Zhu Han. 2022. To Talk or to Work: Dynamic Batch Sizes Assisted Time Efficient Federated Learning Over Future Mobile Edge Devices. IEEE Transactions on Wireless Communications 21, 12 (2022), 11038--11050.Google ScholarCross Ref
- Tarik Taleb, Adlen Ksentini, and Pantelis A. Frangoudis. 2019. Follow-Me Cloud: When Cloud Services Follow Mobile Users. IEEE Transactions on Cloud Computing 7, 2 (2019), 369--382.Google ScholarCross Ref
- Md. Zia Uddin. 2019. A wearable sensor-based activity prediction system to facilitate edge computing in smart healthcare system. J. Parallel and Distrib. Comput. 123 (2019), 46--53.Google ScholarCross Ref
- Prabal Verma and Sandeep K. Sood. 2018. Fog Assisted-IoT Enabled Patient Health Monitoring in Smart Homes. IEEE Internet of Things Journal 5, 3 (2018), 1789--1796.Google ScholarCross Ref
- Shangguang Wang, Jinliang Xu, Ning Zhang, and Yujiong Liu. 2018. A survey on service migration in mobile edge computing. IEEE Access 6 (2018), 23511--23528.Google ScholarCross Ref
- Wei Yu, Fan Liang, Xiaofei He, William Grant Hatcher, Chao Lu, Jie Lin, and Xinyu Yang. 2018. A Survey on the Edge Computing for the Internet of Things. IEEE Access 6 (2018), 6900--6919.Google ScholarCross Ref
- Amir Zadeh, Rowan Zellers, Eli Pincus, and Louis-Philippe Morency. 2016. MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos. https://arxiv.org/abs/1606.06259Google Scholar
- Qixun Zhang, Hao Wen, Ying Liu, Shuo Chang, and Zhu Han. 2022. Federated-Reinforcement-Learning-Enabled Joint Communication, Sensing, and Computing Resources Allocation in Connected Automated Vehicles Networks. IEEE Internet of Things Journal 9, 22 (2022), 23224--23240.Google ScholarCross Ref
- Xiaonan Zhang, Sihan Yu, Hansong Zhou, Pei Huang, Linke Guo, and Ming Li. 2023. Signal Emulation Attack and Defense for Smart Home IoT. IEEE Transactions on Dependable and Secure Computing 20, 3 (2023), 2040--2057.Google ScholarDigital Library
- Hansong Zhou, Sihan Yu, Xiaonan Zhang, Linke Guo, and Beatriz Lorenzo. 2022. DQN-based QoE Enhancement for Data Collection in Heterogeneous IoT Network. In 2022 IEEE 19th International Conference on Mobile Ad Hoc and Smart Systems (MASS). IEEE, Denver, CO, 188--194.Google Scholar
- Jiancun Zhou, Tao Xu, Sheng Ren, and Kehua Guo. 2020. Two-Stage Spatial Mapping for Multimodal Data Fusion in Mobile Crowd Sensing. IEEE Access 8 (2020), 96727--96737.Google ScholarCross Ref
Index Terms
- Waste Not, Want Not: Service Migration-Assisted Federated Intelligence for Multi-Modality Mobile Edge Computing
Recommendations
Context-Aware Multi-User Offloading in Mobile Edge Computing: a Federated Learning-Based Approach
AbstractMobile edge computing (MEC) provides an effective solution to help the Internet of Things (IoT) devices with delay-sensitive and computation-intensive tasks by offering computing capabilities in the proximity of mobile device users. Most of the ...
Modelling Task Offloading Mobile Edge Computing
ICCDE '22: Proceedings of the 2022 8th International Conference on Computing and Data EngineeringWith the rapid growth of mobile devices (such as smart phones and IoT devices) and the upcoming 5G era, it has been considered that edge computing will play a significant role, which together with the Cloud server forms the Mobile Edge Computing (MEC) ...
Edge-Assisted Federated Learning: An Empirical Study from Software Decomposition Perspective
Algorithms and Architectures for Parallel ProcessingAbstractFederated learning is considered to be a privacy-preserving collaborative machine learning training method. However, due to the general limitation of the computing ability of the terminal device, the training efficiency becomes an issue when ...
Comments