Vladimir Podolskiy
ABSTRACT
Federated learning (FL) applications face the challenges of managing the temporary state at scale combined with the limited compute and storage capacity of the infrastructure and the heterogeneity/dynamism of the deployment environment. The paper proposes Rubberband - a middleware designed to manage the state in the FL deployments at scale.
- TensorFlow Authors. 2019. TensorFlow Federated. https://www.tensorflow.org/federated. Accessed: 2020-10-28.Google Scholar
- M. Dian. Bah, Adel Hafiane, and Raphael Canals. 2018. Deep Learning with unsupervised data labeling for weeds detection on UAV images. arXiv:1805.12395 [cs.CV]Google Scholar
- Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, and Karol Zieba. 2016. End to End Learning for Self-Driving Cars. arXiv:1604.07316 [cs.CV]Google Scholar
- Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. 2019. Towards Federated Learning at Scale: System Design. arXiv:1902.01046 [cs.LG]Google Scholar
- Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Practical Secure Aggregation for Privacy-Preserving Machine Learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA) (CCS '17). Association for Computing Machinery, New York, NY, USA, 1175--1191. https://doi.org/10.1145/3133956.3133982 Google ScholarDigital Library
- C/LM LAN/MAN Standards Committee. [n.d.]. IEEE P802.11ax -IEEE Draft Standard for Information Technology - Telecommunications and Information Exchange Between Systems Local and Metropolitan Area Networks - Specific Requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications Amendment Enhancements for High Efficiency WLAN. https://standards.ieee.org/project/802_11ax.html. Accessed: 2020-08-26.Google Scholar
- Ayn de Jesus. 2019. Drones for Agriculture - Current Applications. https://emerj.com/ai-sector-overviews/drones-for-agriculture-current-applications/. Accessed: 2020-07-18.Google Scholar
- J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09.Google Scholar
- Kevin Dick, Luke Russell, Yasmina Souley Dosso, Felix Kwamena, and James R. Green. 2019. Deep Learning for Critical Infrastructure Resilience. Journal of Infrastructure Systems 25, 2 (2019), 05019003. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000477Google ScholarCross Ref
- Jessica Dolcourt. [n.d.]. We tested 5G speeds across the globe. https://www.cnet.com/features/we-ran-5g-speed-tests-on-verizon-at-t-ee-and-more-heres-what-we-found/. Accessed: 2020-08-24.Google Scholar
- Adam Grzywaczewski. [n.d.]. Training AI for Self-Driving Vehicles: the Challenge of Scale. https://developer.nvidia.com/blog/training-self-driving-vehicles-challenge-scale/. Accessed: 2020-07-18.Google Scholar
- Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao. 2019. Advances and Open Problems in Federated Learning.arXiv:1912.04977 [cs.LG]Google Scholar
- Wenqi Li, Fausto Milletarì, Daguang Xu, Nicola Rieke, Jonny Hancox, Wentao Zhu, Maximilian Baust, Yan Cheng, Sébastien Ourselin, M. Jorge Cardoso, and Andrew Feng. 2019. Privacy-Preserving Federated Brain Tumour Segmentation. In Machine Learning in Medical Imaging, Heung-Il Suk, Mingxia Liu, Pingkun Yan, and Chunfeng Lian (Eds.). Springer International Publishing, Cham, 133--141.Google Scholar
- M. Liu, H.Jiang, J. Chen, A. Badokhon, X. Wei, and M. Huang. 2016. A Collaborative Privacy-Preserving Deep Learning System in Distributed Mobile Environment. In 2016 International Conference on Computational Science and Computational Intelligence (CSCI). 192--197.Google Scholar
- Elmer R. Magsino and Ivan Wang-Hei Ho. 2018. Roadside Unit Allocation for Fog-Based Information Sharing in Vehicular Networks. In Proceedings of the 1st ACM International Workshop on Smart Cities and Fog Computing (Shenzhen, China) (CitiFog'18). Association for Computing Machinery, New York, NY, USA, 7--12. https://doi.org/10.1145/3277893.3277897 Google ScholarDigital Library
- H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2016. Communication-Efficient Learning of Deep Networks from Decentralized Data. arXiv:1602.05629 [cs.LG]Google Scholar
- Pitch Patarasuk and Xin Yuan. 2009. Bandwidth Optimal All-Reduce Algorithms for Clusters of Workstations. J. Parallel Distrib. Comput. 69, 2 (Feb. 2009), 117--124. https://doi.org/10.1016/j.jpdc.2008.09.002 Google ScholarDigital Library
- Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. 2019. Do ImageNet Classifiers Generalize to ImageNet? arXiv:1902.10811 [cs.CV]Google Scholar
- Nicola Rieke, Jonny Hancox, Wenqi Li, Fausto Milletarì, Holger R. Roth, Shadi Albarqouni, Spyridon Bakas, Mathieu N. Galtier, Bennett A. Landman, Klaus Maier-Hein, Sébastien Ourselin, Micah Sheller, Ronald M. Summers, Andrew Trask, Daguang Xu, Maximilian Baust, and M. Jorge Cardoso. 2020. The future of digital health with federated learning. npj Digital Medicine 3, 1 (14 Sep 2020), 119. https://doi.org/10.1038/s41746-020-00323-1Google Scholar
- Antony Rowstron and Peter Druschel. 2001. Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems. In Middleware 2001, Rachid Guerraoui (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 329--350. Google ScholarDigital Library
- Abhijit Guha Roy, Shayan Siddiqui, Sebastian Pölsterl, Nassir Navab, and Christian Wachinger. 2019. BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning. arXiv:1905.06731 [cs.LG]Google Scholar
- Theo Ryffel, Andrew Trask, Morten Dahl, Bobby Wagner, Jason Mancuso, Daniel Rueckert, and Jonathan Passerat-Palmbach. 2018. A generic framework for privacy preserving deep learning. arXiv:1811.04017 [cs.LG]Google Scholar
- S. Samarakoon, M. Bennis, W. Saad, and M. Debbah. 2018. Federated Learning for Ultra-Reliable Low-Latency V2V Communications. In 2018 IEEE Global Communications Conference (GLOBECOM). 1--7.Google Scholar
- Alexander Sergeev and Mike Del Balso. 2018. Horovod: fast and easy distributed deep learning in TensorFlow. arXiv preprint arXiv:1802.05799 (2018).Google Scholar
- Muhammad Shayan, Clement Fung, Chris J. M. Yoon, and Ivan Beschastnikh. 2018. Biscotti: A Ledger for Private and Secure Peer-to-Peer Machine Learning. arXiv:1811.09904 [cs.LG]Google Scholar
- Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, and Hari Balakrishnan. 2001. Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications. In Proceedings of the 2001 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (San Diego, California, USA) (SIGCOMM '01). Association for Computing Machinery, New York, NY, USA, 149--160. https://doi.org/10.1145/383059.383071 Google ScholarDigital Library
- CISCO Systems. 2019. Cisco Visual Networking Index 2018. Technical Report. https://cyrekdigital.com/pl/blog/content-marketing-trendy-na-rok-2019/white-paper-c11-741490.pdfGoogle Scholar
- CISCO Systems. 2019. Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update 2017--2022. Technical Report. https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.htmlGoogle Scholar
- Keysight Technologies. [n.d.]. MIMO Technology in LTE Operation and Measurement. https://www.keysight.com/en/en/assets/7018-02324/application-notes/5990-4760.pdf. Accessed: 2020-08-26.Google Scholar
- Animesh Trivedi, Lin Wang, Henri Bal, and Alexandru Iosup. 2020. Sharing and Caring of Data at the Edge. In 3rd USENIX Workshop on Hot Topics in Edge Computing (HotEdge 20). USENIX Association. https://www.usenix.org/conference/hotedge20/presentation/trivediGoogle Scholar
- Spyros Voulgaris, Daniela Gavidia, and Maarten van Steen. 2005. CYCLON: Inexpensive Membership Management for Unstructured P2P Overlays. Journal of Network and Systems Management 13, 2 (01 Jun 2005), 197--217. https://doi.org/10.1007/s10922-005-4441-xGoogle ScholarCross Ref
- M. Wu and J. Luo. 2019. Wearable technology applications in healthcare: A literature review. Online Journal of Nursing Informatics (OJNI) 23, 3 (2019). https://www.himss.org/resources/wearable-technology-applications-healthcare-literature-reviewGoogle Scholar
- Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, and Françoise Beaufays. 2018. Applied Federated Learning: Improving Google Keyboard Query Suggestions. arXiv:1812.02903 [cs.LG]Google Scholar
Index Terms
- Rubberband: Enabling Elastic Federated Learning with the Temporary State Management
Recommendations
On the feasibility of Federated Learning towards on-demand client deployment at the edge
AbstractNowadays, researchers are investing their time and devoting their efforts in developing and motivating the 6G vision and resources that are not available in 5G. Edge computing and autonomous vehicular driving applications are more ...
Highlights- A novel on-demand client deployment in Federated learning.
- Targeting the ...
RuCIL: Enabling Privacy-Enhanced Edge Computing for Federated Learning
Edge Computing – EDGE 2023AbstractFederated learning has emerged as a promising approach for collaborative machine learning while preserving data privacy in distributed settings. Despite recent advancements, challenges such as privacy preservation and communication overhead ...
A Federated Learning Algorithms Development Paradigm
Engineering of Computer-Based SystemsAbstractAt present many distributed and decentralized frameworks for federated learning algorithms are already available. However, development of such a framework targeting smart Internet of Things in edge systems is still an open challenge. A solution to ...
Comments