Abstract
Deep learning has attracted a lot of research attention in the past few years for its efficiency and accuracy. However, there exist two problems of their study. Firstly, the computing power of one single machine is limited and not suitable for handling with training deep learning models with massive cells. Secondly, it costs much to train models on different deep learning frameworks. Motivated by these problems, this paper proposed a deep learning training management platform based on distributed technologies, which integrates different kinds of deep learning frameworks through virtualization technologies and coordinates machines through distributed technologies. Specially, specific algorithms are proposed to solve the multi-task scheduling problem, the computing resources allocation problem and the fault tolerance problem in resource limited scenarios. It turns out that the platform can be widely used in small and medium-sized research teams.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., Barham, P., Chen, J., et al.: TensorFlow: a system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 265–283 (2016)
Paszke, A., Gross, S., Chintala, S., et al.: PyTorch: tensors and dynamic neural networks in Python with strong GPU acceleration (2017). https://github.com/pytorch/pytorch
Jia, Y., Shelhamer, E., Donahue, J., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
Chen, T., Li, M., Li, Y., et al.: MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274 (2015)
Chollet, F.: Keras: the python deep learning library. Astrophysics Source Code Library (2018)
Yeager, L., Bernauer, J., Gray, A., et al.: DIGITS: the deep learning GPU training system. In: ICML 2015 AutoML Workshop (2015)
NVIDIA Docker. https://github.com/NVIDIA/nvidia-docker. Accessed 09 July 2017
Hunt, P., Konar, M., Junqueira, F.P., et al.: ZooKeeper: wait-free coordination for internet-scale systems. In: USENIX Annual Technical Conference, vol. 8, no. 9 (2010)
Chodorow, K.: MongoDB: The Definitive Guide: Powerful and Scalable Data Storage. O’Reilly Media, Inc., Sebastopol (2013)
Hightower, K., Burns, B., Beda, J.: Kubernetes: Up and Running: Dive Into the Future of Infrastructure. O’Reilly Media, Inc., Sebastopol (2017)
Dean, J., Corrado, G., Monga, R., et al.: Large scale distributed deep networks. In: Advances in Neural Information Processing Systems, pp. 1223–1231 (2012)
Sun, S., Chen, W., Bian, J., et al.: Slim-DP: a multi-agent system for communication-efficient distributed deep learning. In: Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems, pp. 721–729. International Foundation for Autonomous Agents and Multiagent Systems (2018)
Ooi, B.C., Tan, K.L., Wang, S., et al.: SINGA: a distributed deep learning platform. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 685–688. ACM (2015)
Gupta, S., Zhang, W., Wang, F.: Model accuracy and runtime tradeoff in distributed deep learning: a systematic study. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 171–180. IEEE (2016)
Cui, H., Zhang, H., Ganger, G.R., et al.: GeePS: scalable deep learning on distributed GPUs with a GPU-specialized parameter server. In: Proceedings of the Eleventh European Conference on Computer Systems, p. 4. ACM (2016)
Frederic, J.: How to automate deep learning training with Kubernetes GPU-cluster [EB/OL], 07 September 2018. https://github.com/Langhalsdino/Kubernetes-GPU-Guide
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, J., Wang, G., Zhang, C., Zhang, B. (2020). Deep Learning Training Management Platform Based on Distributed Technologies in Resource-Constrained Scenarios. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1074. Springer, Cham. https://doi.org/10.1007/978-3-030-32456-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-32456-8_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32455-1
Online ISBN: 978-3-030-32456-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)