Abstract
Deep learning-based visual analytic applications have drawn attention by suggesting fruitful combinations with Deep Neural Network (DNN) models and visual data sensors. Because of the high cost of DNN inference, most systems adopt offloading techniques utilizing a high-end cloud. However, tasks that require real-time streaming often suffer from the problem of an imbalanced pipeline due to the limited bandwidth and latency between camera sensors and the cloud. Several DNN slicing approaches show that effectively utilizing the edge computing paradigm effectively lowers the frame drop rate and overall latency, but recent research has primarily focused on building a general framework that only considers a few fixed settings. However, we observed that the optimal split strategy for DNN models can vary significantly based on application requirements. Hence, we focus on the characteristics and explainability of split points derived from various application goals. First, we propose a new simulation framework for flexible software-level configuration, including latency and bandwidth, using dockercompose, and we experiment on a 14-layered Convolutional Neural Network (CNN) model with diverse layer types. We report the results of the total process time and frame drop rate of 50 frames with three different configurations and further discuss recommendations for providing proper decision guidelines on split points, considering the target goals and properties of the CNN layers.
E. Cho and J. Yoon—These authors contributed equally.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Fowers, J., et al.: A configurable cloud-scale DNN processor for real-time AI. In: 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), pp. 1–14. IEEE (2018)
Hsu, K.J., Bhardwaj, K., Gavrilovska, A.: Couper: DNN model slicing for visual analytics containers at the edge. In: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, pp. 179–194 (2019)
Lockhart, L., Harvey, P., Imai, P., Willis, P., Varghese, B.: Scission: performance-driven and context-aware cloud-edge distribution of deep neural networks. In: 2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC), pp. 257–268. IEEE (2020)
Ren, P., Qiao, X., Huang, Y., Liu, L., Dustdar, S., Chen, J.: Edge-assisted distributed DNN collaborative computing approach for mobile web augmented reality in 5G networks. IEEE Network 34(2), 254–261 (2020)
Teerapittayanon, S., McDanel, B., Kung, H.T.: Distributed deep neural networks over the cloud, the edge and end devices. In: IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 328–339 (2017)
Talagala, N., et al.: ECO: harmonizing edge and cloud with ML/DL orchestration. In: USENIX Workshop on Hot Topics in Edge Computing (HotEdge 2018) (2018)
Tao, Z., Li, Q.: ESGD: communication efficient distributed deep learning on the edge. In: USENIX Workshop on Hot Topics in Edge Computing (HotEdge 2018) (2018)
Teerapittayanon, S., McDanel, B., Kung, H.T.: Distributed deep neural networks over the cloud, the edge and end devices. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 328–339. IEEE (2017)
Wang, X., Luo, Y., Crankshaw, D., Tumanov, A., Yu, F., Gonzalez, J.E.: IDK cascades: fast deep learning by learning not to overthink. arXiv preprint arXiv:1706.00885 (2017)
Acknowledgement
This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2021-2020-0-01795) and (No. 2015-0-00250, (SW Star Lab) Software R&D for Model-based Analysis and Verification of Higher-order Large Complex System) supervised by the IITP (Institute of Information & Communications Technology Planning & Evaluation).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Cho, E., Yoon, J., Baek, D., Lee, D., Bae, DH. (2022). DNN Model Deployment on Distributed Edges. In: Bakaev, M., Ko, IY., Mrissa, M., Pautasso, C., Srivastava, A. (eds) ICWE 2021 Workshops. ICWE 2021. Communications in Computer and Information Science, vol 1508. Springer, Cham. https://doi.org/10.1007/978-3-030-92231-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-92231-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92230-6
Online ISBN: 978-3-030-92231-3
eBook Packages: Computer ScienceComputer Science (R0)