Abstract
Edge computing has been developed to utilize heterogeneous computing resources from different physical locations for privacy, cost, and Quality of Service (QoS) reasons. Edge workloads have the characteristics of data-driven, latency-sensitive, and privacy-critical. As a result, edge systems have been developed to be both heterogeneous and distributed to utilize different computing tiers’ resources and features. The unique characteristics of edge workloads and edge systems have motivated EdgeBench, a workflow-based benchmark aiming to provide the ability to explore the full design space of edge applications and edge systems. EdgeBench is both customizable and representative. It allows users to customize the workflow logic of edge workloads, the data storage backends, and the distribution of the individual workflow function to different computing tiers. To illustrate the usability of EdgeBench, we implement two representative edge workflows, a video analytics workflow, and an IoT hub workflow that represent a large portion of today’s edge applications. Both workflows are evaluated using the workflow-level and system-level metrics reported by EdgeBench. We show that EdgeBench can effectively discover the performance bottlenecks and provide improvement implications for the edge workloads and the edge systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
EdgeBench is open-sourced at https://github.com/njjry/EdgeBench.
References
NATS - Open Source Messaging System (2011). https://nats.io/
Eclipse - Paho (2014). https://www.eclipse.org/paho/
VerneMQ - high-performance, distributed MQTT broker (2014). https://vernemq.com/
Choosing between aws lambda data storage options in web apps (2020). https://aws.amazon.com/blogs/compute/choosing-between-aws-lambda-data-storage-options-in-web-apps/
Amazon - Amazon Elastic Kubernetes Service (2021). https://aws.amazon.com/eks/
Amazon - Amazon S3 (2021). https://aws.amazon.com/s3/
Amazon - AWS IoT Greengrass (2021). https://aws.amazon.com/greengrass/
Amazon EC2 - Secure and resizable compute capacity to support virtually any workload (2021). https://aws.amazon.com/ec2/
FFmpeg - FFmpeg (2021). https://ffmpeg.org/
Lightweight Kubernetes - The certified Kubernetes distribution built for IoT & Edge computing (2021). https://k3s.io/
Microsoft - Azure IoT Edge (2021). https://azure.microsoft.com/en-us/services/iot-edge/
Minio - Kubernetes Native, High Performance Object Storage (2021). https://min.io/
OpenCV - = OpenCV (2021). https://opencv.org/
OpenFaaS - Serverless Functions, Made Simple (2021). https://www.openfaas.com/
Prometheus - From metrics to insight. Power your metrics and alerting with a leading open-source monitoring solution (2021). https://prometheus.io/
TSBS Time Series Benchmark Suite (TSBS) (2021). https://github.com/timescale/tsbs
kubernetes (2023). https://kubernetes.io/
Bäurle, S., Mohan, N.: Comb: a flexible, application-oriented benchmark for edge computing. In: Proceedings of the 5th International Workshop on Edge Systems, Analytics and Networking, EdgeSys ’22, pp. 19–24. Association for Computing Machinery, New York (2022)
Das, A., Patterson, S., Wittie, M.: Edgebench: benchmarking edge computing platforms. In: 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), pp. 175–180. IEEE (2018)
Gan, Y., et al.: An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 3–18 (2019)
Hao, T., et al.: Edge aibench: towards comprehensive end-to-end edge computing benchmarking. In: International Symposium on Benchmarking, Measuring and Optimization, pp. 23–30. Springer (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Lin, S.-C., et al.: The architectural implications of autonomous driving: Constraints and acceleration. In: Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 751–766 (2018)
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Luo, C., Zhang, F., Huang, C., Xiong, X., Chen, J., Wang, L., Gao, W., Ye, H., Wu, T., Zhou, R., Zhan, J.: AIoT bench: towards comprehensive benchmarking mobile and embedded device intelligence. In: Zheng, C., Zhan, J. (eds.) Bench 2018. LNCS, vol. 11459, pp. 31–35. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32813-9_4
McChesney, J., Wang, N., Tanwer, A., de Lara, E., Varghese, B.: Defog: fog computing benchmarks. In: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, pp. 47–58 (2019)
Olguín, M., Muñoz, Wang, J., Satyanarayanan, M., Gross, J.: Edgedroid: an experimental approach to benchmarking human-in-the-loop applications. In: Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications, pp. 93–98 (2019)
Rajput, K.R., Kulkarni, C.D., Cho, B., Wang, W., Kim, I.K.: Edgefaasbench: benchmarking edge devices using serverless computing. In: 2022 IEEE International Conference on Edge Computing and Communications (EDGE), pp. 93–103 (2022)
Shi, W., Cao, J., Zhang, Q., Li, Y., Xu, L.: Edge computing: vision and challenges. IEEE Internet Things J. 3(5), 637–646 (2016)
Varghese, B., et al.: A survey on edge benchmarking. ACM Computing Surveys (2020)
Wang, Y., Liu, S., Wu, X., Shi, W.: Cavbench: a benchmark suite for connected and autonomous vehicles. In: 2018 IEEE/ACM Symposium on Edge Computing (SEC), pp. 30–42. IEEE (2018)
Zhang, X., Wang, Y., Shi, W.: pcamp: performance comparison of machine learning packages on the edges. In: USENIX Workshop on Hot Topics in Edge Computing (HotEdge 18) (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, Q., Jin, R., Gandhi, N., Ge, X., Khouzani, H.A., Zhao, M. (2024). EdgeBench: A Workflow-Based Benchmark for Edge Computing. In: Arai, K. (eds) Advances in Information and Communication. FICC 2024. Lecture Notes in Networks and Systems, vol 921. Springer, Cham. https://doi.org/10.1007/978-3-031-54053-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-54053-0_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54052-3
Online ISBN: 978-3-031-54053-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)