skip to main content
10.1145/3629104.3666037acmconferencesArticle/Chapter ViewAbstractPublication PagesdebsConference Proceedingsconference-collections
research-article
Open access

Color-based Lightweight Utility-aware Load Shedding for Real-Time Video Analytics at the Edge

Published: 22 July 2024 Publication History

Abstract

Real-time video analytics typically require video frames to be processed by a query to identify objects or activities of interest while adhering to an end-to-end frame processing latency constraint. This imposes a continuous and heavy load on backend compute and network infrastructure. Video data has inherent redundancy and does not always contain an object of interest for a given query. We leverage this property of video streams to propose a lightweight Load Shedder that can be deployed on edge servers or on inexpensive edge devices co-located with cameras. The proposed Load Shedder uses pixel-level color-based features to calculate a utility score for each ingress video frame and a minimum utility threshold to select interesting frames to send for query processing. Dropping unnecessary frames enables the video analytics query in the backend to meet the end-to-end latency constraint with fewer compute and network resources. To guarantee a bounded end-to-end latency at runtime, we introduce a control loop that monitors the backend load and dynamically adjusts the utility threshold. Performance evaluations show that the proposed Load Shedder selects a large portion of frames containing each object of interest while meeting the end-to-end frame processing latency constraint. Furthermore, it does not impose a significant latency overhead when running on edge devices with modest compute resources.

References

[1]
Z. Xu, S. Sinha, S. Harshil S, and U. Ramachandran, "Space-time vehicle tracking at the edge of the network," in Proc. of the 2019 Workshop on Hot Topics in Video Analytics and Intelligent Edges. ACM, 2019.
[2]
T. T. Le, S. T. Tran, S. Mita, and T. D. Nguyen, "Real time traffic sign detection using color and shape-based features," in Asian Conference on Intelligent Information and Database Systems. Springer, 2010.
[3]
Wobot.ai. (2021) Team Wobot how is video analytics driving industry 4.0. Wobot Intelligence. [Online]. Available: https://wobot.ai/video-analytics/how-is-video-analytics-driving-industry-4-0/
[4]
G. Ananthanarayanan, P. Bahl, P. Bodík, K. Chintalapudi, M. Philipose, L. Ravindranath, and S. Sinha, "Real-time video analytics: The killer app for edge computing," computer, vol. 50, no. 10, pp. 58--67, 2017.
[5]
U. Ramachandran, H. Gupta, A. Hall, E. Saurez, and Z. Xu, "A case for elevating the edge to be a peer of the cloud," GetMobile: Mobile Computing and Communications, vol. 24, no. 3, pp. 14--19, 2021.
[6]
M. Zhang, F. Wang, Y. Zhu, J. Liu, and Z. Wang, "Towards cloud-edge collaborative online video analytics with fine-grained serverless pipelines," in Proc. of the ACM Multimedia Systems Conference, 2021.
[7]
N. Tatbul and S. Zdonik, "Window-aware load shedding for aggregation queries over data streams," in Proc. of VLDB, 2006.
[8]
A. Slo, S. Bhowmik and K. Rothermel, "espice: Probabilistic load shedding from input event streams in complex event processing," in Proc. of the 20th International Middleware Conference, 2019.
[9]
A. Slo, S. Bhowmik, A. Flaig, and K. Rothermel, "pspice: partial match shedding for complex event processing," in Proc. of the Int. Conf. on Big Data (Big Data). IEEE, 2019.
[10]
C. Canel, T. Kim, G. Zhou, C. Li, H. Lim, D. G. Andersen, M. Kaminsky, and S. R. Dulloor, "Scaling video analytics on constrained edge nodes," CoRR, vol. abs/1905.13536, 2019.
[11]
C. Zhang, Q. Cao, H.Jiang, W. Zhang, J. Li, and J. Yao, "A fast filtering mechanism to improve efficiency of large-scale video analytics," IEEE Transactions on Computers, vol. 69, no. 6, pp. 914--928, 2020.
[12]
Y. Li, A. Padmanabhan, P. Zhao, Y. Wang, G. H. Xu, and R. Netravali, "Reducto: On-camera filtering for resource-efficient real-time video analytics," in Proc. of SIGCOMM, 2020.
[13]
U. K. Pillai and D. Valles, "Vehicle type and color classification and detection for amber and silver alert emergencies using machine learning," in IEEE Int. IOT, Electronics and Mechatronics Conference, 2020.
[14]
S. Roy and M. S. Rahman, "Emergency vehicle detection on heavy traffic road from cctv footage using deep convolutional neural network," in Proc. of IEEE ECCE, 2019.
[15]
L. A. Varga, B. Kiefer, M. Messmer, and A. Zell, "Seadronessee: A maritime benchmark for detecting humans in open water," in Proc. of the IEEE/CVF Conference on Applications of Computer Vision, 2022.
[16]
H. Zhang, G. Ananthanarayanan, P. Bodik, M. Philipose, P. Bahl, and M. J. Freedman, "Live video analytics at scale with approximation and delay-tolerance," in Proc. of 14th { USENIX} NSDI, 2017.
[17]
J. Jiang, G. Ananthanarayanan, P. Bodik, S. Sen, and I. Stoica, "Chameleon: scalable adaptation of video analytics," in Proc. of the Conf. of the ACM Special Interest Group on Data Communication, 2018.
[18]
K. Wu, Y. Jin, W. Miao, Z. Zeng, Z. Qian, J. Wang, M. Zhou, and T. Cao, "Soudain: Online adaptive profile configuration for real-time video analytics," in IEEE/ACM 29th Int. Symp. on Quality of Service, 2021.
[19]
S. Naderiparizi, P. Zhang, M. Philipose, B. Priyantha, J. Liu, and D. Ganesan, "Glimpse: A programmable early-discard camera architecture for continuous mobile vision," in Proc. of the 15th Annual Int. Conf. on Mobile Systems, Applications, and Services. ACM, 2017.
[20]
C. Zhang, Q. Cao, H.Jiang, W. Zhang, J. Li, and J. Yao, "A fast filtering mechanism to improve efficiency of large-scale video analytics," IEEE Transactions on Computers, vol. 69, no. 6, pp. 914--928, 2020.
[21]
J. Wang, Z. Feng, Z. Chen, S. George, M. Bala, P. Pillai, S.-W. Yang, and M. Satyanarayanan, "Bandwidth-efficient live video analytics for drones via edge computing," in 2018 IEEE/ACM Symposium on Edge Computing (SEC), 2018.
[22]
B. Haynes, A. Mazumdar, M. Balazinska, L. Ceze, and A. Cheung, "Visual road: A video data management benchmark," in SIGMOD, 2019.
[23]
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, "CARLA: An open urban driving simulator," in Conf. on robot learning, 2017.
[24]
ZeroMQ, "Zeromq," https://zeromq.org/, 2022, [Online; accessed 2022-01-14].
[25]
Cap'n Proto, "Cap'n proto: Serialization protocol," https://capnproto.org/, 2022, [Online; accessed 2022-01-14].
[26]
M. Tan, R. Pang, and Q. V. Le, "Efficientdet: Scalable and efficient object detection," in Proc. of the IEEE/CVF conference on computer vision and pattern recognition, 2020.
[27]
PyPi, "Dominant color detection," https://pypi.org/project/dominant-color-detection/, 2020, [Online; accessed 2022-01-31].

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
DEBS '24: Proceedings of the 18th ACM International Conference on Distributed and Event-based Systems
June 2024
239 pages
ISBN:9798400704437
DOI:10.1145/3629104
This work is licensed under a Creative Commons Attribution-NoDerivatives International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 July 2024

Check for updates

Author Tags

  1. Load Shedding
  2. QoS
  3. Video Analytics
  4. latency bound

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

DEBS '24

Acceptance Rates

DEBS '24 Paper Acceptance Rate 15 of 30 submissions, 50%;
Overall Acceptance Rate 145 of 583 submissions, 25%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 124
    Total Downloads
  • Downloads (Last 12 months)124
  • Downloads (Last 6 weeks)34
Reflects downloads up to 30 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media