skip to main content
10.1145/3592473.3592566acmconferencesArticle/Chapter ViewAbstractPublication PagesmmsysConference Proceedingsconference-collections
research-article

A Low Cost Cross-Platform Video/Image Process Framework Empowers Heterogeneous Edge Application

Published: 07 June 2023 Publication History

Abstract

Recently, video/image intelligent analytics has been widely used in industrial Artificial Intelligence (AI) applications, such as defect detection, face recognition, and security monitoring. To provide better applicability and compatibility in such applications, the embedded AI models must be developed, compiled, and deployed under different development frameworks, such as cuDNN, RKNN, etc. Unfortunately, these frameworks are supported by various Graphic Processing Unit (GPU) hardware vendors, resulting in different model parameter structures and increased development costs. To address these issues, we propose LiGo, a low cost cross-platform video/image process framework, that simplifies and accelerates video intelligent processing in practical heterogeneous hardware systems. LiGo1 provides video processing pipeline, cross-platform development environments, and unified model serving structures. We demonstrate LiGo's efficiency and flexibility in model generation and deployment through its use in supporting multiple real-world commercial industrial systems.

References

[1]
Apple. 2023. Metal. https://developer.apple.com/metal/. [Online; accessed 10-March-2023].
[2]
JEAN-LUC AUFRANC. 2018. More Details about Rockchip RK3399Pro SoC, and RK1808 NPU. https://www.cnx-software.com/2018/09/24/rockchip-rk3399pro-details-rk1808-npu/. [Online; accessed 20-Jan-2022].
[3]
Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. 2018. TVM: An automated end-to-end optimizing compiler for deep learning. In Proc. OSDI. 578--594.
[4]
Robert David, Jared Duke, Advait Jain, Vijay Janapa Reddi, Nat Jeffries, Jian Li, Nick Kreeger, Ian Nappier, Meghna Natraj, Tiezhen Wang, et al. 2021. Tensorflow lite micro: Embedded machine learning for tinyml systems. Proceedings of Machine Learning and Systems 3 (2021), 800--811.
[5]
Haihan Duan, Junhua Liao, Lehao Lin, and Wei Cai. 2022. FLAD: a human-centered video content flaw detection system for meeting recordings. In Proceedings of the 32nd Workshop on Network and Operating Systems Support for Digital Audio and Video. 43--49.
[6]
Linux Foundation. 2023. PyTorch. https://pytorch.org/. [Online; accessed 10-March-2023].
[7]
Linux Foundation. 2023. PyTorch Mobile. https://pytorch.org/mobile/home/. [Online; accessed 10-March-2023].
[8]
Google. 2021. Tensorflow Serving. https://github.com/tensorflow/serving/. [Online; accessed 15-June-2021].
[9]
Google. 2023. Tensorflow Lite. https://www.tensorflow.org/lite/. [Online; accessed 10-March-2023].
[10]
Google. 2023. TPU. https://cloud.google.com/tpu/docs/tpus/. [Online; accessed 10-March-2023].
[11]
GStreamer. 2023. GStreamer. https://gstreamer.freedesktop.org/. [Online; accessed 10-March-2023].
[12]
Huawei. 2023. Atlas. https://e.huawei.com/en/products/computing/ascend/. [Online; accessed 10-March-2023].
[13]
Fortune Business Insights. 2022. Video Analytics Market Research Report. https://www.fortunebusinessinsights.com/industry-reports/video-analytics-market-101114. [Online; accessed 20-Jan-2023].
[14]
Intel. 2021. EII Documentation - Architecture. https://open-edge-insights.github.io/pages/architecture.html/. [Online; accessed 20-Jan-2022].
[15]
Khronos. 2023. Vulkan. https://www.vulkan.org/. [Online; accessed 10-March-2023].
[16]
KVM. 2023. KVM. https://www.linux-kvm.org/page/Main_Page/. [Online; accessed 10-March-2023].
[17]
NVIDIA. 2023. VideoProcessFramework. https://github.com/NVIDIA/VideoProcessingFramework/. [Online; accessed 10-March-2023].
[18]
QEMU. 2023. QEMU. https://www.qemu.org/. [Online; accessed 10-March-2023].
[19]
Rockchip. 2023. RKNN. https://github.com/rockchip-linux/rknn-toolkit/. [Online; accessed 10-March-2023].
[20]
Francisco Romero, Qian Li, Neeraja J Yadwadkar, and Christos Kozyrakis. 2021. INFaaS: Automated Model-less Inference Serving. In USENIX Annual Technical Conference. 397--411.
[21]
Francisco Romero, Mark Zhao, Neeraja J Yadwadkar, and Christos Kozyrakis. 2021. Llama: A heterogeneous & serverless framework for auto-tuning video analytics pipelines. In Proceedings of the ACM Symposium on Cloud Computing. 1--17.
[22]
Xen. 2023. Xen. https://xenproject.org/. [Online; accessed 10-March-2023].

Cited By

View all
  • (2024)Parallel Computing with GPU: An accelerator for Data-Centric High Performance Computing2024 1st International Conference on Innovative Engineering Sciences and Technological Research (ICIESTR)10.1109/ICIESTR60916.2024.10798202(1-6)Online publication date: 14-May-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
NOSSDAV '23: Proceedings of the 33rd Workshop on Network and Operating System Support for Digital Audio and Video
June 2023
77 pages
ISBN:9798400701849
DOI:10.1145/3592473
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. video process
  2. cross-platform
  3. framework
  4. edge computing

Qualifiers

  • Research-article

Conference

NOSSDAV '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 118 of 363 submissions, 33%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)45
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Parallel Computing with GPU: An accelerator for Data-Centric High Performance Computing2024 1st International Conference on Innovative Engineering Sciences and Technological Research (ICIESTR)10.1109/ICIESTR60916.2024.10798202(1-6)Online publication date: 14-May-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media