Abstract
Object detection is one of the key tasks in many applications of computer vision. Deep Neural Networks (DNNs) are undoubtedly a well-suited approach for object detection. However, such DNNs need highly adapted hardware together with hardware-specific optimization to guarantee high efficiency during inference. This is especially the case when aiming for efficient object detection in video streaming applications on limited hardware such as edge devices. Comparing vendor-specific hardware and related optimization software pipelines in a fair experimental setup is a challenge. In this paper, we propose a framework that uses a host computer with a host software application together with a light-weight interface based on the Message Queuing Telemetry Transport (MQTT) protocol. Various different target devices with target apps can be connected via MQTT with this host computer. With well-defined and standardized MQTT messages, object detection results can be reported to the host computer, where the results are evaluated without harming or influencing the processing on the device. With this quite generic framework, we can measure the object detection performance, the runtime, and the energy efficiency at the same time. The effectiveness of this framework is demonstrated in multiple experiments that offer deep insights into the optimization of DNNs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Blalock, D., Gonzalez Ortiz, J., Frankle, J., Guttag, J.: What is the state of neural network pruning? arXiv preprint arXiv:2004.10934 (2020)
Blott, M., et al.: Evaluation of optimized CNNs on heterogeneous accelerators using a novel benchmarking approach. IEEE Trans. Comput. 70(10), 1654–1669 (2021)
Bochkovskiy, A., Wang, C., Liao, H.: YOLOv4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)
Bradski, G.: The OpenCV library. Dobb’s J. Softw. Tools (2000)
Cai, Z., He, X., Sun, J., Vasconcelos, N.: Deep learning with low precision by half-wave gaussian quantization. In: IEEE CVPR (2017)
david8862: keras-yolov3-model-set. https://github.com/david8862/keras-YOLOv3-model-set/tree/v1.3.0
Eclipse: Eclipse paho mqtt c++ client library. https://github.com/eclipse/paho.mqtt.cpp
Gemirter, C., Senturca, C., Baydere, S.: A comparative evaluation of AMQP, MQTT and HTTP protocols using real-time public smart city data. In: 6th International Conference on Computer Science and Engineering (UBMK) (2021)
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M., Keutzer, K.: A survey of quantization methods for efficient neural network inference. arXiv preprint arXiv:2103.13630 (2021)
Gog, I., Kalra, S., Schafhalter, P., Wright, M., Gonzalez, J., Stoica, I.: Pylot: a modular platform for exploring latency-accuracy tradeoffs in autonomous vehicles. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2021)
Gündogan, C., Kietzmann, P., Lenders, M., Petersen, H., Schmidt, T., Wählisch, M.: NDN, CoAP, and MQTT: a comparative measurement study in the IoT. In: Proceedings of the 5th ACM Conference on Information-Centric Networking (ICN) (2018)
Hamerski, J.C., Domingues, A.R., Moraes, F.G., Amory, A.: Evaluating serialization for a publish-subscribe based middleware for mpsocs. In: 2018 25th IEEE International Conference on Electronics, Circuits and Systems (ICECS) (2018). https://doi.org/10.1109/ICECS.2018.8618003
Huang, J., et al.: Speed/accuracy trade-offs for modern convolutional object detectors. In: IEEE CVPR (2017)
Intel: OpenVINO Repository. https://github.com/openvinotoolkit/openvino
Jung, S., Hwang, S., Shin, H., Shim, D.: Perception, guidance, and navigation for indoor autonomous drone racing using deep learning. IEEE Rob. Autom. Lett. 3(3), 2539–2544 (2018)
Lin, T., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: IEEE ICCV (2017)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Lin, Z., Yih, M., Ota, J., Owens, J., Muyan-Özcelik, P.: Benchmarking deep learning frameworks and investigating FPGA deployment for traffic sign classification and detection. IEEE Trans. Intell. Veh. 4(3), 385–395 (2019)
Liu, D., Kong, H., Luo, X., Liu, W., Subramaniam, R.: Bringing AI to edge: from deep learning’s perspective. Neurocomputing 485, 297–320 (2022)
Mishra, B., Kertesz, A.: The use of MQTT in M2M and IoT systems: a survey. IEEE Access 8, 201071–201086 (2021)
NVIDIA: TensorRT Repository. https://github.com/NVIDIA/TensorRT/
Nvidia: Jetson agx xavier developer kit - user guide (2019). https://developer.download.nvidia.com/embedded/L4T/r32-3-1_Release_v1.0/jetson_agx_xavier_developer_kit_user_guide.pdf
ONNX: Tensorflow backend for onnx. https://github.com/onnx/onnx-tensorflow/
Redmon, J.: Darknet: Open source neural networks in c (2013–2016). http://pjreddie.com/darknet/
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: IEEE CVPR (2016)
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. CoRR abs/1804.02767 (2018). http://arxiv.org/abs/1804.02767
Rungsuptaweekoon, K., Visoottiviseth, V., Takano, R.: Evaluating the power efficiency of deep learning inference on embedded gpu systems. In: International Conference on Information Technology (INCIT) (2017)
Stäcker, L., et al.: Deployment of deep neural networks for object detection on edge ai devices with runtime optimization. In: IEEE International Conference on Computer Vision Workshops (ICCVW) (2021)
Verucchi, M., et al.: A Systematic assessment of embedded neural networks for object detection. In: 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), vol. 1, pp. 937–944 (2020). https://doi.org/10.1109/ETFA46521.2020.9212130
Wang, J., Gu, S.: FPGA implementation of object detection accelerator based on Vitis-AI. In: 2021 11th International Conference on Information Science and Technology (ICIST), pp. 571–577 (2021)
Xilinx: Vitis AI Repository. https://github.com/Xilinx/Vitis-AI/
Xilinx: Zcu104 board user guide (2018). https://www.xilinx.com/support/documentation/boards_and_kits/zcu104/ug1267-zcu104-eval-bd.pdf
Xilinx: Dpuczdx8g for zynq ultrascale+ mpsocs (2021). https://www.xilinx.com/content/dam/xilinx/support/documentation/ip_documentation/dpu/v3_3/pg338-dpu.pdf
Xiong, Y., et al.: MobileDets: searching for object detection architectures for mobile accelerators. In: IEEE CVPR (2021)
Yokotani, T., Sasaki, Y.: Comparison with HTTP and MQTT on required network resources for IoT. In: International Conference on Control, Electronics, Renewable Energy and Communications (ICCEREC) (2016)
Yu, J., et al.: Real-time object detection towards high power efficiency. In: Design, Automation & Test in Europe Conference & Exhibition (DATE) (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Schlosser, M., König, D., Teutsch, M. (2022). A Framework for Benchmarking Real-Time Embedded Object Detection. In: Andres, B., Bernard, F., Cremers, D., Frintrop, S., Goldlücke, B., Ihrke, I. (eds) Pattern Recognition. DAGM GCPR 2022. Lecture Notes in Computer Science, vol 13485. Springer, Cham. https://doi.org/10.1007/978-3-031-16788-1_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-16788-1_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16787-4
Online ISBN: 978-3-031-16788-1
eBook Packages: Computer ScienceComputer Science (R0)