Abstract
Object detection and tracking are vital for video analysis. As the development of Deep Neural Network (DNN), multiple object tracking is recently performed on the detection results from DNN. However, DNN-based detection is computation-intensive. In order to accelerate multiple object detection and tracking for real-time application, we present a framework to import the tracking knowledge into detection to allow a less accurate but faster DNN for detection and recover the accuracy loss. By combining different DNNs with accuracy-speed trade-offs using space-aware color information, our framework achieves significant speedup (6.8\(\times \)) and maintains high accuracy. Targeting NVIDIA Xavier, we further optimize the implementation from system and platform level.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abdelali, H.A., et al.: Fast and robust object tracking via accept-reject color histogram-based method. J. Vis. Commun. Image Represent. 34, 219–229 (2016)
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. J. Image Video Process. 2008, 1 (2008)
Bewley, A., et al.: Alextrac: affinity learning by exploring temporal reinforcement within association chains. In: ICRA, pp. 2212–2218. IEEE (2016)
Bewley, A., et al.: Simple online and realtime tracking. In: ICIP, pp. 3464–3468. IEEE (2016)
Bochinski, E., et al.: High-speed tracking-by-detection without using image information. In: AVSS, pp. 1–6. IEEE (2017)
Bochinski, E., et al.: Extending IOU based multi-object tracking by visual information. In: AVSS, pp. 1–6. IEEE (2018)
Danelljan, M., et al.: Adaptive color attributes for real-time visual tracking. In: CVPR, pp. 1090–1097 (2014)
Dollár, P., et al.: Fast feature pyramids for object detection. TPAMI 36(8), 1532–1545 (2014)
Hamid Rezatofighi, S., et al.: Joint probabilistic data association revisited. In: ICCV, pp. 3047–3055 (2015)
Han, S., et al.: Learning both weights and connections for efficient neural network. In: Advances in Neural Information Processing Systems, pp. 1135–1143 (2015)
He, K., et al.: Mask R-CNN. In: ICCV, pp. 2961–2969 (2017)
Hubara, I., et al.: Quantized neural networks: training neural networks with low precision weights and activations. J. Mach. Learn. Res. 18(1), 6869–6898 (2017)
Kim, C., et al.: Multiple hypothesis tracking revisited. In: ICCV, pp. 4696–4704 (2015)
Leal-Taixé, L., et al.: Motchallenge 2015: towards a benchmark for multi-target tracking. arXiv:1504.01942 (2015)
Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Possegger, H., et al.: In defense of color-based model-free tracking. In: CVPR, pp. 2113–2120 (2015)
Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv:1804.02767 (2018)
Ren, S., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
Van De Weijer, J., et al.: Learning color names for real-world applications. TIP 18(7), 1512–1523 (2009)
Wojke, N., et al.: Simple online and realtime tracking with a deep association metric. In: ICIP, pp. 3645–3649. IEEE (2017)
Womg, A., et al.: Tiny SSD: a tiny single-shot detection deep convolutional neural network for real-time embedded object detection. In: CRV, pp. 95–101. IEEE (2018)
Xiang, Y., et al.: Learning to track: Online multi-object tracking by decision making. In: ICCV, pp. 4705–4713 (2015)
Yang, B., Nevatia, R.: An online learned CRF model for multi-target tracking. In: CVPR, pp. 2034–2041. IEEE (2012)
Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., Yan, J.: POI: multiple object tracking with high performance detection and appearance feature. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 36–42. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_3
Zhang, L., et al.: Global data association for multi-object tracking using network flows. In: CVPR, pp. 1–8. IEEE (2008)
Zhu, G., et al.: MC-HOG correlation tracking with saliency proposal. In: 30th AAAI (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Feng, L., Igarashi, H., Shibata, S., Kobayashi, Y., Takenaka, T., Zhang, W. (2020). Real-Time Detection and Tracking Using Hybrid DNNs and Space-Aware Color Feature: From Algorithm to System. In: Palaiahnakote, S., Sanniti di Baja, G., Wang, L., Yan, W. (eds) Pattern Recognition. ACPR 2019. Lecture Notes in Computer Science(), vol 12046. Springer, Cham. https://doi.org/10.1007/978-3-030-41404-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-41404-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-41403-0
Online ISBN: 978-3-030-41404-7
eBook Packages: Computer ScienceComputer Science (R0)