Skip to main content
SpringerLink
Log in

Online Multiple Object Tracking Algorithm Based on Heat Map Propagation

  • Conference paper
  • First Online:
Algorithms and Architectures for Parallel Processing (ICA3PP 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 13155))

  • 1846 Accesses

Abstract

Online multiple object tracking is an important computer vision task with a wide range of application scenarios which integrates target detection, re-ID (Re-identification) and feature association matching. Most of the existing methods have two problems: ignoring the spatio-temporal information, resulting in poor tracking effect, and using two-stage detector makes the tracking speed slow. In this paper, we design an online multiple object tracking algorithm by generating a heat map based on the spatio-temporal information of the tracker, and propagate it to a one-stage detector to improve tracking speed and stabilize detection quality. Without introducing additional regressors, the generation of the propagation heat map are both simply and efficiently. Consequently, the proposed algorithm can achieve a balance between the speed and accuracy, and provides a paradigm for the utilization of the spatio-temporal characteristic information of the one-stage detector. Experiments are carried out on MOT-17 and 2DMOT-15 which verifies that 43.27% and 63.7% improvement in tracking speed is obtained with a small accuracy compromise.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hou, X., Wang. Y., Chau, L.: Vehicle tracking using deep sort with low confidence track filtering. In: 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, Taiwan, pp. 1–6 (2019). https://doi.org/10.1109/AVSS.2019.8909903

  2. Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv preprint arXiv:804.02767 (2018)

  3. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016).https://doi.org/10.1109/CVPR.2016.91

  4. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  5. Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3645–3649. IEEE (2017)

    Google Scholar 

  6. Feichtenhofer, C., Pinz, A., Zisserman, A.: Detect to track and track to detect. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3038–3046 (2017)

    Google Scholar 

  7. Zhou, X., Wang, D., Krahenb, P.: Objects as points. arXiv preprint arXiv:1904.07850 (2019)

  8. Wang, Z., Zheng, L., Liu, Y., Wang, S.: Towards real-time multiobject tracking. arXiv preprint arXiv:1909.12605 (2019)

  9. Zhou, X., Koltun, V., Krahenb, P.: Tracking objects as points. arXiv arXiv:2004.01177 (2020)

  10. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: YOLOv4: optimal speed and accuracy of object detection. arXiv arXiv:2004.10934 (2020)

  11. Zhang, Y., Wang, C., Wang, X., Zeng, W., Liu, W.: FairMOT: on the fairness of detection and re-identification in multiple object tracking. arXiv: Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  12. Yu, F., Wang, D., Shelhamer, E., Darrell, T.: Deep layer aggregation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2403–2412 (2018). https://doi.org/10.1109/CVPR.2018.00255

  13. Zhu, X., et al.: Deep feature flow for video recognition. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4141–4150 (2017)

    Google Scholar 

  14. Zhang, Z., Cheng, D., Zhu, X., Lin, S., Dai, J.: Integrated object detection and tracking with tracklet-conditioned detection. arXiv arXiv:1811.11167 (2018)

  15. Zhu, X., Dai, J., Zhu, X., Wei, Y., Yuan, L.: Towards high performance video object detection for mobiles. arXiv arXiv:1804.05830 (2018)

  16. Chen, Y., Cao, Y., Hu, H., Wang, L.: Memory enhanced global-local aggregation for video object detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10334–10343 (2020)

    Google Scholar 

  17. Xu, Z., Hrustic, E., Vivet, D.: CenterNet heatmap propagation for real-time video object detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12370, pp. 220–234. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58595-2_14

    Chapter  Google Scholar 

  18. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: ICIP, pp. 3464–3468. IEEE (2016)

    Google Scholar 

  19. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). https://doi.org/10.1109/TPAMI.2016.2577031

  20. Voigtlaender, P., et al.: MOTS: multi-object tracking and segmentation. In: CVPR, pp. 7942–7951 (2019)

    Google Scholar 

  21. Bochinski, E., Eiselein, V., Sikora, T.: High-speed tracking-by-detection without using image information. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6. IEEE (2017)

    Google Scholar 

  22. Ess, A., Leibe, B., Schindler, K., Van Gool, L.: A mobile vision system for robust multi-person tracking. In: CVPR. IEEE, pp. 1–8 (2008)

    Google Scholar 

  23. Zhang, S., Benenson, R., Schiele, B.: CityPersons: a diverse dataset for pedestrian detection. In: CVPR, pp. 3213–3221 (2017)

    Google Scholar 

  24. Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: a benchmark. In: CVPR, pp. 304–311. IEEE (2009)

    Google Scholar 

  25. Xiao, T., Li, S., Wang, B., Lin, L., Wang, X.: Joint detection and identification feature learning for person search. In: CVPR, pp. 3415–3424 (2017)

    Google Scholar 

  26. Zheng, L., Zhang, H., Sun, S., Chandraker, M., Yang, Y., Tian, Q.: Person re-identification in the wild. In: CVPR, pp. 1367–1376 (2017)

    Google Scholar 

  27. Milan, A., Leal-Taixe, L., Reid, I., Roth, S., Schindler, K.: Mot16: a benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)

  28. Shao, S., et al.: CrowdHuman: a benchmark for detecting human in a crowd. arXiv preprint arXiv:1805.00123 (2018)

  29. Xiang, Y., Alahi, A., Savarese, S.: Learning to track: online multiobject tracking by decision making. In: ICCV, pp. 4705–4713 (2015)

    Google Scholar 

  30. Bae, S.-H., Yoon, K.-J.: Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1218–1225 (2014)

    Google Scholar 

  31. Dicle, C., Camps, O.I., Sznaier, M.: The way they move: tracking multiple targets with similar appearance. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2304–2311 (2013)

    Google Scholar 

  32. Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)

    Google Scholar 

  33. Berclaz, J., Fleuret, F., Turetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1806–1819 (2011)

    Article  Google Scholar 

  34. Zamir, A.R., Dehghan, A., Shah, M.: GMCP-tracker: global multi-object tracking using generalized minimum clique graphs. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 343–356. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33709-3_25

    Chapter  Google Scholar 

  35. Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 58–72 (2013)

    Article  Google Scholar 

  36. Wen, L., Li, W., Yan, J., Lei, Z., Yi, D., Li, S.Z.: Multiple target tracking based on undirected hierarchical relation hypergraph. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1282–1289 (2014)

    Google Scholar 

  37. Choi, W.: Near-online multi-target tracking with aggregated local flow descriptor. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3029–3037 (2015)

    Google Scholar 

  38. Mahmoudi, N., Ahadi, S.M., Rahmati, M.: Multi-target tracking using CNN-based features: CNNMTT. Multimedia Tools Appl. 78(6), 7077–7096 (2019)

    Article  Google Scholar 

  39. Zhou, Z., Xing, J., Zhang, M., Hu, W.: Online multi-target tracking with tensor-based high-order graph matching. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 1809–1814. IEEE (2018)

    Google Scholar 

  40. Fang, K., Xiang, Y., Li, X., Savarese, S.: Recurrent autoregressive networks for online multi-object tracking. In: WACV, pp. 466–475. IEEE (2018)

    Google Scholar 

  41. Jiang, M., Huang, W., Huang, Z., Yen, G.G.: Integration of global and local metrics for domain adaptation learning via dimensionality reduction. IEEE Trans. Cybern. 47(1), 38–51 (2017)

    Article  Google Scholar 

  42. Jiang, M., Huang, Z., Qiu, L., Huang, W., Yen, G.: Transfer learning based dynamic multi-objective optimization algorithms. IEEE Trans. Evol. Comput. 99, 1–14 (2017)

    Google Scholar 

  43. Jiang, M., Qiu, L., Huang, Z., Yen, G.G.: Dynamic multi-objective estimation of distribution algorithm based on domain adaptation and nonparametric estimation. Inf. Sci. 435, 203–223 (2018)

    Article  MathSciNet  Google Scholar 

  44. Jiang, M., Wang, Z., Qiu, L., Guo, S., Gao, X., Tan, K.C.: A fast dynamic evolutionary multiobjective algorithm via manifold transfer learning. IEEE Trans. Cybern. 51(7), 3417–3428 (2021)

    Article  Google Scholar 

  45. Jiang, M., Wang, Z., Hong, H., Yen, G.G.: Knee point based imbalanced transfer learning for dynamic multi-objective optimization. IEEE Trans. Evol. Comput. 25, 117–129 (2020)

    Article  Google Scholar 

  46. Jiang, M., Wang, Z., Guo, S., Gao, X., Tan, K.C.: Individual-based transfer learning for dynamic multi-objective optimization. IEEE Trans. Cyben. 51, 4968–4981 (2020)

    Article  Google Scholar 

  47. Xu, D., Jiang, M., Hu, W., Li, S., Pan, R., Yen, G.G.: An online prediction approach based on incremental support vector machine for dynamic multi-objective optimization. IEEE Trans. Evol. Comput., 1 (2021). https://doi.org/10.1109/TEVC.2021.3115036

  48. Wang, Z., Hong, H., Ye, K., Zhang, G.-E., Jiang, M., Tan, K.C.: Manifold interpolation for large-scale multi-objective optimization via generative adversarial networks. IEEE Trans. Neural Netw. Learn. Syst., 1–15 (2021). https://doi.org/10.1109/TNNLS.2021.3113158

Download references

Acknowledgement

This work was supported in part by the National Natural Science Foundation of China under Grant 61673328, and in part by the Collaborative Project Foundation of Fuzhou-Xiamen-Quanzhou Innovation Demonstration Zone under Grant 3502ZCQXT202001.

Author information

Authors and Affiliations

  1. School of Informatics, Xiamen University, Fujian, 361005, China

    Wei Ding, Haokai Hong, Dejun Xu & Min Jiang

  2. Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism, Fujian, China

    Wei Ding, Haokai Hong, Dejun Xu & Min Jiang

Authors
  1. Wei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haokai Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dejun Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Min Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Jiang .

Editor information

Editors and Affiliations

  1. Xiamen University, Xiamen, China

    Yongxuan Lai

  2. Beijing Normal University, Zhuhai, China

    Tian Wang

  3. Xiamen University, Xiamen, China

    Min Jiang

  4. Tianjin University, Tianjin, China

    Guangquan Xu

  5. Hunan University, Changsha, China

    Wei Liang

  6. University of Naples Parthenope, Naples, Italy

    Aniello Castiglione

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ding, W., Hong, H., Xu, D., Jiang, M. (2022). Online Multiple Object Tracking Algorithm Based on Heat Map Propagation. In: Lai, Y., Wang, T., Jiang, M., Xu, G., Liang, W., Castiglione, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2021. Lecture Notes in Computer Science(), vol 13155. Springer, Cham. https://doi.org/10.1007/978-3-030-95384-3_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-95384-3_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-95383-6

  • Online ISBN: 978-3-030-95384-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation