Skip to main content

Advertisement

SpringerLink
Log in

Multi-cue multi-hypothesis tracking with re-identification for multi-object tracking

  • Regular Article
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Multi-object tracking is an important research topic in the field of computer vision. In multi-object tracking, overlapping targets and dramatic changes in object appearance are major challenging problems. In this paper, we propose a multi-cue and multi-hypothesis tracking algorithm for multi-object tracking, which is based on re-identification (Re-ID) to alleviate the Id-Switch problem in the case of occlusion. Our approach has two major advantages: (1) Using metric learning with person Re-ID technology, our approach is able to reduce the distance within similar classes and increase the distance between classes. The appearance model we learn is more distinguishable than the appearance model obtained by the classification network trained on ImageNet. Our method can reduce the addition of different targets to hypothetical trajectory tree operations, thereby reducing hypothetical branches. (2) We build a re-identification search library. When the target is lost, we can find the trajectory of the target from the Re-ID search library and add it to the hypothesis trees. Therefore, our approach can be used to alleviate the problem of occlusion or target loss in multi-object tracking. To improve tracking accuracy further, we use the online training discriminative model kernel correlation filtering (KCF) to verify whether the branch added can be assigned to the current trajectory. Experiments show that our method outperforms other state-of-the-art methods on MOT challenge15, MOT challenge16.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Zhao, X., Kim, T.K.: Multiple object tracking: A literature review. arXiv preprint arXiv:1409.7618 (2014)

  2. Reid, D.: An algorithm for tracking multiple targets. IEEE Trans. Autom. Control 24(6), 843–854 (1979)

    Article  Google Scholar 

  3. Kim, C., Li, F., Ciptadi, A., Rehg, J.M.: Multiple hypothesis tracking revisited. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4696–4704 (2015)

  4. Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M.: Video segmentation by tracking many figure-ground segments. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2192–2199 (2013)

  5. Ciaparrone, G., Sánchez, F.L., Tabik, S., Troiano, L., Tagliaferri, R., Herrera, F.: Deep learning in video multi-object tracking: A survey. Neurocomputing 381, 61–88 (2020)

    Article  Google Scholar 

  6. Leal-Taixe, L., Pons-Moll, G., Rosenhahn, B.: Branch-and-price global optimization for multi-view multi-target tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 1987–1994 (2012)

  7. Emami, P., Pardalos, P.M., Elefteriadou, L., Ranka, S.: Machine learning methods for solving assignment problems in multi-target tracking. arXiv preprint arXiv:1802.06897 (2018)

  8. Shafique, K., Shah, M.: A noniterative greedy algorithm for multiframe point correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 51–65 (2005)

    Article  Google Scholar 

  9. Shu, G., Dehghan, A., Oreifej, O., Hand, E., Shah, M.: Part-based multiple-person tracking with partial occlusion handling. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 1815–1821 (2012)

  10. Fang, K., Xiang, Y., Li, X., Savarese, S.: Recurrent autoregressive networks for online multi-object tracking. In: IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp. 466–475 (2018)

  11. Sadeghian, A., Alahi, A., Savarese, S.: Tracking the untrackable: Learning to track multiple cues with long-term dependencies. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 300–311 (2017)

  12. Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. IEEE Int. Conf. Image Process. 2017, 3645–3649 (2017)

    Google Scholar 

  13. Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., Yu, N.: Online multi-object tracking using cnn-based single object tracker with spatial-temporal attention mechanism. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4836–4845 (2017)

  14. Milan, A., Rezatofighi, S.H., Dick, A., Reid, I., Schindler, K.: Online multi-target tracking using recurrent neural networks. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)

  15. Son, J., Baek, M., Cho, M., Han, B.: Multi-object tracking with quadruplet convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5620–5629 (2017)

  16. Sun, S., Akhtar, N., Song, H., Mian, A.S., Shah, M.: Deep affinity network for multiple object tracking. IEEE Trans. Pattern Anal. Mach. Intell 43, 104–119 (2019)

    Google Scholar 

  17. Xu, Y., Ban, Y., Alameda-Pineda, X., Horaud, R.: Deepmot: A differentiable framework for training multiple object trackers. arXiv preprint arXiv:1906.06618 (2019)

  18. Zamir, A.R., Dehghan, A., Shah, M.: Gmcp-tracker: Global multi-object tracking using generalized minimum clique graphs. In: European conference on computer vision, pp. 343–356. Springer, Berlin (2012)

    Google Scholar 

  19. Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors. Int. J. Comput. Vision 75(2), 247–266 (2007)

    Article  Google Scholar 

  20. Dehghan, A., Modiri Assari, S., Shah, M.: Gmmcp tracker: Globally optimal generalized maximum multi clique problem for multiple object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4091–4099 (2015)

  21. Berclaz, J., Fleuret, F., Turetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1806–1819 (2011)

    Article  Google Scholar 

  22. Robicquet, A., Sadeghian, A., Alahi, A., Savarese, S.: Learning social etiquette: Human trajectory understanding in crowded scenes. In: European conference on computer vision, pp. 549–565. Springer, Berlin (2016)

    Google Scholar 

  23. Leal-Taixé, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: Siamese cnn for robust target association. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 33–40 (2016)

  24. Wang, B., Wang, L., Shuai, B., Zuo, Z., Liu, T., Luk Chan, K., Wang, G.: Joint learning of convolutional neural networks and temporally constrained metrics for tracklet association. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8 (2016)

  25. Ma, C., Yang, C., Yang, F., Zhuang, Y., Zhang, Z., Jia, H., Xie, X.: Trajectory factory: Tracklet cleaving and re-connection by deep siamese bi-gru for multiple object tracking. In: IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp. 1–6 (2018)

  26. Tang, S., Andres, B., Andriluka, M., Schiele, B.: Subgraph decomposition for multi-target tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (2015) 5033–5041

  27. Tang, S., Andres, B., Andriluka, M., Schiele, B.: Multi-person tracking by multicut and deep matching. In: European conference on computer vision, pp. 100–111. Springer, Berlin (2016)

    Google Scholar 

  28. Schulter, S., Vernaza, P., Choi, W., Chandraker, M.: Deep network flow for multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6951–6960 (2017)

  29. Philipp Bergmann, Tim Meinhardt, L.L.T.: Tracking without bells and whistles. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 941–951 (2019)

  30. Yihong Xu, A. sep, Y.B.: How to train your deep multi-object tracker. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6786–6795 (2020)

  31. Collins, R.T.: Multitarget data association with higher-order motion models. In: IEEE conference on computer vision and pattern recognition, IEEE, pp. 1744–1751 (2012)

  32. Kim, C., Li, F., Rehg, J.M.: Multi-object tracking with neural gating using bilinear lstm. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 200–215 (2018)

  33. Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Informationp-theoretic metric learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 209–216 (2007)

  34. Wojke, N., Bewley, A.: Deep cosine metric learning for person re-identification. In: IEEE winter conference on applications of computer vision (WACV), IEEE, pp. 748–756 (2018)

  35. Blackman, S., Blackman, S.S., Popoli, R.: Design and analysis of modern tracking systems. (1999)

  36. Cox, I.J., Hingorani, S.L.: An efficient implementation of reid’s multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking. IEEE Trans. Pattern Anal. Mach. Intell. 18(2), 138–150 (1996)

    Article  Google Scholar 

  37. Papageorgiou, D.J., Salpukas, M.R.: The maximum weight independent set problem for data association in multiple hypothesis tracking. In: Optimization and cooperative control strategies, pp. 235–255. Springer, Berlin arXiv preprint arXiv:1603.00831(2009)

    Chapter  Google Scholar 

  38. Busygin, S.: A new trust region technique for the maximum weight clique problem. Discret. Appl. Math. 154(15), 2080–2096 (2006)

    Article  MathSciNet  Google Scholar 

  39. Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: Mot16: A benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831 (2016)

  40. Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. EURASIP J. Image Video Process. 2008, 1–10 (2008)

    Article  Google Scholar 

  41. Yang, B., Huang, C., Nevatia, R.: Learning affinities and dependencies for multi-target tracking using a crf model. In: Proceedings of the IEEE International Conference on Computer Vision, IEEE, pp. 1233–1240 (2011)

  42. Bergmann, P., Meinhardt, T., Leal-Taixe, L.: Tracking without bells and whistles. arXiv preprint arxiv190305625v3 (2019)

  43. Chu, P., Fan, H., Tan, C.C., Ling, H.: Online multi-object tracking with instance-aware tracker and dynamic model refreshment. In: IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp. 161–170 (2019)

  44. Tang, S., Andriluka, M., Andres, B., Schiele, B.: Multiple people tracking by lifted multicut and person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3539–3548 (2017)

  45. Sheng, H., Chen, J., Zhang, Y., Ke, W., Xiong, Z., Yu, J.: Iterative multiple hypothesis tracking with tracklet-level association. IEEE Trans. Circuits Syst. Video Technol. 29(12), 3660–3672 (2018)

    Article  Google Scholar 

  46. Xu, J., Cao, Y., Zhang, Z., Hu, H.: Spatial-temporal relation networks for multi-object tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3988–3998 (2019)

  47. Chen, L., Ai, H., Shang, C., Zhuang, Z., Bai, B.: Online multi-object tracking with convolutional neural networks. In: IEEE International Conference on Image Processing (ICIP), IEEE, pp. 645–649 (2017)

  48. Choi, W.: Near-online multi-target tracking with aggregated local flow descriptor. In: Proceedings of the IEEE international conference on computer vision, pp. 3029–3037 (2015)

  49. Zhou, H., Ouyang, W., Cheng, J., Wang, X., Li, H.: Deep continuous conditional random fields with asymmetric inter-object constraints for online multi-object tracking. IEEE Trans. Circuits Syst. Video Technol. 29(4), 1011–1022 (2018)

    Article  Google Scholar 

  50. Yang, M., Jia, Y.: Temporal dynamic appearance modeling for online multi-person tracking. Comput. Vis. Image Underst. 153, 16–28 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

The author would like to thank the anonymous reviewers for their helpful comments on an earlier draft of this paper. The work was supported in part by the National Natural Science Foundation of China under Grant 62072286 and Grant 61572296.

Author information

Authors and Affiliations

  1. School of Information and Electrical Engineering, Shandong Technology and Business University, Yantai, 264005, China

    Wen Guo, Yuelong Jin, Bin Shan, Xinmiao Ding & Minghao Wang

Authors
  1. Wen Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuelong Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinmiao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Minghao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen Guo.

Additional information

Communicated by B-K. Bao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, W., Jin, Y., Shan, B. et al. Multi-cue multi-hypothesis tracking with re-identification for multi-object tracking. Multimedia Systems 28, 925–937 (2022). https://doi.org/10.1007/s00530-022-00895-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00895-w

Keywords

Search

Navigation