Skip to main content
SpringerLink
Log in

Multi-object tracking based on network flow model and ORB feature

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

We present a multi-object tracking algorithm based on network flow model and ORB feature for solving occlusion problem. In our work, we extract the ORB features from the detection response and match them to achieve data association at the intra-tracklet stage. Then, tracklets are associated by network flow model to realize data association at the inter-tracklet stage. Each trajectory fragment is regarded as a node in the network flow model. Owing to the different occlusion situations, different network flow cost functions are proposed, by integrating the motion information obtained by the Edge Multi-channel Gradient Model, the appearance information of the tracklet and the time information. Experimental results demonstrate that compared with other state-of-art algorithms, our method improves tracking performance in complex environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Li F, Tian C, Zuo W et al (2018) Learning spatial-temporal regularized correlation filters for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4904–4913

  2. Liang Y, Lu X, He Z et al (2019) Multiple object tracking by reliable tracklets. Signal, Image and Video Processing 13(3):823–831

    Article  Google Scholar 

  3. Li ZX, Liu JM, Li S et al (2015) Group targets tracking algorithm based on box particle filter. Acta Automatica Sinica 41(4):785–798

    MathSciNet  Google Scholar 

  4. Dewi DEO, Fadzil MM, Faudzi AAM et al (2015) Position tracking systems for ultrasound imaging: A survey. Medical imaging technology, Springer, Singapore

  5. Yu F, Li W, Li Q et al (2016) Poi, Multiple object tracking with high performance detection and appearance feature. In: European conference on computer vision, Springer, Cham, pp 36-42

  6. Bewley A, Ge Z, Ott L, et al (2016) Simple online and realtime tracking. In: 2016 IEEE International conference on image processing (ICIP), IEEE, pp 3464–3468

  7. Felzenszwalb PF, Girshick RB, McAllester D (2009) Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9):16271645

    Google Scholar 

  8. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards realtime object detection with region proposal networks. Advances in Neural Information Processing Systems 9199

  9. Ren J, Chen X, Liu J, Sun W, Pang J, Yan Q, Tai Y-W, Xu L (2017) Accurate single stage detector using recurrent rolling convolution, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 54205428

  10. He S, Shin H-S, Tsourdos A (2020) Trajectory optimization for multitarget tracking using joint probabilistic data association filter. Journal of Guidance, Control, and Dynamics 43(1):70–178

    Article  Google Scholar 

  11. Karunasekera H, Wang H, Zhang H (2019) Multiple object tracking with attention to appearance, structure, motion and size. IEEE Access 7:104423–104434

    Article  Google Scholar 

  12. Lee S, Kim E (2018) Multiple object tracking via feature pyramid siamese networks. IEEE Access 7:8181–8194

    Article  Google Scholar 

  13. Wang H, Nguang SK (2016) Multi-target video tracking based on improved data association and mixed kalman filtering. IEEE Sensors Journal 16(21):7693–7704

    Article  Google Scholar 

  14. Rezatofighi SH et al (2015) Joint probabilistic data association revisited. In: Proceedings of the IEEE international conference on computer vision

  15. Lili SUN et al (2020) A multi-target tracking algorithm based on Gaussian mixture model. Journal of Systems Engineering and Electronics 99:1–6

    Google Scholar 

  16. Zhu J, Yang H, Liu N et al (2018) Online multi-object tracking with dual matching attention networks. In: Computer visionECCV 2018, Springer, Berlin, pp 366382

  17. Chu P, Ling H (2019) Famnet: joint learning of feature, affinity and multi-dimensional assignment for online multiple object tracking. In: Proceedings of the IEEE international conference on computer vision, Seoul, South Korea, pp 61726181

  18. Sun SJ et al (2019) Deep affinity network for multiple object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 43(1):104–119

    Google Scholar 

  19. A. Milan, S. Roth, and K. Schindler. Continuous energy minimization for multitarget tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(1)5872, 2014

  20. Zuo G, Du T, Ma L (2018) Dynamic target tracking based on corner enhancement with Markov decision process. J Eng 1617–1622

  21. Shen J, Liang L, Liu J et al (2019) Multiobject tracking by submodular optimization. IEEE Transactions on Cybernetics 49(6):19902001

    Article  Google Scholar 

  22. Luo W et al (2020) Multiple object tracking: A literature review. Artif Intell 103448

  23. Weng X, Kitani K (2019) A baseline for 3d multi-object tracking. arXiv:1907.03961

  24. Feng W et al (2019) Multi-object tracking with multiple cues and switcher-aware classification. arXiv:1901.06129

  25. Schulter S et al (2017) Deep network flow for multi-object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  26. Berclaz J, Fleuret F, Turetken E, Fua P (2011) Multiple object tracking using k-shortest paths optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(9):18061819

    Article  Google Scholar 

  27. Xi Z et al (2015) Multiple object tracking using A* association algorithm with dynamic weights. Journal of Intelligent and Fuzzy Systems 29(5):2059–2072

    Article  Google Scholar 

  28. Xi Z et al (2015) Dynamic shortest path association for multiple object tracking in video sequence. Journal of Electronic Imaging 24(1):013009

    Article  Google Scholar 

  29. Butt AA, Collins RT (2013) Multi-target tracking by lagrangian relaxation to min-cost network flow. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1846–1853

  30. Zhang L, Li Y, Nevatia R (2008) Global data association for multi-object tracking using network flows. In: Proceeding of the 26th IEEE conference on computer vision and pattern recognition, pp 342349

  31. Berclaz J, Fleuret F, Turetken E et al (2011) Multiple object tracking using k-shortest paths optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(9):1806–1819

    Article  Google Scholar 

  32. Pirsiavash H, Ramanan D, Fowlkes CC (2011) Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR 2011, IEEE, pp 1201–1208

  33. Wang C, Wang Y, Wang Y, Wu C-T, Yu G (2019) muSSP: Efficient min-cost flow algorithm for multi-object tracking. NeurIPS

  34. Xiang J et al (2020) End-to-End learning deep CRF models for multi-object tracking deep crf models. IEEE Transactions on Circuits and Systems for Video Technology 31(1):275–288

    Article  Google Scholar 

  35. Huang C, Li Y, Nevatia R (2013) Multiple target tracking by learning-based hierarchical association of detection responses. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(4):898–910

    Article  Google Scholar 

  36. Zhou H et al (2018) Deep continuous conditional random fields with asymmetric inter-object constraints for online multi-object tracking. IEEE Transactions on Circuits and Systems for Video Technology 29(4):1011–1022

    Article  Google Scholar 

  37. Yang B, Nevatia R (2014) Multi-target tracking by online learning a crf model of appearance and motion patterns. International Journal of Computer Vision 107(2):203–217

    Article  MathSciNet  Google Scholar 

  38. Luo W, Xing J, Milan A et al (2020) Multiple object tracking: A literature review. Artif Intell 103448

  39. Feng W, Hu Z, Wu W et al (2019) Multi-object tracking with multiple cues and switcher-aware classification. arXiv:1901.06129

  40. Wang C, Wang Y, Yu G (2019) Efficient global multi-object tracking under minimum-cost circulation framework. arXiv:1911.00796

  41. Lenz P, Geiger A, Urtasun R (2015) Followme: Efficient online min-cost flow tracking with bounded memory and computation. In: Proceedings of the IEEE international conference on computer vision

  42. Shitrit HB et al (2011) Tracking multiple people under global appearance constraints. In: 2011 International conference on computer vision, IEEE

  43. Rublee E, Rabaud V, Konolige K et al (2011) ORB: An efficient alternative to SIFT or SURF. In: 2011 International conference on computer vision, IEEE, pp 2564–2571

  44. Ciaparrone G, Snchez FL, Tabik S et al (2019) Deep learning in video multi-object tracking: a survey

  45. Stadler D, Beyerer J (2021) Improving multiple pedestrian tracking by track management and occlusion handling. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10958–10967

  46. Andriyenko A, Schindler K (2011) Multi-target tracking by continuous energy minimization. In: CVPR vol 2, no 6

  47. Andriyenko A, Schindler K, Roth S (2012) Discrete-continuous optimization for multi-target tracking. In: 2012 IEEE Conference on computer vision and pattern recognition, IEEE, pp 1926-1933

  48. Henriques JF, Caseiro R, Batista J (2011) Globally optimal solution to multi-object tracking with merged measurements. In: ICCV

  49. Hofmann M, Haag M, Rigoll G (2013) Unified hierarchical multi-object tracking using global data association. In: Performance evaluation of tracking and surveillance (PETS), 2013 IEEE international workshop on, IEEE

  50. Hofmann M, Wolf D, Rigoll G (2013) Hypergraphs for joint multi-view reconstruction and multi-object tracking. In: 2013 IEEE conference on computer vision and pattern recognition, IEEE

  51. Izadinia H, Saleemi I, Li W, et al (2012) (MP) 2 T: Multiple people multiple parts tracker. In: European conference on computer vision

  52. Milan A, Schindler K, Roth S (2013) Detection- and trajectory-level exclusion in multiple object tracking. In: Computer vision and pattern recognition, IEEE

  53. Zamir AR, Dehghan A, Shah M (2012) Gmcp-tracker: Global multi-object tracking using generalized minimum clique graphs. In: ECCV

  54. Breitenstein MD, Reichlin F, Leibe B, Koller-Meier E, Gool LV (2011) Online multi-person tracking-by-detection from a single, uncalibrated camera. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(9)18201833, 201133(9):18201833

    Google Scholar 

  55. Possegger H, Mauthner T, Roth PM et al (2014) Occlusion geodesics for online multi-object tracking. In: IEEE Conference on computer vision and pattern recognition (CVPR), IEEE

  56. Chen S, Fern A, Todorovic S (2014) Multi-object tracking via constrained sequential labeling. In: CVPR

  57. Yamaguchi K, Berg AC, Ortiz LE et al (2011) Who are you with and where are you going. In: Computer vision and pattern recognition, IEEE

  58. Leal-Taix L, Pons-Moll G , Rosenhahn B (2012) Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker. In: IEEE International conference on computer vision workshops, IEEE

  59. Benfold B, Reid I (2011) Stable multi-target tracking in real-time surveillance video. In: Computer vision pattern recognition, IEEE

  60. Pellegrini S, Ess A, Schindler K et al (2009) Gool.You’ll never walk alone: modeling social behavior for multi-target tracking. Proceedings, pp 261–268

  61. Nevatia R, Bo Y (2012) An online learned CRF model for multi-target tracking. In: Computer vision pattern recognition, IEEE Computer Society

  62. Kuo CH, Nevatia R (2011) How does person identity recognition help multi-person tracking? In CVPR

  63. Chen J, Xi Z (2020) Wei C (2020) Multiple object tracking using edge multi-channel gradient model with ORB feature. IEEE Access 9:2294:2309

    Google Scholar 

  64. Bae S, Yoon K (2018) Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking. In: IEEE Transactions on pattern analysis and machine intelligence, pp 595–610

  65. Dehghan A et al (2015) Target identity-aware network flow for online multiple target tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  66. Farahi F, Yazdi HS (2020) Probabilistic Kalman filter for moving object tracking. Signal Processing: Image Communication 82:115751

    Google Scholar 

  67. Cui Y et al (2019) Multiple pedestrian tracking by combining particle filter and network flow model. Neurocomputing 351:217–227

    Article  Google Scholar 

  68. Jiang Z, Huynh DQ (2017) Multiple pedestrian tracking from monocular videos in an interacting multiple model framework. IEEE Transactions on Image Processing 27(3):1361–1375

    Article  MathSciNet  Google Scholar 

  69. Byeon M et al (2019) Variational inference for 3-D localization and tracking of multiple targets using multiple cameras. IEEE Transactions on Neural Networks and Learning Systems 30(11):3260–3274

    Article  Google Scholar 

  70. Shi X et al (2019) Rank-1 tensor approximation for high-order association in multi-target tracking. International Journal of Computer Vision 127(8):1063–1083

    Article  MathSciNet  Google Scholar 

  71. Ju J et al (2017) Online multi-object tracking with efficient track drift and fragmentation handling. JOSA A 34(2):280–293

    Article  Google Scholar 

  72. Bae S-H (2020) Online multi-object tracking with visual and radar features. IEEE Access 8:90324–90339

    Article  Google Scholar 

  73. Hao T, Wang Q, Wu D et al (2018) Multiple person tracking based on slow feature analysis. Multimedia Tools and Applications 77(3):3623–3637

    Article  Google Scholar 

  74. Yang M, Pei M, Jia Y (2020) Online maximum a posteriori tracking of multiple objects using sequential trajectory prior. Image and Vision Computing 94:103867

    Article  Google Scholar 

  75. Zhang B et al (2018) Transfer learningbased online multiperson tracking with Gaussian process regression. Concurrency and Computation: Practice and Experience 30(23):4917

    Article  Google Scholar 

  76. Tian Y, Dehghan A, Shah M (2019) On detection, data association and segmentation for multi-target tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(9):2146–2160

    Article  Google Scholar 

  77. Liang-qun L et al (2018) Fuzzy logic approach to visual multi-object tracking. Neurocomputing 281:139–151

    Article  Google Scholar 

  78. Wen L, Lei Z, Lyu S et al (2015) Exploiting hierarchical dense structures on hypergraphs for multi-object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(10):1983–1996

    Article  Google Scholar 

  79. Yang H et al (2018) Multiple objects tracking with improved sparse representation and rank based dynamic estimation. IEEE Access 6:42264–42278

    Article  Google Scholar 

  80. Shen H, Huang L, Huang C et al (2018) Tracklet association tracker: An end-to-end learning-based association approach for multi-object tracking. arXiv:1808.01562

  81. Hornakova A, Henschel R, Rosenhahn B et al (2020) Lifted disjoint paths with application in multiple object tracking. In: International conference on machine learning, PMLR, pp 4364-4375

Download references

Author information

Authors and Affiliations

  1. School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, 201620, Shanghai, China

    Jieyu Chen, Zhenghao Xi, Junxin Lu & Jingjing Ji

Authors
  1. Jieyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenghao Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junxin Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jingjing Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jieyu Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, J., Xi, Z., Lu, J. et al. Multi-object tracking based on network flow model and ORB feature. Appl Intell 52, 12282–12300 (2022). https://doi.org/10.1007/s10489-021-03042-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03042-6

Keywords

Search

Navigation