Skip to main content
SpringerLink
Log in

Multi-target tracking by non-linear motion patterns based on hierarchical network flows

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

We propose a multi-target tracking method for pedestrians, through locating them and maintaining their identities to make their individual trajectories. In most existing data association-based tracking approaches, the motion information in the affinity model is always linear, but the motion of target is non-linear in real-world surveillance video, especially when there is occlusion. This paper proposes a global optimization method based on hierarchical network flows with a non-linear motion model to track multiple targets. In our method, each node in network represents a tracklet, and each edge represents the likelihood of neighboring tracklets belonging to the same trajectory measured by affinity score that we proposed, it differs from other network flow formulation whose nodes represent detection responses. A non-linear motion map is used for explaining non-linear motion pattern between neighboring tracklets and getting motion affinity when there are time gaps. We evaluate our approach on 2DMOT2015, MOT16 database, and several real surveillance videos. Experimental results demonstrate that the proposed method can achieve continuous tracking trajectories under the case of motion direction changes and complete occlusions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Tian, S., Yuan, F., Xia, G.S.: Multi-object tracking with inter-feedback between detection and tracking. Neurocomputing 171, 768–780 (2016)

    Article  Google Scholar 

  2. Chen, X., Qin, Z., An, L., et al.: Multiperson tracking by online learned grouping model with nonlinear motion context. IEEE Trans. Circuits Syst. Video Technol. 26(12), 2226–2239 (2016)

    Article  Google Scholar 

  3. Qin, Z., Shelton, C.R.: Improving multi-target tracking via social grouping. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1972–1978. IEEE, New York (2012)

  4. Shu, G., Dehghan, A., Oreifej, O., et al.: Part-based multiple-person tracking with partial occlusion handling. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1815–1821. IEEE, New York (2012)

  5. Tang, S., Andres, B., Andriluka, M., et al.: Subgraph decomposition for multi-target tracking. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5033–5041. IEEE, New York (2015)

  6. Tang, S., Andres, B., Andriluka, M., et al.: Multi-person tracking by multicut and deep matching. In: European Conference on Computer Vision, pp. 100–111. Springer, Cham (2016)

  7. Lenz, P., Geiger, A., Urtasun, R.: FollowMe: efficient online min-cost flow tracking with bounded memory and computation. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 4364–4372. IEEE, Santiago, Chile (2015)

  8. Le, N., Heili, A., Odobez, J.M.: Long-term time-sensitive costs for crf-based tracking by detection. In: European Conference on Computer Vision, pp. 43–51. Springer, Cham (2016)

  9. Brendel, W., Amer, M., Todorovic, S.: Multiobject tracking as maximum weight independent set. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1273–1280. IEEE, New York (2011)

  10. Yilmaz, A., Javed, O., Shah, M.: Object tracking: a survey. ACM Comput. Surv. 38(4), 13 (2006)

    Article  Google Scholar 

  11. Li, X., Hu, W., Shen, C., et al.: A survey of appearance models in visual object tracking. ACM Trans. Intell. Syst. Technol. 4(4), 58 (2013)

    Article  Google Scholar 

  12. Topkaya, I.S., Erdogan, H., Porikli, F.: Tracklet clustering for robust multiple object tracking using distance dependent Chinese restaurant processes. Signal Image Video Process. 10(5), 795–802 (2016)

    Article  Google Scholar 

  13. Ying, L., Zhang, T., Xu, C., et al.: Multi-object tracking via MHT with multiple information fusion in surveillance video. Multimedia Syst. 21(3), 313–326 (2015)

    Article  Google Scholar 

  14. Milan, A., Schindler, K., Roth, S.: Multi-target tracking by discrete-continuous energy minimization. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2054–2068 (2016)

    Article  Google Scholar 

  15. Yu, F., Li, W., Li, Q., et al.: POI: multiple object tracking with high performance detection and appearance feature. In: 2016 European Conference on Computer Vision (ECCV) 2016 Workshops, Part II, vol. 9914, pp. 36–42. Springer, Cham (2016)

  16. Dehghan, A., Modiri Assari, S., Shah, M.: GMMCP tracker: globally optimal generalized maximum multi clique problem for multiple object tracking. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4091–4099. IEEE, Boston, MA, USA (2015)

  17. He, Z., Cui, Y., Wang, H., et al.: One global optimization method in network flow model for multiple object tracking. Knowl. Based Syst. 86, 21–32 (2015)

    Article  Google Scholar 

  18. Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1201–1208. IEEE, New York (2011)

  19. Chari, V., Lacoste-Julien, S., Laptev, I., et al.: On pairwise costs for network flow multi-object tracking. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5537–5545. IEEE, Boston, MA, USA (2015)

  20. Wang, B., Wang, G., Chan, K.L., et al.: Tracklet association by online target-specific metric learning and coherent dynamics estimation. IEEE Trans. Pattern Anal. Mach. Intell. 39(3), 589–602 (2017)

    Article  Google Scholar 

  21. McLaughlin, N., Del Rincon, J.M., Miller, P.: Enhancing linear programming with motion modeling for multi-target tracking. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 71–77. IEEE, New York (2015)

  22. Breitenstein, M.D., Reichlin, F., Leibe, B., et al.: Robust tracking-by-detection using a detector confidence particle filter. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1515–1522. IEEE, New York (2009)

  23. Milan, A., Roth, S., Schindler, K.: Continuous energy minimization for multitarget tracking. IEEE Trans. Pattern Anal. Mach. Intell. 36(1), 58–72 (2014)

    Article  Google Scholar 

  24. Xing, J., Ai, H., Lao, S.: Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 1200–1207. IEEE, New York (2009)

  25. Kuo, C.H., Nevatia, R.: How does person identity recognition help multi-person tracking? In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1217–1224. IEEE, New York (2011)

  26. Yang, B., Nevatia, R.: Multi-target tracking by online learning of non-linear motion patterns and robust appearance models. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1918–1925. IEEE, New York (2012)

  27. Wang, B., Wang, L., Shuai, B., et al.: Joint learning of convolutional neural networks and temporally constrained metrics for tracklet association. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8. IEEE, Las Vegas, USA (2016)

  28. Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008. CVPR 2008, pp. 1–8. IEEE, New York (2008)

  29. Wang, B., Wang, G., Luk Chan, K., et al.: Tracklet association with online target-specific metric learning. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1234–1241. IEEE, New York, USA (2014)

  30. Farenzena, M., Bazzani, L., Perina, A., et al.: Person re-identification by symmetry-driven accumulation of local features. In: Computer Vision & Pattern Recognition. IEEE, New York (2010)

  31. Zhao, R., Ouyang, W., Wang, X.: Person re-identification by saliency learning. IEEE Trans. Pattern Anal. Mach. Intell. 39(2), 356–370 (2017)

    Article  Google Scholar 

  32. Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1201–1208. IEEE, Colorado Springs, USA (2011)

  33. Qin, Z., Shelton, C.R.: Improving multi-target tracking via social grouping. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1972–1978. IEEE, Providence, USA (2012)

  34. Wu, B., Nevatia, R.: Tracking of multiple, partially occluded humans based on static body part detection. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), pp. 951–958. IEEE, New York, USA (2006)

  35. Dollar, P., Appel, R., Belongie, S., et al.: Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1532–1545 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Internet of Things Engineering, Jiangnan University, Wuxi, 214122, China

    Weigang Lu, Zhiping Zhou, Lijuan Zhang & Guoqiang Zheng

  2. Engineering Research Center of Internet of Things Technology Applications Ministry of Education, Jiangnan University, Wuxi, 214122, China

    Zhiping Zhou

Authors
  1. Weigang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiping Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lijuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guoqiang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiping Zhou.

Additional information

Communicated by I. Ide.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, W., Zhou, Z., Zhang, L. et al. Multi-target tracking by non-linear motion patterns based on hierarchical network flows. Multimedia Systems 25, 383–394 (2019). https://doi.org/10.1007/s00530-019-00614-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-019-00614-y

Keywords

Search

Navigation