Skip to main content
SpringerLink
Log in

Multi-object trajectory tracking

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The majority of existing tracking algorithms are based on the maximum a posteriori solution of a probabilistic framework using a Hidden Markov Model, where the distribution of the object state at the current time instance is estimated based on current and previous observations. However, this approach is prone to errors caused by distractions such as occlusions, background clutters and multi-object confusions. In this paper, we propose a multiple object tracking algorithm that seeks the optimal state sequence that maximizes the joint multi-object state-observation probability. We call this algorithm trajectory tracking since it estimates the state sequence or “trajectory” instead of the current state. The algorithm is capable of tracking unknown time-varying number of multiple objects. We also introduce a novel observation model which is composed of the original image, the foreground mask given by background subtraction and the object detection map generated by an object detector. The image provides the object appearance information. The foreground mask enables the likelihood computation to consider the multi-object configuration in its entirety. The detection map consists of pixel-wise object detection scores, which drives the tracking algorithm to perform joint inference on both the number of objects and their configurations efficiently. The proposed algorithm has been implemented and tested extensively in a complete CCTV video surveillance system to monitor entries and detect tailgating and piggy-backing violations at access points for over six months. The system achieved 98.3% precision in event classification. The violation detection rate is 90.4% and the detection precision is 85.2%. The results clearly demonstrate the advantages of the proposed detection based trajectory tracking framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Avidan S. (2004). Support vector tracking. PAMI 26(8): 1064–1072

    Google Scholar 

  2. Avidan, S.: Ensemble tracking. CVPR05, p II 494–501 (2005)

  3. Bar-Shalom, Y., Li, X.R.: Multitarget Multisensor Tracking: Principles and Techniques. YBS Publishing (1995)

  4. Birchfield S. and Sriram R. (2005). Spatiograms versus histograms for region-based tracking. CVPR05 II: 1158–1163

    Google Scholar 

  5. Black M. and Jepson A. (1996). Eigentracking: robust matching and tracking of articulated objects using a view-based representation. ECCV96 II: 329–342

    Google Scholar 

  6. Collins, R.: Mean-shift blob tracking through scale space. CVPR03 , 234–240 (2003)

  7. Collins R., Liu Y. and Leordeanu M. (2005). On-line selection of discriminative tracking features. PAMI 27(10): 1631–1643

    Google Scholar 

  8. Comaniciu D., Ramesh V. and Meer P. (2003). Kernel-based object tracking. PAMI 25(5): 564–577

    Google Scholar 

  9. Cootes T., Edwards G. and Taylor C. (2001). Active appearance models. PAMI 23(6): 681–685

    Google Scholar 

  10. Cox I.J. and Hingorani S.L. (1996). An efficient implementation of reid’s multiple hypotheses tracking algorithm and its evaluation for the purpose of visual tracking. PAMI 18(2): 138–150

    Google Scholar 

  11. De la Torre F. and Black M. (2003). A framework for robust subspace learning. IJCV 54(1–3): 117–142

    Article  MATH  Google Scholar 

  12. Fan Z. and Wu Y. (2005). Multiple collaborative kernel tracking. CVPR05 II: 502–509

    Google Scholar 

  13. Fortmann T.E., Bar-Shalom Y. and Scheffe M. (1983). Sonar tracking of multiple targets using joint probabilistic data association. IEEE J. Oceanic Eng. OE-8: 173–184

    Article  Google Scholar 

  14. Gauvrit H. and Le Cadre J.P. (1997). A formulation of multitarget tracking as an incomplete data problem. IEEE Trans. Aerosp. Electron. Systems. 33(4): 1242–1257

    Article  Google Scholar 

  15. Hager G. and Belhumeur P. (1998). Efficient region tracking with parametric models of geometry and illumination.. PAMI 20(10): 1025–1039

    Google Scholar 

  16. Hager G., Dewan M. and Stewart C. (2004). Multiple kernel tracking with ssd. CVPR 04(I): 790–797

    Google Scholar 

  17. Han B. and Davis L. (2005). On-line density-based appearance modeling for object tracking. ICCV05 II: 1492–1499

    Google Scholar 

  18. Haritaoglu, I., Harwood, D., Davis, L.S.: W4s: a real-time system for detecting and tracking people in 2 1/2-d. ECCV98 (1998)

  19. Haritaoglu, I., Harwood, D., Davis, L.S.: Hydra: multiple people detection and tracking using silhouettes. In: IEEE Workshop on Visual Surveillance (1999)

  20. Ho J., Lee K., Yang M. and Kriegman D. (2004). Visual tracking using learned subspaces. CVPR04 I: 782–789

    Google Scholar 

  21. Hue C., Le Cadre J.P. and Perez P. (2002). Tracking multiple objects with particle filtering. IEEE Trans. Aerosp Electron. Systems 38(3): 791–812

    Article  Google Scholar 

  22. Isard M. and Blake A. (1998). Condensation–conditional density propagation for visual tracking. IJCV 29(1): 5–28

    Article  Google Scholar 

  23. Isard M. and MacCormick J.P. (2001). Bramble: a bayesian multiple-blob tracker. ICCV01 II: 34–41

    Google Scholar 

  24. Jepson A.D., Fleet D.J. and El-Maraghi T.F. (2001). Robust online appearance models for visual tracking. CVPR01 I: 415–422

    Google Scholar 

  25. Jojic N., Petrovic N., Frey B.J. and Huang T.S. (2000). Transformed hidden markov models: estimating mixture models of images and inferring spatial transformations in video sequences. CVPR00 II: 26–33

    Google Scholar 

  26. Kass M., Witkin A. and Terzopoulos D. (1988). Snakes: active contour models. IJCV 1(4): 321–331

    Article  Google Scholar 

  27. Kirubarajan, Y., Bar-Shalom, Y., Pattipati, K.R.: Multiassignment for tracking a large number of overlapping objects. IEEE Trans. Aerosp. Electron. Systems 37(1), (2001)

  28. Lanz, O., Manduchi, R.: Hybrid joint-separable multibody tracking. CVPR05 (2005)

  29. LeCun Y., Bottou L., Bengio Y. and Haffner P. (1998). Gradient-based learning applied to document recognition. Proc. IEEE 86(11): 2278–2324

    Article  Google Scholar 

  30. Lee K. and Kriegman D.J. (2005). Online learning of probabilistic appearance manifolds for video-based recognition and tracking. CVPR05 I: 852–859

    Google Scholar 

  31. MacCormick J.P. and Blake A. (1999). A probabilistic exclusion principle for tracking multiple objects. ICCV 99: 572–578

    Google Scholar 

  32. Matthews I., Ishikawa T. and Baker S. (2004). The template update problem. PAMI 26(6): 810–815

    Google Scholar 

  33. Rabiner L.R. (1989). A tutorial on hidden markov models and selected applications in speech recognition. Proc. IEEE 77: 257–286

    Article  Google Scholar 

  34. Reid D.B. (1979). An algorithm for tracking multiple targets. IEEE Trans. Autom. Control 24(6): 843–854

    Article  Google Scholar 

  35. Rittscher J., Tu P.H. and Krahnstoever N. (2005). Simultaneous estimation of segmentation and shape. CVPR05 II: 486–493

    Google Scholar 

  36. Shi J. and Tomasi C. (1994). Good features to track. CVPR 94: 593–600

    Google Scholar 

  37. Stauffer C. and Grimson W.E.L. (2000). Learning patterns of activity using real-time tracking. PAMI 22(8): 747–757

    Google Scholar 

  38. Streit, R.L., Luginbuhl, T.E.: Maximum likelihood method for probabilistic multi-hypothesis tracking. In: Proceedings of SPIE International Symposium, Signal and Data Processing of Small Targets (1994)

  39. Tang, F.,Tao,H.: Object trackingwith dynamic feature graphs.VSPETS05, 25–32 (2005)

  40. Tao, H., Sawhney, H.S., Kumar, R.: A sampling algorithm for tracking multiple objects. In: International Workshop on Vision Algorithms (1999)

  41. Tao H., Sawhney H.S. and Kumar R. (2002). Object tracking with bayesian estimation of dynamic layer representations. PAMI 24(1): 75–89

    Google Scholar 

  42. Verma R.C., Schmid C. and Mikolajczyk K. (2003). Face detection and tracking in a video by propagating detection probabilities. PAMI 25(10): 1215–1228

    Google Scholar 

  43. Williams, O., Blake, A., Cipolla, R.: A sparse probabilistic learning algorithm for real-time tracking. ICCV03 , 353–360 (2003)

  44. Yu T. and Wu Y. (2004). Collaborative tracking of multiple targets. CVPR04 I: 834–841

    Google Scholar 

  45. Zhao L. and Davis L.S. (2005). Closely coupled object detection and segmentation. ICCV05 I: 454–461

    Google Scholar 

  46. Zhao T., Nevatia R. and Lv F. (2001). Segmentation and tracking of multiple humans in complex situations. CVPR01 II: 194–201

    Google Scholar 

  47. Zhou, Y., Tao, H.: An background layer model for object tracking through occlusion. ICCV03 (2003)

Download references

Author information

Authors and Affiliations

  1. NEC Laboratories America, Cupertino, CA, USA

    Mei Han, Wei Xu & Yihong Gong

  2. University of California, Santa Cruz, CA, USA

    Hai Tao

Authors
  1. Mei Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hai Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yihong Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hai Tao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Han, M., Xu, W., Tao, H. et al. Multi-object trajectory tracking. Machine Vision and Applications 18, 221–232 (2007). https://doi.org/10.1007/s00138-007-0071-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-007-0071-5

Keywords

Search

Navigation