Abstract
The majority of existing tracking algorithms are based on the maximum a posteriori solution of a probabilistic framework using a Hidden Markov Model, where the distribution of the object state at the current time instance is estimated based on current and previous observations. However, this approach is prone to errors caused by distractions such as occlusions, background clutters and multi-object confusions. In this paper, we propose a multiple object tracking algorithm that seeks the optimal state sequence that maximizes the joint multi-object state-observation probability. We call this algorithm trajectory tracking since it estimates the state sequence or “trajectory” instead of the current state. The algorithm is capable of tracking unknown time-varying number of multiple objects. We also introduce a novel observation model which is composed of the original image, the foreground mask given by background subtraction and the object detection map generated by an object detector. The image provides the object appearance information. The foreground mask enables the likelihood computation to consider the multi-object configuration in its entirety. The detection map consists of pixel-wise object detection scores, which drives the tracking algorithm to perform joint inference on both the number of objects and their configurations efficiently. The proposed algorithm has been implemented and tested extensively in a complete CCTV video surveillance system to monitor entries and detect tailgating and piggy-backing violations at access points for over six months. The system achieved 98.3% precision in event classification. The violation detection rate is 90.4% and the detection precision is 85.2%. The results clearly demonstrate the advantages of the proposed detection based trajectory tracking framework.
References
Avidan S. (2004). Support vector tracking. PAMI 26(8): 1064–1072
Avidan, S.: Ensemble tracking. CVPR05, p II 494–501 (2005)
Bar-Shalom, Y., Li, X.R.: Multitarget Multisensor Tracking: Principles and Techniques. YBS Publishing (1995)
Birchfield S. and Sriram R. (2005). Spatiograms versus histograms for region-based tracking. CVPR05 II: 1158–1163
Black M. and Jepson A. (1996). Eigentracking: robust matching and tracking of articulated objects using a view-based representation. ECCV96 II: 329–342
Collins, R.: Mean-shift blob tracking through scale space. CVPR03 , 234–240 (2003)
Collins R., Liu Y. and Leordeanu M. (2005). On-line selection of discriminative tracking features. PAMI 27(10): 1631–1643
Comaniciu D., Ramesh V. and Meer P. (2003). Kernel-based object tracking. PAMI 25(5): 564–577
Cootes T., Edwards G. and Taylor C. (2001). Active appearance models. PAMI 23(6): 681–685
Cox I.J. and Hingorani S.L. (1996). An efficient implementation of reid’s multiple hypotheses tracking algorithm and its evaluation for the purpose of visual tracking. PAMI 18(2): 138–150
De la Torre F. and Black M. (2003). A framework for robust subspace learning. IJCV 54(1–3): 117–142
Fan Z. and Wu Y. (2005). Multiple collaborative kernel tracking. CVPR05 II: 502–509
Fortmann T.E., Bar-Shalom Y. and Scheffe M. (1983). Sonar tracking of multiple targets using joint probabilistic data association. IEEE J. Oceanic Eng. OE-8: 173–184
Gauvrit H. and Le Cadre J.P. (1997). A formulation of multitarget tracking as an incomplete data problem. IEEE Trans. Aerosp. Electron. Systems. 33(4): 1242–1257
Hager G. and Belhumeur P. (1998). Efficient region tracking with parametric models of geometry and illumination.. PAMI 20(10): 1025–1039
Hager G., Dewan M. and Stewart C. (2004). Multiple kernel tracking with ssd. CVPR 04(I): 790–797
Han B. and Davis L. (2005). On-line density-based appearance modeling for object tracking. ICCV05 II: 1492–1499
Haritaoglu, I., Harwood, D., Davis, L.S.: W4s: a real-time system for detecting and tracking people in 2 1/2-d. ECCV98 (1998)
Haritaoglu, I., Harwood, D., Davis, L.S.: Hydra: multiple people detection and tracking using silhouettes. In: IEEE Workshop on Visual Surveillance (1999)
Ho J., Lee K., Yang M. and Kriegman D. (2004). Visual tracking using learned subspaces. CVPR04 I: 782–789
Hue C., Le Cadre J.P. and Perez P. (2002). Tracking multiple objects with particle filtering. IEEE Trans. Aerosp Electron. Systems 38(3): 791–812
Isard M. and Blake A. (1998). Condensation–conditional density propagation for visual tracking. IJCV 29(1): 5–28
Isard M. and MacCormick J.P. (2001). Bramble: a bayesian multiple-blob tracker. ICCV01 II: 34–41
Jepson A.D., Fleet D.J. and El-Maraghi T.F. (2001). Robust online appearance models for visual tracking. CVPR01 I: 415–422
Jojic N., Petrovic N., Frey B.J. and Huang T.S. (2000). Transformed hidden markov models: estimating mixture models of images and inferring spatial transformations in video sequences. CVPR00 II: 26–33
Kass M., Witkin A. and Terzopoulos D. (1988). Snakes: active contour models. IJCV 1(4): 321–331
Kirubarajan, Y., Bar-Shalom, Y., Pattipati, K.R.: Multiassignment for tracking a large number of overlapping objects. IEEE Trans. Aerosp. Electron. Systems 37(1), (2001)
Lanz, O., Manduchi, R.: Hybrid joint-separable multibody tracking. CVPR05 (2005)
LeCun Y., Bottou L., Bengio Y. and Haffner P. (1998). Gradient-based learning applied to document recognition. Proc. IEEE 86(11): 2278–2324
Lee K. and Kriegman D.J. (2005). Online learning of probabilistic appearance manifolds for video-based recognition and tracking. CVPR05 I: 852–859
MacCormick J.P. and Blake A. (1999). A probabilistic exclusion principle for tracking multiple objects. ICCV 99: 572–578
Matthews I., Ishikawa T. and Baker S. (2004). The template update problem. PAMI 26(6): 810–815
Rabiner L.R. (1989). A tutorial on hidden markov models and selected applications in speech recognition. Proc. IEEE 77: 257–286
Reid D.B. (1979). An algorithm for tracking multiple targets. IEEE Trans. Autom. Control 24(6): 843–854
Rittscher J., Tu P.H. and Krahnstoever N. (2005). Simultaneous estimation of segmentation and shape. CVPR05 II: 486–493
Shi J. and Tomasi C. (1994). Good features to track. CVPR 94: 593–600
Stauffer C. and Grimson W.E.L. (2000). Learning patterns of activity using real-time tracking. PAMI 22(8): 747–757
Streit, R.L., Luginbuhl, T.E.: Maximum likelihood method for probabilistic multi-hypothesis tracking. In: Proceedings of SPIE International Symposium, Signal and Data Processing of Small Targets (1994)
Tang, F.,Tao,H.: Object trackingwith dynamic feature graphs.VSPETS05, 25–32 (2005)
Tao, H., Sawhney, H.S., Kumar, R.: A sampling algorithm for tracking multiple objects. In: International Workshop on Vision Algorithms (1999)
Tao H., Sawhney H.S. and Kumar R. (2002). Object tracking with bayesian estimation of dynamic layer representations. PAMI 24(1): 75–89
Verma R.C., Schmid C. and Mikolajczyk K. (2003). Face detection and tracking in a video by propagating detection probabilities. PAMI 25(10): 1215–1228
Williams, O., Blake, A., Cipolla, R.: A sparse probabilistic learning algorithm for real-time tracking. ICCV03 , 353–360 (2003)
Yu T. and Wu Y. (2004). Collaborative tracking of multiple targets. CVPR04 I: 834–841
Zhao L. and Davis L.S. (2005). Closely coupled object detection and segmentation. ICCV05 I: 454–461
Zhao T., Nevatia R. and Lv F. (2001). Segmentation and tracking of multiple humans in complex situations. CVPR01 II: 194–201
Zhou, Y., Tao, H.: An background layer model for object tracking through occlusion. ICCV03 (2003)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Han, M., Xu, W., Tao, H. et al. Multi-object trajectory tracking. Machine Vision and Applications 18, 221–232 (2007). https://doi.org/10.1007/s00138-007-0071-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-007-0071-5