Skip to main content
Log in

Multi-human tracking using part-based appearance modelling and grouping-based tracklet association for visual surveillance applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Although much progress has been made in multi-object tracking in recent decades due to its variety of applications including visual surveillance, traffic monitoring and medical image analysis, some difficult challenges such as the variation of object appearance and partial occlusion are still going on. In this work, we propose an effective multi-human tracking system called part-based appearance modelling and grouping-based tracklet association-based multi-human tracking (PAM-GTA-MHT). The proposed appearance model based on the upper body-centered multi-view human body part model can effectively resolve the drawback caused by inter-object occlusions and low camera positions. The grouping method embedded in global tracklet association can improve discriminability among targets with similar appearances when they are located sufficiently far away from each other. Thus, there is no need to compare all possible pairs of the detected targets in the tracklet association stage and thus it has the potential to enhance the tracking speed. We quantitatively evaluated the performance of our proposed approach on four challenging publicly available datasets and achieved a significant improvement compared to the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Andriluka M, Roth S, Schiele B et al. (2008) People-tracking-by detection and people-detection-by-tracking. Proc IEEE Conf Comput Vision Pattern Recognit 1–8

  2. Andriyenko A, Roth S, Schindler K et al. (2011) An analytical formulation of global occlusion reasoning for multi-target tracking. 2011 I.E. Int Conf ICCV Workshops 1839–1846

  3. Andriyenko A, Schindler K (2010) Globally optimal multi-target tracking on a hexagonal lattices. Proc 11th Europ Conf Comput Vision 466–479

  4. Andriyenko A, Schindler K (2011) Multi-target tracking by continuous energy minimization. Proc IEEE Conf Comput Vision Pattern Recognit 1265–1272

  5. Avidan S (2007) Ensemble tracking. IEEE Trans Pattern Anal Mach Intell 29(2):261–271

    Article  Google Scholar 

  6. Berclaz J, Fleuret F, Fua P (2006) Robust people tracking with global trajectory optimization. Proc IEEE Conf Comput Vision Pattern Recognit 1:744–750

    Google Scholar 

  7. Birchfield S-T, Rangarajan S (2005) Spatiograms versus histograms for region-based tracking. Proc IEEE Conf Comput Vision Pattern Recognit 2:1158–1163

    Google Scholar 

  8. Breitenstein M-D, Reichlin F, Leibe B et al. (2009) Robust tracking-by-detection using a detector confidence particle filter. Proc IEEE Int Conf Comput Vision 1515–1522

  9. Brendel W, Amer M, Todorovic S ET A et al. (2011) Multiobject tracking as maximum weight independent set. Proc IEEE Conf Comput Vision Pattern Recognit 1273–1280

  10. Cai Y, de Freitas N, Little J-J (2006) Robust visual tracking for multiple targets. Pro 9th Europ Conf Comput Vision 3954:107–118

    Google Scholar 

  11. Cannons K-J, Gryn J-M, Wildes R-P (2010) Visual tracking using a pixelwise spatiotemporal oriented energy representation. Proc 11th Europ Conf Comput Vision 6314:511–524

    Google Scholar 

  12. CAVIAR Dataset, http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1

  13. Collins R-T, Liu Y (2005) On-line selection of discriminative tracking features. IEEE Trans Pattern Anal Mach Intell 27(10):1631–1643

    Article  Google Scholar 

  14. Dalal N; Triggs B (2005) Histograms of oriented gradients for human detection. Proc IEEE Conf Comput Vision Pattern Recognit 886–893

  15. Ess A, Leibe B, Schindler K et al. (2008) A mobile vision system for robust multi-person tracking. Proc IEEE Conf Comput Vision Pattern Recognit 1–8

  16. Farenzena M, Bazzani L, Perina1 A et al. (2010) Person re-identification by symmetry-driven accumulation of local features. Proc IEEE Conf Comput Vision Pattern Recognit 2360–2367

  17. Felzenszwalb P, Girshick R, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645

    Article  Google Scholar 

  18. Fulkerson B, Vedaldi A, Soatto S (2008) Localizing objects with smart dictionaries. Proc 9th Europ Conf Comput Vision 5302:179–192

    Google Scholar 

  19. Grabner H, Bischof H (2006) On-line boosting and vision. Proc IEEE Conf Comput Vision Pattern Recognit 1:260–267

    Google Scholar 

  20. Grabner H, Matas J, Gool L V et al. (2010) Tracking the invisible: learning where the object might be. Proc IEEE Conf Comput Vision Pattern Recognit 1285–1292

  21. Gray D, Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features. Proc 10th Europ Conf Comput Vision 262–275

  22. Heili A, Chen C, Odobez J-M et al. (2011) Detection-based multi-human tracking using a CRF model. Proc IEEE Int Conf Comput Vision Workshop 1673–1680

  23. Hou C, Ai H, Lao S (2007) Multiview pedestrian detection based on vector boosting. Proc 6th Asian Conf Comput Vision 4843:210–219

    Google Scholar 

  24. Huang C, Wu B, Nevatia R (2008) Robust object tracking by hierarchical association of detection responses. Proc 10th Europ Conf Comput Vision 5305:788–801

    Google Scholar 

  25. Jiang H, Fels S, Little J-J et al. (2007) A linear programming approach for multiple object tracking. Proc IEEE Conf Comput Vision Pattern Recognit 1–8

  26. Kim S, Kwak S, Feyereisl J, Han B (2012) Online multi-target tracking by large margin structured learning. Proc 11th Asian Conf Comput Vision 7726:98–111

    Google Scholar 

  27. Kuo C-H, Huang C, Nevatia R et al. (2010) Multi-target tracking by online learned discriminative appearance models. Proc IEEE Conf Comput Vision Pattern Recognit 685–692

  28. Kuo C-H., Nevatia R (2011) How does person identity recognition help multi-person tracking?. Proc IEEE Conf Comput Vision Pattern Recognit 1217–1224

  29. Leibe B, Leonardis A, Schiele B (2008) Robust object detection with interleaved categorization and segmentation. Int J Comput Vis 77:259–289

    Article  Google Scholar 

  30. Leibe B, Schindler K, Gool L-V et al. (2007) Coupled detection and trajectory estimation for multi-object tracking. Proc IEEE Int Conf Comput Vision 1–8

  31. Levi K, Weiss Y (2004) Learning object detection from a small number of examples: the importance of good features. Proc IEEE Conf Comput Vision Pattern Recognit 2:53–60

    Google Scholar 

  32. Li Y, Ai H, Yamashita T, Lao S, Kawade M (2008) Tracking in low frame rate video: a cascade particle filter with discriminative observers of different lifespans. IEEE Trans Pattern Anal Mach Intell 30(10):1728–1740

    Article  Google Scholar 

  33. Li Y, Huang C, Nevatia R et al. (2009) Learning to associate: hybridboosted multi-target tracker for crowded scene. Proc IEEE Conf Comput Vision Pattern Recognit 2953–2960

  34. Lowe D-G (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  35. Okuma K, Taleghani A, Freitas OD, Little J-J, Lowe D-G (2004) A boosted particle filter: Multitarget detection and tracking. Proc 8th Europ Conf Comput Vision 3021:28–39

    MATH  Google Scholar 

  36. Perera AGA, Srinivas C, Hoogs A, Brooksby G, Hu W (2006) Multi-object tracking through simultaneous long occlusions and split-merge conditions. Proc IEEE Conf Comput Vision Pattern Recognit 1:666–673

    Google Scholar 

  37. PETS 2009 dataset, http://ftp.pets.rdg.ac.uk/pub/PETS2009/Crowd_PETS09_dataset/a_data/a.html

  38. Pirsiavash H, Ramanan D, Fowlkes C et al. (2011) Globally-optimal greedy algorithms for tracking a variable number of objects. Proc IEEE Conf Comput Vision Pattern Recognit 1201–1208

  39. Song B, Jeng T-Y, Staudt E, Roy-Chowdhury A-K (2010) A stochastic graph evolution framework for robust multi-target tracking. Proc 11th Europ Conf Comput Vision 6311:05–619

    Google Scholar 

  40. Tsochantaridis I, Hofmann T, Joachims T et al. (2004) Support vector machine learning for interdependent and structured output space. Int Conf Mach Learn 104–112

  41. Tuzel O, Porikli F, Meer P et al. (2007) Human detection via classification on riemannian manifolds. Proc IEEE Conf Comput Vision Pattern Recognit 1–8

  42. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. ProcIEEE Conf ComputVision Pattern Recognit 1:511–518

    Google Scholar 

  43. Walk S, Majer N, Schindler K et al. (2010) New features and insights for pedestrian detection. Proc IEEE Conf Comput Vision Pattern Recognit 1030–1037

  44. Wang X, Han TX., Yan S et al. (2009) An hog-lbp human detector with partial occlusion handling. Proc IEEE Int Conf Comput Vision 32–29

  45. Wu B, Nevatia R (2005) Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors. Proc IEEE Conf Computer Vision Pattern Recognit 1:90–97

    Google Scholar 

  46. Wu B, Nevatia R (2007) Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. Int J Comput Vis 75(2):247–266

    Article  Google Scholar 

  47. Xing J, Ai H, Lao S et al. (2009) Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses. Proc IEEE Conf Comput Vision Pattern Recognit 1200–1207

  48. Yang B, Huang C, Nevatia R et al. (2011) Learning affinities and dependencies for multi-target tracking using a CRF model. Proc IEEE Conf Comput Vision Pattern Recognit 1233–1240

  49. Yang M, Lv F, Xu W et al. (2009) Detection driven adaptive multi-cue integration for multiple human tracking. Proc IEEE Int Conf Comput Vision 1554–1561

  50. Yang B, Nevatia R (2012) Multi-target tracking by online learning of non-linear motion patterns and robust appearance models. Proc IEEE Conf Comput Vision Pattern Recognit 1918–1925

  51. Yang B, Nevatia R (2012) An online learned CRF model for multi-target tracking. Proc IEEE Conf Comput Vision Pattern Recognit 2034–2041

  52. Yu Q, Medioni G (2009) Multiple-target tracking by spatiotemporal monte carlo markov chain data association. IEEE Trans Pattern Anal Mach Intell 31(12):2196–2210

    Article  Google Scholar 

  53. Zhang L, Li Y, Nevatia R et al. (2008) Global data association for multi-object tracking using network flows. Proc IEEE Conf Comput Vision Pattern Recognit 1–8

Download references

Acknowledgments

This work was supported by the ICT R&D Program of MSIP/IITP (Grant No. B0101-15-0525, Development of global multi-target tracking and event prediction techniques based on real-time large-scale video analysis), and Center for Integrated Smart Sensors as Global Frontier (CISS-2013M3A6A6073718).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Moongu Jeon.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, E., Gwak, J. & Jeon, M. Multi-human tracking using part-based appearance modelling and grouping-based tracklet association for visual surveillance applications. Multimed Tools Appl 76, 6731–6754 (2017). https://doi.org/10.1007/s11042-015-3219-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3219-8

Keywords

Navigation