Skip to main content
SpringerLink
Log in

Visual tracking based on hierarchical framework and sparse representation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As the main challenge for object tracking is to account for drastic appearance change, a hierarchical framework that exploits the strength of both generative and discriminative models is devised in this paper. Our hierarchical framework consists of three appearance models: local-histogram-based model, weighted alignment pooling model, and sparsity-based discriminative model. Sparse representation is adopted in local-histogram-based model layer that considers the spatial information among local patches with a dual-threshold update schema to deal with occlusion. The weighted alignment pooling layer is introduced to weight the local image patches of the candidates after sparse representation. Different from the above two generative methods, the global discriminant model layer employs candidates to sparsely represent positive and negative templates. After that, an effective hierarchical fusion strategy is developed to fuse the three models via their similarities and the confidence. In addition, three reasonable online dictionary and template update strategies are proposed. Finally, experiments on various current popular image sequences demonstrate that our proposed tracker performs favorably against several state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Arulampalam MS, Maskell S, Gordon N, Clapp T (2002) A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans Signal Process 50:174–188. https://doi.org/10.1109/78.978374

    Article  Google Scholar 

  2. Babenko B, Belongie S (2009) Visual tracking with online multiple instance learning. IEEE Conf Comput Vis Pattern Recognit 33(8):983–990. https://doi.org/10.1109/CVPRW.2009.5206737

  3. Bao C, Wu Y, Ling H, Ji H (2012) Real time robust L1 tracker using accelerated proximal gradient approach. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 1830–1837. https://doi.org/10.1109/CVPR.2012.6247881

  4. Chang WY, Chen CS, Hung YP (2009) Tracking by parts: A Bayesian approach with component collaboration. IEEE Trans Syst Man, Cybern Part B Cybern 39:375–388. https://doi.org/10.1109/TSMCB.2008.2005417

    Article  Google Scholar 

  5. Chen D, Yuan Z, Hua G, et al (2014) Description-discrimination collaborative tracking. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 345–360. https://doi.org/10.1007/978-3-319-10590-1_23

  6. Chi Z, Li H, Lu H, Yang MH (2017) Dual Deep Network for Visual Tracking. IEEE Trans Image Process 26:2005–2015. https://doi.org/10.1109/TIP.2017.2669880

    Article  MathSciNet  Google Scholar 

  7. Cuevas E, Zaldivar D, Rojas R (2005) Kalman filter for vision tracking. Measurement 1–18. 

  8. Danelljan M, Khan FS, Felsberg M, Van De Weijer J (2014) Adaptive color attributes for real-time visual tracking. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 1090–1097. https://doi.org/10.1109/CVPR.2014.143

  9. Dinh TB, Medioni G (2011) Co-training framework of generative and discriminative trackers with partial occlusion handling. 2011 I.E. Work Appl Comput Vision, WACV 2011 642–649. https://doi.org/10.1109/WACV.2011.5711565

  10. Godec M, Roth PM, Bischof H (2013) Hough-based tracking of non-rigid objects. Comput Vis Image Underst 117:1245–1256. https://doi.org/10.1016/j.cviu.2012.11.005

    Article  Google Scholar 

  11. Grabner H, Leistner C, Bischof H (2008) Semi-supervised boosting on-line boosting for robust tracking. Proc Eur Conf Comput Vis 234–247. https://doi.org/10.1007/978-3-540-88682-2_19

  12. Hager GD, Dewan M, Stewart C V (2004) Multiple kernel tracking with SSD. Comput Vis Pattern Recognition, 2004 CVPR 2004 Proc 2004 I.E. Comput Soc Conf 1:I–790–I–797 Vol. 1. https://doi.org/10.1109/cvpr.2004.1315112

  13. Han S, Fu R, Wang S, Wu X (2013) Online adaptive dictionary learning and weighted sparse coding for abnormality detection. IEEE Int Conf Image Process 151–155. https://doi.org/10.1109/ICIP.2013.6738032

  14. Hong Z, Chen Z, Wang C, et al (2015) MUlti-Store Tracker (MUSTer): a cognitive psychology inspired approach to object tracking. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 749–758. https://doi.org/10.1109/CVPR.2015.7298675

  15. Hu W, Li W, Zhang X, Maybank S (2015) Single and Multiple Object Tracking Using a Multi-Feature Joint Sparse Representation. Tpami 37:816–833. https://doi.org/10.1109/TPAMI.2014.2353628

    Article  Google Scholar 

  16. Huber PJ (1964) Robust Estimation of a Location Parameter. Ann Math Stat 35:73–101. https://doi.org/10.1214/aoms/1177703732

    Article  MathSciNet  MATH  Google Scholar 

  17. Jia X, Lu H (2012) Visual tracking via adaptive structural local sparse appearance model. Computer Vision and Pattern Recognition (CVPR), 2012 I.E. Conference on, vol. 157(10). IEEE, Providence, RI, pp 1822-1829. https://doi.org/10.1109/CVPR.2012.6247880

  18. Kalal Z, Matas J, Mikolajczyk K (2010) P-N learning: Bootstrapping binary classifiers by structural constraints. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 49–56. https://doi.org/10.1109/CVPR.2010.5540231

  19. Kolsch M, Turk M (2004) Fast 2D Hand Tracking with Flocks of Features and Multi-Cue Integration. Conf Comput Vis Pattern Recognit Work 10:158–158. https://doi.org/10.1109/CVPR.2004.345

  20. Kristan M, Perš J, Kovačič S, Leonardis A (2009) A local motion-based probabilistic model for visual tracking. Pattern Recogn 42:2160–2168. https://doi.org/10.1016/j.patcog.2009.01.002

    Article  MATH  Google Scholar 

  21. Kwon J, Lee KM (2009) Tracking of a non-rigid object via patch-based dynamic appearance modeling and adaptive basin hopping monte carlo sampling. IEEE Comput Soc Conf Comput Vis Pattern Recognit Work 1208–1215. https://doi.org/10.1109/CVPRW.2009.5206502

  22. Li M, Chen W, Huang K, Tan T (2010) Visual tracking via incremental self-tuning particle filtering on the affine group. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 1315–1322. https://doi.org/10.1109/CVPR.2010.5539815

  23. Li X, Hu W, Shen C et al (2013) A Survey of Appearance Models in Visual Object Tracking. ACM Trans Intell Syst Technol 4:1–42. https://doi.org/10.1145/2508037.2508039

    Google Scholar 

  24. Liu B, Yang L, Huang J, et al (2010) Robust and fast collaborative tracking with two stage sparse optimization. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 624–637. https://doi.org/10.1007/978-3-642-15561-1_45

  25. Liu B, Huang J, Yang L, Kulikowsk C (2011) Robust tracking using local sparse appearance model and K-selection. Computer Vision and Pattern Recognition (CVPR), 2011 I.E. Conference on. IEEE, Colorado Springs, pp 1313–1320. https://doi.org/10.1109/CVPR.2011.5995730

  26. Lu H, Zhou Q, Wang D, Xiang R (2011) A co-training framework for visual tracking with multiple instance learning. Face Gesture 2011:539–544. https://doi.org/10.1109/FG.2011.5771455

    Google Scholar 

  27. Ma AJ, Yuen PC, Lai JH (2013) Linear dependency modeling for classifier fusion and feature combination. IEEE Trans Pattern Anal Mach Intell 35:1135–1148. https://doi.org/10.1109/TPAMI.2012.198

    Article  Google Scholar 

  28. Mei X, Ling H (2009) Robust visual tracking using L1 minimization. IEEE Int Conf Comput Vis 1436–1443. https://doi.org/10.1109/ICCV.2009.5459292

  29. Ndiour IJ, Vela PA. (2010) A local extended Kalman filter for visual tracking. Decis Control (CDC), 2010 49th IEEE Conf 2420–2426. https://doi.org/10.1109/CDC.2010.5717339

  30. Nejhum SMS, Ho J, Yang MH (2010) Online visual tracking with histograms and articulating blocks. Comput Vis Image Underst 114:901–914. https://doi.org/10.1016/j.cviu.2010.04.002

    Article  Google Scholar 

  31. Pan Z, Liu S, Fu W (2016) A review of visual moving target tracking. Multimed Tools Appl 1–30. https://doi.org/10.1007/s11042-016-3647-0

  32. Pérez P, Hue C, Vermaak J, Gangnet M (2002) Color-based probabilistic tracking. Proc Eur Conf Comput Vis 661–675. https://doi.org/10.1007/3-540-47969-4_44

  33. Ross DA, Lim J, Lin R-S, Yang M-H (2007) Incremental Learning for Robust Visual Tracking. Int J Comput Vis 77:125–141. https://doi.org/10.1007/s11263-007-0075-7

    Article  Google Scholar 

  34. Smeulders AWM, Chu DM, Cucchiara R et al (2014) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36:1442–1468. https://doi.org/10.1109/TPAMI.2013.230

    Article  Google Scholar 

  35. Son J, Jung I, Park K, Han B (2016) Tracking-by-segmentation with online gradient boosting decision tree. Proc IEEE Int Conf Comput Vis 3056–3064. https://doi.org/10.1109/ICCV.2015.350

  36. Tang F, Brennan S, Zhao Q, Tao H (2007) Co-tracking using semi-supervised support vector machines. Proc IEEE Int Conf Comput Vis 1-8. https://doi.org/10.1109/ICCV.2007.4408954

  37. Wang D, Lu H (2014) Visual tracking via probability continuous outlier model. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 3478–3485. https://doi.org/10.1109/CVPR.2014.445

  38. Wang N, Wang J, Yeung DY (2013) Online robust non-negative dictionary learning for visual tracking. Proc IEEE Int Conf Comput Vis 657–664. https://doi.org/10.1109/ICCV.2013.87

  39. Wang D, Lu H, Yang MH (2013) Least soft-threshold squares tracking. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2371–2378. https://doi.org/10.1109/CVPR.2013.307

  40. Wang L, Yan H, Lv K, Pan C (2014) Visual tracking via kernel sparse representation with multikernel fusion. IEEE Trans Circuits Syst Video Technol 24:1132–1141. https://doi.org/10.1109/TCSVT.2014.2302496

    Article  Google Scholar 

  41. Wang D, Lu H, Bo C (2014) Online visual tracking via two view sparse representation. IEEE Signal Process Lett 21:1031–1034. https://doi.org/10.1109/LSP.2014.2322389

    Google Scholar 

  42. Wang D, Lu H, Xiao Z, Yang MH (2015) Inverse sparse tracker with a locally weighted distance metric. IEEE Trans Image Process 24:2646–2657. https://doi.org/10.1109/TIP.2015.2427518

    Article  MathSciNet  Google Scholar 

  43. Wang D, Lu H, Bo C (2015) Visual Tracking via Weighted Local Cosine Similarity. IEEE Trans Cybern 45:1838–1850. https://doi.org/10.1109/TCYB.2014.2360924

    Article  Google Scholar 

  44. Wang X, Valstar M, Martinez B, et al (2016) TRIC-track: tracking by regression with incrementally learned cascades. Proc IEEE Int Conf Comput Vis 4337–4345. https://doi.org/10.1109/ICCV.2015.493

  45. Xiao Z, Lu H, Wang D (2014) L2-RLS-based object tracking. IEEE Trans Circuits Syst Video Technol 24:1301–1309. https://doi.org/10.1109/TCSVT.2013.2291355

    Article  Google Scholar 

  46. Xie C, Tan J, Chen P et al (2014) Collaborative object tracking model with local sparse representation. J Vis Commun Image Represent 25:423–434. https://doi.org/10.1016/j.jvcir.2013.12.012

    Article  Google Scholar 

  47. Xing J, Gao J, Li B et al (2013) Robust object tracking with online multi-lifespan dictionary learning. Proc IEEE Int Conf Comput Vis 665–672. https://doi.org/10.1109/ICCV.2013.88

  48. Xiong J, Tang Q, He X et al (2016) Tracking in multimedia data via robust reweighted local multi-task sparse representation for transportation surveillance. Multimed Tools Appl 75(24):17531-17552. https://doi.org/10.1007/s11042-016-3464-5

  49. Yang F, Lu H, Yang M-H (2014) Robust Visual Tracking via Multiple Kernel Boosting With Affinity Constraints. IEEE Trans Circuits Syst Video Technol 24:242–254. https://doi.org/10.1109/TCSVT.2013.2276145

    Article  Google Scholar 

  50. Yi Y, Xu H (2014) Hierarchical data association framework with occlusion handling for multiple targets tracking. IEEE Signal Process Lett 21:288–291. https://doi.org/10.1109/LSP.2014.2300497

    Article  Google Scholar 

  51. Yi Y, Mo Z, Tan JW (2016) A novel hierarchical data association with dynamic viewpoint model for multiple targets tracking. J Vis Commun Image Represent 34:37–49. https://doi.org/10.1016/j.jvcir.2015.10.010

    Article  Google Scholar 

  52. Zhang S, Yao H, Sun X, Lu X (2013) Sparse coding based visual tracking: Review and experimental comparison. Pattern Recogn 46:1772–1788. https://doi.org/10.1016/j.patcog.2012.10.006

    Article  Google Scholar 

  53. Zhang H, Tao F, Yang G (2013) Robust visual tracking based on structured sparse representation model. Multimed Tools Appl 74:1021–1043. https://doi.org/10.1007/s11042-013-1709-0

    Article  Google Scholar 

  54. Zhang S, Yao H, Zhou H et al (2013) Robust visual tracking based on online learning sparse representation. Neurocomputing 100:31–40. https://doi.org/10.1016/j.neucom.2011.11.031

    Article  Google Scholar 

  55. Zhang S, Zhou H, Jiang F, Li X (2015) Robust Visual Tracking Using Structurally Random Projection and Weighted Least Squares. IEEE Trans Circuits Syst Video Technol 25:1749–1760. https://doi.org/10.1109/TCSVT.2015.2406194

    Article  Google Scholar 

  56. Zhang S, Lan X, Yao H, et al (2016) A biologically inspired appearance model for robust visual tracking. IEEE Trans Neural Networks Learn Syst 1-14. https://doi.org/10.1109/TNNLS.2016.2586194

  57. Zhang S, Lan X, Qi Y, Yuen PC (2017) Robust Visual Tracking via Basis Matching. IEEE Trans Circuits Syst Video Technol 27:421–430. https://doi.org/10.1109/TCSVT.2016.2539860

    Article  Google Scholar 

  58. Zhang D, Maei H, Wang X, Wang Y-F (2017) Deep reinforcement learning for visual object tracking in videos. http://arxiv.org/abs/1701.08936

  59. Zhong W, Lu H, Yang M-H (2012) Robust object tracking via sparsity-based collaborative model. CVPR 1838–1845. https://doi.org/10.1109/CVPR.2012.6247882

  60. Zhu G, Porikli F, Li H (2016) Robust visual tracking with deep convolutional neural network based object proposals on PETS. 2016 I.E. Conf Comput Vis Pattern Recognit Work 1265–1272. https://doi.org/10.1109/CVPRW.2016.160

Download references

Acknowledgements

The authors would like to thank Fang Li M.Sc. for her insightful and inspirational comments which have greatly helped us to improve the technical contents and experiments of the study. And this work was partly supported by National Natural Science Foundation of China with NO: 61672546 and 61573385, Guangzhou Science and Technology Project with No. 201707010127 and No. 2014B010112001.

Author information

Authors and Affiliations

  1. School of Data & Computer Science, Sun Yat-sen University, Guangzhou, 510006, China

    Yang Yi, Yang Cheng & Chuping Xu

  2. Xinhua College of Sun Yat-sen University, Guangzhou, 510520, China

    Yang Yi

  3. Guangdong Province Key Laboratory of Big Data Analysis and Processing, Guangzhou, China

    Yang Yi

Authors
  1. Yang Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chuping Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Cheng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yi, Y., Cheng, Y. & Xu, C. Visual tracking based on hierarchical framework and sparse representation. Multimed Tools Appl 77, 16267–16289 (2018). https://doi.org/10.1007/s11042-017-5198-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5198-4

Keywords

Search

Navigation