Skip to main content
SpringerLink
Log in

Robust visual tracker integrating adaptively foreground segmentation into multi-feature fusion framework

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Existing Discriminative Correlation Filter (DCF) based methods suffer from the limitations of rectangular shape assumptions. Aiming at this issue, in this paper, we propose an effective tracking approach which integrates a pixel-wise foreground segmentation mask into the correlation filter within a multi-feature fusion framework. Specifically, we first propose a novel segmentation algorithm which combines the color histogram with the spatial prior. On this basis, we implement a target-masked correlation filter (TMCF) tracker by introducing the foreground mask into a ridge regression, which successfully suppresses unexpected background information inside the bounding box. Secondly, we apply the alternating direction method of multipliers (ADMM) to solve our TMCF model efficiently to obtain the closed-form solution. Finally, a complementary fusion tracker by the combining of TMCF and color histogram scores (fTMCFCH) is formulated, which is robust to deformations and illumination changes simultaneously. The fusion factor is determined adaptively by the reliability derived from the target resolution of the trackers separately in each frame. We perform extensive experiments on three benchmarks: OTB-2013, OTB-2015 and Temple-Color-128. The concrete experimental results demonstrate that our tracker outperforms several state-of-the-art trackers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Aeschliman C, Park J, Kak AC (2010) A probabilistic framework for joint segmentation and tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1371–1378

  2. Avidan S (2004) Support vector tracking. IEEE Trans Pattern Anal Mach Intell 26(8):1064–1072

    Article  Google Scholar 

  3. Bertinetto L, Valmadre J, Golodetz S, Miksik O, Torr PHS (2016) Staple: complementary learners for real-time tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1401–1409

  4. Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2544–2550

  5. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  6. Danelljan M, Hager G, Khan FS, Felsberg M (2014) Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference, pp. 1–11

  7. Danelljan M, Khan FS, Felsberg M, Weijer JVD (2014) Adaptive color attributes for real-time visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1090–1097

  8. Danelljan M, Häger G, Khan FS, Felsberg M (2015) Learning spatially regularized correlation filters for visual tracking. In: IEEE International Conference on Computer Vision, pp. 4310–4318

  9. Danelljan M, Bhat G, Khan FS, Felsberg M (2017) ECO: efficient convolution operators for tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6931–6939

  10. Dinh TB , Vo N , Gérard G. Medioni (2011) Context tracker: exploring supporters and distracters in unconstrained environments. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1177–1184

  11. Duffner S, Garcia C (2013) PixelTrack: a fast adaptive algorithm for tracking non-rigid objects. In: IEEE International Conference on Computer Vision, pp. 2480–2487

  12. Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: IEEE International Conference on Computer Vision, pp. 1144–1152

  13. Grabner H, Bischof H (2006) Online boosting and vision. IEEE Conf CompVision Patt Recog 1(1):260–267

    Google Scholar 

  14. He S, Yang Q, Lau RWH, Wang J, Yang MH(2013) Visual tracking via locality sensitive histograms. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2427–2434

  15. Henriques JF, Caseiro R, Martins P, Batista JP (2012) Exploiting the circulant structure of tracking-by-detection with kernels. European Conference on Computer Vision, In, pp 702–715

    Google Scholar 

  16. Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with Kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596

    Article  Google Scholar 

  17. Kart U, Kamarainen JK, Matas J (2018) How to make an RGBD tracker. In:European Conference on Computer Vision, pp 0–0

  18. Keuper M , Andres B , Brox T (2015) Motion trajectory segmentation via minimum cost multicuts. In: IEEE International Conference on Computer Vision, pp. 3271–3279

  19. Kwon J, Lee KM (2010) Visual tracking decomposition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1269–1276

  20. Li Y, Zhu J (2014) A scale adaptive kernel correlation filter tracker with feature integration. In: European Conference on Computer Vision, pp. 254–265

  21. Li Y, Zhu J, Hoi SC (2015) Reliable patch trackers: robust visual tracking by exploiting reliable patches. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 353–361

  22. Li C, Lin L, Zuo W, Tang J (2017) Learning patch-based dynamic graph for visual tracking. In: AAAI Conf Artificial Intelligence, pp. 4126–4132

  23. Li F, Tian C, Zuo W, Zhang L, Yang MH (2018) Learning spatial-temporal regularized correlation filters for visual tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4904–4913

  24. Liang P, Blasch E, Ling H (2015) Encoding color information for visual tracking: algorithms and benchmark. IEEE Trans Image Process 24(12):5630–5644

    Article  MathSciNet  Google Scholar 

  25. Lukežič A, VojířT, Čehovin L, Matas J, Kristan M (2017) Discriminative correlation filter with channel and spatial reliability. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4847–4856

  26. Ma C, Yang X, Zhang C, Yang MH (2015) Long-term correlation tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5388–5396

  27. Ma C, Huang J, Yang X, Yang MH (2015) Hierarchical convolutional features for visual tracking. In: IEEE International Conference on Computer Vision, pp. 3074–3082

  28. Mei X, Ling H (2011) Robust visual tracking and vehicle classification via sparse representation. IEEE Trans Pattern Anal Mach Intell 33(11):2259–2272

    Article  Google Scholar 

  29. Papazoglou A, Ferrari V (2013) Fast object segmentation in unconstrained video. In: IEEE International Conference on Computer Vision, pp1777–1784

  30. Possegger H, Mauthner T, Bischof H (2015) In defense of color-based model-free tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2113–2120

  31. Ross DA, Lim J, Lin RS, Yang MH (2008) Incremental learning for robust visual tracking. Int J Comput Vis 77(1–3):125–141

    Article  Google Scholar 

  32. Saffari A, Leistner C, Santner J, Godec M, Bischof H (2009) On-line random forests. In: IEEE Conference on Computer Vision Workshop, pp. 1393–1400

  33. Tang M, Feng J (2015) Multi-kernel correlation filter for visual tracking. In: IEEE International Conference on Computer Vision, pp. 3038–3046

  34. Tsai YH , Yang MH , Black MJ (2016) Video segmentation via object flow. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3899–3908

  35. Wang S, Lu H, Yang F, Yang MH (2011) Superpixel tracking. In: Proc. IEEE International Conference on Computer Vision, pp. 1323–1330

  36. Wen J, Xu Y, Li Z, Ma Z, Xu Y (2018) Inter-class sparsity based discriminative least square regression. Neural Netw 102:36–47

    Article  Google Scholar 

  37. Wu Y, Lim J, Yang MH (2013) Online object tracking: a benchmark. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2411–2418

  38. Wu Y, Lim J, Yang MH (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834–1848

    Article  Google Scholar 

  39. Yeo D, Son J, Han B, Han JH (2017) Superpixel-based tracking-by-segmentation using markov chains. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 511–520

  40. Yilmaz A, Javed O, Shah M (2006) Object tracking: a Survey. ACM Comput Surv 38(4):13

    Article  Google Scholar 

  41. Zhang H, Liu G (2018) Coupled-layer based visual tracking via adaptive kernelized correlation filters. Vis Comput 34(1):41–54

    Article  MathSciNet  Google Scholar 

  42. Zhang H, Liu G, Hao Z (2018) Robust visual tracking via multi-feature response maps fusion using a collaborative local-global layer visual model. J Vis Commun Image Represent 56:1–14

    Article  Google Scholar 

  43. Zhu G, Wang J, Wu Y, Zhang X, Lu H (2016) MC-HOG correlation tracking with saliency proposal. In: Thirtieth AAAI Conference on Artificial Intelligence, pp. 3690–3696

Download references

Acknowledgments

This work is supported by the Foundation of Preliminary Research Field of China (Grant No. 6140312030217, 61405170206), the 13th Five-Year Equipment Development Project of China (Grant No. 41412010202), National Natural Science Foundation of China(Grant No.61972307, and the Open Foundation of Shaanxi Key Laboratory of Integrated and Intelligent Navigation under Grant No. SKLIIN-20180108.

Author information

Authors and Affiliations

  1. School of Mechano-Electronic Engineering, Xidian University, Xi’an, 710071, China

    Yi Zhang, Guixi Liu & Haoyang Zhang

  2. Shaanxi Key Laboratory of Integrated and Intelligent Navigation, Xi’an, 710068, China

    Yi Zhang, Guixi Liu, Jiayu Gao & Haoyang Zhang

  3. Xi’an Research Institute of Navigation Technology, Xi’an, 710068, China

    Jiayu Gao

Authors
  1. Yi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guixi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiayu Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haoyang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guixi Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Liu, G., Gao, J. et al. Robust visual tracker integrating adaptively foreground segmentation into multi-feature fusion framework. Multimed Tools Appl 79, 31865–31888 (2020). https://doi.org/10.1007/s11042-020-09443-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09443-y

Keywords

Search

Navigation