Skip to main content
SpringerLink
Log in

Thermal infrared object tracking using correlation filters improved by level set

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Existing thermal infrared (TIR) trackers based on correlation filters cannot adapt to the abrupt scale variation of nonrigid objects. This deficiency could even lead to tracking failure. To address this issue, we propose a TIR tracker, called ECO_LS, which improves the performance of efficient convolution operators (ECO) via the level set method. We first utilize the level set to segment the local region estimated by the ECO tracker to gain a more accurate size of the bounding box when the object changes its scale suddenly. Then, to accelerate the convergence speed of the level set contour, we leverage its historical information and continuously encode it to effectively decrease the number of iterations. In addition, our variant, ECOHG_LS, also achieves better performance via concatenating histogram of oriented gradient (HOG) and gray features to represent the object. Furthermore, experimental results on three infrared object tracking benchmarks show that the proposed approach performs better than other competing trackers. ECO_LS improves the EAO by 20.97% and 30.59% over the baseline ECO on VOT-TIR2016 and VOT-TIR2015, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Danelljan, M., Bhat, G., Shahbaz Khan, F., Felsberg, M.: Eco: efficient convolution operators for tracking. In: CVPR, pp. 6638–6646 (2017)

  2. Wang, N., Zhou, W., Tian, Q., Hong, R., Wang, M., Li, H.: Multi-cue correlation filters for robust visual tracking. In: CVPR, pp. 4844–4853 (2018)

  3. Xu, T., Feng, Z.-H., Wu, X.-J., Kittler, J.: Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking. IEEE Trans. Image Process. 28(11), 5596–5609 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  4. Li, B., Yan, J., Wu, W., Zhu, Z., Hu, X.: High performance visual tracking with siamese region proposal network. In: CVPR, pp. 8971–8980 (2018)

  5. Wang, Q., Zhang, L., Bertinetto, L., Hu, W., Torr, P.H.: Fast online object tracking and segmentation: a unifying approach. In: CVPR, pp. 1328–1338 (2019)

  6. Liu, Q., Li, X., He, Z., Fan, N., Yuan, D., Liu, W., Liang, Y.: Multi-task driven feature models for thermal infrared tracking. In: AAAI, vol. 34, pp. 11604–11611 (2020)

  7. Liu, Q., Li, X., He, Z., Fan, N., Yuan, D., Wang, H.: Learning deep multi-level similarity for thermal infrared object tracking. IEEE Trans. Multimedia 23, 2114–2126 (2020)

    Article  Google Scholar 

  8. Li, X., Liu, Q., Fan, N., He, Z., Wang, H.: Hierarchical spatial-aware siamese network for thermal infrared object tracking. Knowledge-Based Syst. 166, 71–81 (2019)

    Article  Google Scholar 

  9. Demir, H.S., Cetin, A.E.: Co-difference based object tracking algorithm for infrared videos. In: ICIP, pp. 434–438 (2016). IEEE

  10. Li, C., Huang, R., Ding, Z., Gatenby, J.C., Metaxas, D.N., Gore, J.C.: A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI. IEEE Trans. Image Process. 20(7), 2007–2016 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Liu, Q., He, Z., Li, X., Zheng, Y.: PTB-TIR: a thermal infrared pedestrian tracking benchmark. IEEE Trans. Multimedia 22(3), 666–675 (2019)

    Article  Google Scholar 

  12. Felsberg, M., Kristan, M., Matas, J., Leonardis, A., et al.: The thermal infrared visual object tracking VOT-TIR2016 challenge results. In: ECCVW, pp. 824–849 (2016)

  13. Berg, A., Ahlberg, J., Felsberg, M.: A thermal object tracking benchmark. In: AVSS, pp. 1–6 (2015)

  14. Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: CVPR, pp. 2544–2550 (2010)

  15. Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2014)

    Article  Google Scholar 

  16. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893 (2005)

  17. Danelljan, M., Hager, G., Shahbaz Khan, F., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: CVPR, pp. 4310–4318 (2015)

  18. Danelljan, M., Robinson, A., Khan, F.S., Felsberg, M.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: ECCV, pp. 472–488 (2016)

  19. Liu, Q., Lu, X., He, Z., Zhang, C., Chen, W.-S.: Deep convolutional neural networks for thermal infrared object tracking. Knowledge-Based Syst. 134, 189–198 (2017)

    Article  Google Scholar 

  20. Zhang, L., Gonzalez-Garcia, A., Van De Weijer, J., Danelljan, M., Khan, F.S.: Synthetic data generation for end-to-end thermal infrared tracking. IEEE Trans. Image Process. 28(4), 1837–1850 (2018)

    Article  MathSciNet  Google Scholar 

  21. He, Y.-J., Li, M., Zhang, J., Yao, J.-P.: Infrared target tracking via weighted correlation filter. Infrared Phys. Technol. 73, 103–114 (2015)

    Article  Google Scholar 

  22. Wu, S., Zhang, K., Li, S., Yan, J.: Joint feature embedding learning and correlation filters for aircraft tracking with infrared imagery. Neurocomputing 450, 104–118 (2021)

    Article  Google Scholar 

  23. Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1561–1575 (2017)

    Article  Google Scholar 

  24. Kristan, M., Matas, J., et al.: The visual object tracking VOT2015 challenge results. In: ICCVW, pp. 1–23 (2015)

  25. Li, F., Tian, C., Zuo, W., Zhang, L., Yang, M.-H.: Learning spatial-temporal regularized correlation filters for visual tracking. In: CVPR, pp. 4904–4913 (2018)

  26. Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: CVPR, pp. 4293–4302 (2016)

  27. Song, Y., Ma, C., Wu, X., Gong, L., Bao, L., Zuo, W., Shen, C., Lau, R.W., Yang, M.-H.: Vital: visual tracking via adversarial learning. In: CVPR, pp. 8990–8999 (2018)

  28. Wang, N., Song, Y., Ma, C., Zhou, W., Liu, W., Li, H.: Unsupervised deep tracking. In: CVPR, pp. 1308–1317 (2019)

  29. Dong, X., Shen, J.: Triplet loss in siamese network for object tracking. In: ECCV, pp. 459–474 (2018)

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of Changsha (kq2202176), in part by Key R &D Program of Hunan (2022SK2104), in part by Leading plan for scientific and technological innovation of high-tech industries of Hunan (2022GK4010), in part by National Key R &D Program of China (2021YFF0900600), and in part by the National Natural Science Foundation of China (61672222).

Author information

Authors and Affiliations

  1. School of Design, Hunan University, Changsha, 410082, China

    He Zhang & Hanling Zhang

  2. College of Computer Science and Electronic Engineering, Hunan University, Changsha, 410082, China

    Zhaoyuan Yin

Authors
  1. He Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhaoyuan Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanling Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Yin, Z. & Zhang, H. Thermal infrared object tracking using correlation filters improved by level set. SIViP 17, 791–797 (2023). https://doi.org/10.1007/s11760-022-02289-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02289-x

Keywords

Search

Navigation