Abstract
Existing thermal infrared (TIR) trackers based on correlation filters cannot adapt to the abrupt scale variation of nonrigid objects. This deficiency could even lead to tracking failure. To address this issue, we propose a TIR tracker, called ECO_LS, which improves the performance of efficient convolution operators (ECO) via the level set method. We first utilize the level set to segment the local region estimated by the ECO tracker to gain a more accurate size of the bounding box when the object changes its scale suddenly. Then, to accelerate the convergence speed of the level set contour, we leverage its historical information and continuously encode it to effectively decrease the number of iterations. In addition, our variant, ECOHG_LS, also achieves better performance via concatenating histogram of oriented gradient (HOG) and gray features to represent the object. Furthermore, experimental results on three infrared object tracking benchmarks show that the proposed approach performs better than other competing trackers. ECO_LS improves the EAO by 20.97% and 30.59% over the baseline ECO on VOT-TIR2016 and VOT-TIR2015, respectively.
Similar content being viewed by others
References
Danelljan, M., Bhat, G., Shahbaz Khan, F., Felsberg, M.: Eco: efficient convolution operators for tracking. In: CVPR, pp. 6638–6646 (2017)
Wang, N., Zhou, W., Tian, Q., Hong, R., Wang, M., Li, H.: Multi-cue correlation filters for robust visual tracking. In: CVPR, pp. 4844–4853 (2018)
Xu, T., Feng, Z.-H., Wu, X.-J., Kittler, J.: Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking. IEEE Trans. Image Process. 28(11), 5596–5609 (2019)
Li, B., Yan, J., Wu, W., Zhu, Z., Hu, X.: High performance visual tracking with siamese region proposal network. In: CVPR, pp. 8971–8980 (2018)
Wang, Q., Zhang, L., Bertinetto, L., Hu, W., Torr, P.H.: Fast online object tracking and segmentation: a unifying approach. In: CVPR, pp. 1328–1338 (2019)
Liu, Q., Li, X., He, Z., Fan, N., Yuan, D., Liu, W., Liang, Y.: Multi-task driven feature models for thermal infrared tracking. In: AAAI, vol. 34, pp. 11604–11611 (2020)
Liu, Q., Li, X., He, Z., Fan, N., Yuan, D., Wang, H.: Learning deep multi-level similarity for thermal infrared object tracking. IEEE Trans. Multimedia 23, 2114–2126 (2020)
Li, X., Liu, Q., Fan, N., He, Z., Wang, H.: Hierarchical spatial-aware siamese network for thermal infrared object tracking. Knowledge-Based Syst. 166, 71–81 (2019)
Demir, H.S., Cetin, A.E.: Co-difference based object tracking algorithm for infrared videos. In: ICIP, pp. 434–438 (2016). IEEE
Li, C., Huang, R., Ding, Z., Gatenby, J.C., Metaxas, D.N., Gore, J.C.: A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI. IEEE Trans. Image Process. 20(7), 2007–2016 (2011)
Liu, Q., He, Z., Li, X., Zheng, Y.: PTB-TIR: a thermal infrared pedestrian tracking benchmark. IEEE Trans. Multimedia 22(3), 666–675 (2019)
Felsberg, M., Kristan, M., Matas, J., Leonardis, A., et al.: The thermal infrared visual object tracking VOT-TIR2016 challenge results. In: ECCVW, pp. 824–849 (2016)
Berg, A., Ahlberg, J., Felsberg, M.: A thermal object tracking benchmark. In: AVSS, pp. 1–6 (2015)
Bolme, D.S., Beveridge, J.R., Draper, B.A., Lui, Y.M.: Visual object tracking using adaptive correlation filters. In: CVPR, pp. 2544–2550 (2010)
Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2014)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893 (2005)
Danelljan, M., Hager, G., Shahbaz Khan, F., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: CVPR, pp. 4310–4318 (2015)
Danelljan, M., Robinson, A., Khan, F.S., Felsberg, M.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: ECCV, pp. 472–488 (2016)
Liu, Q., Lu, X., He, Z., Zhang, C., Chen, W.-S.: Deep convolutional neural networks for thermal infrared object tracking. Knowledge-Based Syst. 134, 189–198 (2017)
Zhang, L., Gonzalez-Garcia, A., Van De Weijer, J., Danelljan, M., Khan, F.S.: Synthetic data generation for end-to-end thermal infrared tracking. IEEE Trans. Image Process. 28(4), 1837–1850 (2018)
He, Y.-J., Li, M., Zhang, J., Yao, J.-P.: Infrared target tracking via weighted correlation filter. Infrared Phys. Technol. 73, 103–114 (2015)
Wu, S., Zhang, K., Li, S., Yan, J.: Joint feature embedding learning and correlation filters for aircraft tracking with infrared imagery. Neurocomputing 450, 104–118 (2021)
Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Discriminative scale space tracking. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1561–1575 (2017)
Kristan, M., Matas, J., et al.: The visual object tracking VOT2015 challenge results. In: ICCVW, pp. 1–23 (2015)
Li, F., Tian, C., Zuo, W., Zhang, L., Yang, M.-H.: Learning spatial-temporal regularized correlation filters for visual tracking. In: CVPR, pp. 4904–4913 (2018)
Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: CVPR, pp. 4293–4302 (2016)
Song, Y., Ma, C., Wu, X., Gong, L., Bao, L., Zuo, W., Shen, C., Lau, R.W., Yang, M.-H.: Vital: visual tracking via adversarial learning. In: CVPR, pp. 8990–8999 (2018)
Wang, N., Song, Y., Ma, C., Zhou, W., Liu, W., Li, H.: Unsupervised deep tracking. In: CVPR, pp. 1308–1317 (2019)
Dong, X., Shen, J.: Triplet loss in siamese network for object tracking. In: ECCV, pp. 459–474 (2018)
Acknowledgements
This work was supported in part by the National Natural Science Foundation of Changsha (kq2202176), in part by Key R &D Program of Hunan (2022SK2104), in part by Leading plan for scientific and technological innovation of high-tech industries of Hunan (2022GK4010), in part by National Key R &D Program of China (2021YFF0900600), and in part by the National Natural Science Foundation of China (61672222).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, H., Yin, Z. & Zhang, H. Thermal infrared object tracking using correlation filters improved by level set. SIViP 17, 791–797 (2023). https://doi.org/10.1007/s11760-022-02289-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-022-02289-x