Abstract
Object tracking as a low-level vision task has always been a hot topic in computer vision. It is well known that Challenges such as background clutters, fast object motion and occlusion et al. affect a lot the robustness or accuracy of existing object tracking methods. This paper proposes a reinforcement learning model based on Twin Delayed Deep Deterministic algorithm (TD3) for single object tracking. The model is based on the deep reinforcement learning model, Actor-Critic (AC), in which the Actor network predicts a continuous action that moves the target bounding box in the previous frame to the object position in the current frame and adapts to the object size. The Critic network evaluates the confidence of the new bounding box online to determine whether the Critic model needs to be updated or re-initialized. In further, in our model we use TD3 algorithm to further optimize the AC model by using two Critic networks to jointly predict the bounding box confidence, and to obtain the smaller predicted value as the label to update the network parameters, thereby rendering the Critic network to avoid excessive estimation bias, accelerate the convergence of the loss function, and obtain more accurate prediction values. Also, a small amount of random noise with upper and lower bounds are added to the action in the Actor model, and the search area is reasonably expanded in offline learning to improve the robustness of the tracking method under strong background interference and fast object motion. The Critic model can also guide the Actor model to select the best action and continuously update the state of the tracking object. Comprehensive experimental results on the OTB-2013 and OTB-2015 benchmarks demonstrate that our tracker performs best in precision, robustness, and efficiency when compared with state-of-the-art methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Henriques, J.F., Caseiro, R., Martins, P., et al.: High-speed tracking with kernelized correlation filters. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 583–596 (2014)
Danelljan, M., Häger, G., Khan, F., et al. : Accurate scale estimation for robust visual tracking. In: British Machine Vision Conference, Nottingham, 1–5 September 2017 (2014)
Danelljan, M., Robinson, A., Shahbaz Khan, F., Felsberg, M.: Beyond correlation filters: learning continuous convolution operators for visual tracking. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 472–488. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_29
Danelljan, M., Bhat, G., Shahbaz Khan, F., Felsberg, M.: ECO: efficient convolution operators for tracking. In: CVPR, pp. 6638–6646 (2017)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Ma, C., Huang, J.B., Yang, X., et al.: Hierarchical convolutional features for visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3074–3082 (2015)
Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional Siamese networks for object tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 850–865. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_56
Ma, C., Yang, X., Zhang, C., Yang, M.-H.: Long-term correlation tracking. In: CVPR, pp. 5388–5396 (2018)
Nam, H., Han, B.: Learning multi-domain convolutional neural networks for visual tracking. In: CVPR, pp. 4293–4302 (2016)
Fan H., Ling H.: Parallel tracking and verifying: A framework for real-time and high accuracy visual tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5486–5494(2017)
Yun, S., Choi, J., Yoo, Y., Yun, K., Young Choi, J.: Action-decision networks for visual tracking with deep reinforcement learning. In: CVPR, pp. 2711–2720 (2017)
Chen, B., Wang, D., Li, P., Wang, S., Lu, H.: Real-time ‘Actor-Critic’ tracking. In: ECCV, pp. 318–334 (2018)
Bibi, A., Mueller, M., Ghanem, B.: Target response adaptation for correlation filter tracking. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 419–433. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_25
Acknowledgement
This work is supported by National Science Foundation of China (Grant No. 61703209 and 61773215).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, S., Wang, H. (2019). Real-Time Visual Object Tracking Based on Reinforcement Learning with Twin Delayed Deep Deterministic Algorithm. In: Cui, Z., Pan, J., Zhang, S., Xiao, L., Yang, J. (eds) Intelligence Science and Big Data Engineering. Visual Data Engineering. IScIDE 2019. Lecture Notes in Computer Science(), vol 11935. Springer, Cham. https://doi.org/10.1007/978-3-030-36189-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-36189-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36188-4
Online ISBN: 978-3-030-36189-1
eBook Packages: Computer ScienceComputer Science (R0)