research-article

Robust Correlation Filter Tracking with Shepherded Instance-Aware Proposals

Authors:

Hanzi WangAuthors Info & Claims

MM '18: Proceedings of the 26th ACM international conference on Multimedia

Pages 420 - 428

https://doi.org/10.1145/3240508.3240709

Published: 15 October 2018 Publication History

Abstract

In recent years, convolutional neural network (CNN) based correlation filter trackers have achieved state-of-the-art results on the benchmark datasets. However, the CNN based correlation filters cannot effectively handle large scale variation and distortion (such as fast motion, background clutter, occlusion, etc.), leading to the sub-optimal performance. In this paper, we propose a novel CNN based correlation filter tracker with shepherded instance-aware proposals, namely DeepCFIAP, which automatically estimates the target scale in each frame and re-detects the target when distortion happens. DeepCFIAP is proposed to take advantage of the merits of both instance-aware proposals and CNN based correlation filters. Compared with the CNN based correlation filter trackers, DeepCFIAP can successfully solve the problems of large scale variation and distortion via the shepherded instance-aware proposals, resulting in more robust tracking performance. Specifically, we develop a novel proposal ranking algorithm based on the similarities between proposals and instances. In contrast to the detection proposal based trackers, DeepCFIAP shepherds the instance-aware proposals towards their optimal positions via the CNN based correlation filters, resulting in more accurate tracking results. Extensive experiments on two challenging benchmark datasets demonstrate that the proposed DeepCFIAP performs favorably against state-of-the-art trackers and it is especially feasible for long-term tracking.

References

[1]

L. Bertinetto, J. Valmadre, S. Golodetz, O. Miksik, and P. H. S. Torr. 2016. Staple: Complementary Learners for Real-Time Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 1401--1409.

[2]

L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. S. Torr. 2016. Fully-Convolutional Siamese Networks for Object Tracking. In Proc. of European Conference on Computer Vision Workshops. 850--865.

[3]

M. Danelljan, G. Bhat, F. S. Khan, and M. Felsberg. 2017. ECO: Efficient Convolution Operators for Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 6931--6939.

[4]

M. Danelljan, G. Häger, F. S. Khan, and M. Felsberg. 2015. Convolutional Features for Correlation Filter Based Visual Tracking. In Proc. of IEEE International Conference on Computer Vision Workshops. 621--629.

Digital Library

[5]

M. Danelljan, G. Häger, F. S. Khan, and M. Felsberg. 2015. Learning Spatially Regularized Correlation Filters for Visual Tracking. In Proc. of IEEE International Conference on Computer Vision. 4310--4318.

Digital Library

[6]

M. Danelljan, G. Häger, F. S. Khan, and M. Felsberg. 2017. Discriminative Scale Space Tracking. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 39, 8 (August 2017), 1561--1575.

[7]

M. Danelljan, A. Robinson, F. S. Khan, and M. Felsberg. 2016. Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking. In Proc. of European Conference on Computer Vision . 472--488.

[8]

P. Dollár and C. L. Zitnick. 2013. Structured Forests for Fast Edge Detection. In Proc. of IEEE International Conference on Computer Vision. 1841--1848.

Digital Library

[9]

H. Fan and H. Ling. 2017. SANet: Structure-Aware Network for Visual Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2217--2224.

[10]

H. K. Galoogahi, A. Fagg, and S. Lucey. 2017. Learning Background-Aware Correlation Filters for Visual Tracking. In Proc. of IEEE International Conference on Computer Vision. 1144--1152.

[11]

Q. Guo, W. Feng, C. Zhou, R. Huang, L. Wan, and S. Wang. 2017. Learning Dynamic Siamese Network for Visual Object Tracking. In Proc. of IEEE International Conference on Computer Vision. 1781--1789.

[12]

J. F. Henriques, R. Caseiro, P. Martins, and J. Batista. 2015. High-Speed Tracking with Kernelized Correlation Filters. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 37, 3 (March 2015), 583--596.

[13]

Z. Hong, Z. Chen, C. Wang, X. Mei, D. Prokhorov, and D. Tao. 2015. MUlti-Store Tracker (MUSTer): A Cognitive Psychology Inspired Approach to Object Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 749--758.

[14]

Y. Hua, K. Alahari, and C. Schmid. 2015. Online Object Tracking with Proposal Selection. In Proc. of IEEE International Conference on Computer Vision. 3092--3100.

Digital Library

[15]

C. Huang, S. Lucey, and D. Ramanan. 2017. Learning Policies for Adaptive Tracking with Deep Feature Cascades. In Proc. of IEEE International Conference on Computer Vision. 105--114.

[16]

D. Huang, L. Luo, Z. Chen, M. Wen, and C. Zhang. 2017. Applying Detection Proposals to Visual Tracking for Scale and Aspect Ratio Adaptability. International Journal of Computer Vision, Vol. 122, 3 (May 2017), 524--541.

Digital Library

[17]

M. Kristan, J. Matas, A. Leonardis, M. Felsberg, L. Čehovin, G. Fernandez, T. VojíČ, G. Häger, G. Nebehay, R. Pflugfelder, A. Gupta, A. Bibi, A. Lukevžiž, A. Garcia-Martin, A. Saffari, and A. Petrosino. 2015. The Visual Object Tracking VOT2015 Challenge Results. In Proc. of IEEE International Conference on Computer Vision Workshops. 564--586.

Digital Library

[18]

A. Lukevžiž, T. VojíČ, L. Čehovin, J. Matas, and M. Kristan. 2017. Discriminative Correlation Filter with Channel and Spatial Reliability. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 4847--4856.

[19]

C. Ma, J. B. Huang, X. Yang, and M. H. Yang. 2015a. Hierarchical Convolutional Features for Visual Tracking. In Proc. of IEEE International Conference on Computer Vision. 3074--3082.

Digital Library

[20]

C. Ma, X. Yang, C. Zhang, and M. H. Yang. 2015. Long-Term Correlation Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 5388--5396.

[21]

M. Mueller, N. Smith, and B. Ghanem. 2016. A Benchmark and Simulator for UAV Tracking. In Proc. of European Conference on Computer Vision . 445--461.

[22]

M. Mueller, N. Smith, and B. Ghanem. 2017. Context-Aware Correlation Filter Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 1387--1395.

[23]

H. Nam and B. Han. 2016. Learning Multi-domain Convolutional Neural Networks for Visual Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 4293--4302.

[24]

Y. Qi, S. Zhang, L. Qin, H. Yao, Q. Huang, J. Lim, and M. H. Yang. 2016. Hedged Deep Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 4303--4311.

[25]

S. Ren, K. He, R. Girshick, and J. Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Proc. of International Conference on Neural Information Processing Systems . 91--99.

Digital Library

[26]

K. Simonyan and A. Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proc. of International Conference on Learning Representations .

[27]

Y. Song, C. Ma, L. Gong, J. Zhang, R. W. H. Lau, and M. H. Yang. 2017. CREST: Convolutional Residual Learning for Visual Tracking. In Proc. of IEEE International Conference on Computer Vision. 2574--2583.

[28]

Y. Sui, G. Wang, and L. Zhang. 2018. Correlation Filter Learning Toward Peak Strength for Visual Tracking. IEEE Trans. on Cybernetics, Vol. 48, 4 (April 2018), 1290--1303.

[29]

Y. Sui, Z. Zhang, G. Wang, Y. Tang, and L. Zhang. 2016. Real-Time Visual Tracking: Promoting the Robustness of Correlation Filter Learning. In Proc. of European Conference on Computer Vision. 662--678.

[30]

R. Tao, E. Gavves, and A. W. M. Smeulders. 2016. Siamese Instance Search for Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 1420--1429.

[31]

J. Valmadre, L. Bertinetto, J. Henriques, A. Vedaldi, and P. H. S. Torr. 2017. End-to-End Representation Learning for Correlation Filter Based Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 5000--5008.

[32]

M. Wang, Y. Liu, and Z. Huang. 2017. Large Margin Object Tracking with Circulant Feature Maps. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 4800--4808.

[33]

F. Xiao and Y. J. Lee. 2016. Track and Segment: An Iterative Unsupervised Approach for Video Object Proposals. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition . 933--942.

[34]

J. Lim Y. Wu and M. H. Yang. 2015. Object Tracking Benchmark. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 37, 9 (September 2015), 1834--1848.

[35]

L. Yang, R. Liu, D. Zhang, and L. Zhang. 2017. Deep Location-Specific Tracking. In Proc. of ACM on Multimedia Conference . 1309--1317.

Digital Library

[36]

S. Yun, J. Choi, Y. Yoo, K. Yun, and J. Y. Choi. 2017. Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 1349--1358.

[37]

M. Zhang, J. Feng, and W. Hu. 2017. Robust Visual Object Tracking with Top-down Reasoning. In Proc. of ACM on Multimedia Conference. 226--234.

Digital Library

[38]

T. Zhang, C. Xu, and M. H. Yang. 2017. Multi-task Correlation Particle Filter for Robust Object Tracking. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition. 4819--4827.

[39]

G. Zhu, F. Porikli, and H. Li. 2016. Beyond Local Search: Tracking Objects Everywhere with Instance-Specific Proposals. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition . 943--951.

[40]

G. Zhu, F. Porikli, and H. Li. 2016. Robust Visual Tracking with Deep Convolutional Neural Network Based Object Proposals on PETS. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1265--1272.

[41]

G. Zhu, J. Wang, Y. Wu, X. Zhang, and H. Lu. 2016. MC-HOG Correlation Tracking with Saliency Proposal. In Proc. of AAAI Conference on Artificial Intelligence. 3690--3696.

Digital Library

[42]

L. Zitnick and P. Dollár. 2014. Edge Boxes: Locating Object Proposals from Edges. In Proc. of European Conference on Computer Vision. 391--405.

Cited By

Liang YChen HWu QXia CLi J(2024)Joint Spatio-Temporal Similarity and Discrimination Learning for Visual TrackingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337737934:8(7284-7300)Online publication date: Aug-2024
https://doi.org/10.1109/TCSVT.2024.3377379
Wu QChan A(2024)Learning Tracking Representations from Single Point Annotations2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00267(2606-2615)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00267
Wan JWu QChan A(2023)Modeling Noisy Annotations for Point-Wise SupervisionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.329975345:12(15065-15080)Online publication date: Dec-2023
https://doi.org/10.1109/TPAMI.2023.3299753
Show More Cited By

Index Terms

Robust Correlation Filter Tracking with Shepherded Instance-Aware Proposals
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Tracking

Recommendations

Robust visual tracking via co-trained Kernelized correlation filters

We train a pool of discriminative classifiers jointly in a closed-form fashion for visual tracking.We propose analytic model for datasets of thousands of translated patches.It is able to outperform the baseline by a larger margin. Recent advances in ...
Visual tracking tracker via object proposals and co-trained kernelized correlation filters
Abstract
Visual tracking is a challenging task in the field of computer vision with wide applications in intelligent and surveillance systems. Recently, correlation trackers have shown great achievement in visual tracking due to its high efficiency. ...
Kernel correlation filters for visual tracking with adaptive fusion of heterogeneous cues

Although the correlation filter-based trackers have achieved competitive results both on accuracy and robustness, the performance of trackers can still be improved because the most existing trackers either use a fixed scale or a sole filtering template ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '18: Proceedings of the 26th ACM international conference on Multimedia

October 2018

2167 pages

ISBN:9781450356657

DOI:10.1145/3240508

General Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Kyoung Mu Lee
Seoul National University, Korea
,
Jiebo Luo
University of Rochester, USA
,
Wenwu Zhu
Tsinghua University, China
,
Program Chairs:
Hyeran Byun
Yonsei University, Korea
,
Chang Wen Chen
State Univ. Of New York at Buffalo, USA
,
Rainer Lienhart
University of Augsburg, Germany
,
Tao Mei
JD AI, China

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key Research and Development Program of China
National Natural Science Foundation of China

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22 - 26, 2018

Seoul, Republic of Korea

Acceptance Rates

MM '18 Paper Acceptance Rate 209 of 757 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
253
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liang YChen HWu QXia CLi J(2024)Joint Spatio-Temporal Similarity and Discrimination Learning for Visual TrackingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337737934:8(7284-7300)Online publication date: Aug-2024
https://doi.org/10.1109/TCSVT.2024.3377379
Wu QChan A(2024)Learning Tracking Representations from Single Point Annotations2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00267(2606-2615)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00267
Wan JWu QChan A(2023)Modeling Noisy Annotations for Point-Wise SupervisionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.329975345:12(15065-15080)Online publication date: Dec-2023
https://doi.org/10.1109/TPAMI.2023.3299753
Wu QYang TLiu ZWu BShan YChan A(2023)DropMAE: Masked Autoencoders with Spatial-Attention Dropout for Tracking Tasks2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01399(14561-14571)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01399
Liang YWu QLiu YYan YWang H(2022)Deep Correlation Filter Tracking With Shepherded Instance-Aware ProposalsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2021.310360123:8(11408-11421)Online publication date: Aug-2022
https://doi.org/10.1109/TITS.2021.3103601
Wang YXiong LDu KYan YWang HChua TWang JTian QGurrin CJia JZhang HSun Q(2021)Robust visual tracking via scale-aware localization and peak response strengthProceedings of the 2nd ACM International Conference on Multimedia in Asia10.1145/3444685.3446274(1-7)Online publication date: 7-Mar-2021
https://dl.acm.org/doi/10.1145/3444685.3446274
Sio CMa YShuai HChen JCheng WWen Chen CCucchiara RHua XQi GRicci EZhang ZZimmermann R(2020)S2SiamFCProceedings of the 28th ACM International Conference on Multimedia10.1145/3394171.3413611(1948-1957)Online publication date: 12-Oct-2020
https://dl.acm.org/doi/10.1145/3394171.3413611
Chen HWu QLiang YGao XWang HAmsaleg LHuet BLarson MGravier GHung HNgo CTsang Ooi W(2019)Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object TrackingProceedings of the 27th ACM International Conference on Multimedia10.1145/3343031.3350975(473-481)Online publication date: 15-Oct-2019
https://dl.acm.org/doi/10.1145/3343031.3350975
Xiong LLiang YYan YWang H(2019)Correlation Filter Tracking with Adaptive Proposal Selection for Accurate Scale Estimation2019 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME.2019.00312(1816-1821)Online publication date: Jul-2019
https://doi.org/10.1109/ICME.2019.00312

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten