Loading [MathJax]/extensions/TeX/ietmacros.js
A Complex-Former Tracker With Dynamic Polar Spatio-Temporal Encoding | IEEE Journals & Magazine | IEEE Xplore
Scheduled Maintenance: On Monday, 27 January, the IEEE Xplore Author Profile management portal will undergo scheduled maintenance from 9:00-11:00 AM ET (1400-1600 UTC). During this time, access to the portal will be unavailable. We apologize for any inconvenience.

A Complex-Former Tracker With Dynamic Polar Spatio-Temporal Encoding


Abstract:

Recently, the excellent performance of transformer has attracted the attention of the visual community. Visual transformer models usually reshape images into sequence for...Show More

Abstract:

Recently, the excellent performance of transformer has attracted the attention of the visual community. Visual transformer models usually reshape images into sequence format and encode them sequentially. However, it is difficult to explicitly represent the relative relationship in distance and direction of visual data with typical 2-D spatial structures. Also, the temporal motion properties of consecutive frames are hardly exploited when it comes to dynamic video tasks like tracking. Therefore, we propose a novel dynamic polar spatio-temporal encoding for video scenes. We use spiral functions in polar space to fully exploit the spatial dependences of distance and direction in real scenes. We then design a dynamic relative encoding mode for continuous frames to capture the continuous spatio-temporal motion characteristics among video frames. Finally, we construct a complex-former framework with the proposed encoding applied to video-tracking tasks, where the complex fusion mode (CFM) realizes the effective fusion of scenes and positions for consecutive frames. The theoretical analysis demonstrates the feasibility and effectiveness of our proposed method. The experimental results on multiple datasets validate that our method can improve tracker performance in various video scenarios.
Published in: IEEE Transactions on Neural Networks and Learning Systems ( Volume: 35, Issue: 12, December 2024)
Page(s): 17344 - 17358
Date of Publication: 21 August 2023

ISSN Information:

PubMed ID: 37603473

Funding Agency:


References

References is not available for this document.