Elsevier

Pervasive and Mobile Computing

Volume 35, February 2017, Pages 108-124
Pervasive and Mobile Computing

Kernelized correlation tracker on smartphones

https://doi.org/10.1016/j.pmcj.2016.06.013Get rights and content

Abstract

This paper shows the implementation of a KC tracker (high-speed kernelized correlation tracker) on an Android smartphone. The image processing part is implemented with the Android-NDK in C/C++. Some parts of the tracking algorithm, which can be parallelized very well, are partitioned and calculated on the GPU with OpenGL ES and OpenCL. Other parts, such as the Discrete Fourier Transform (DFT), are calculated on the CPU (partly with the ARM-NEON features). With these hardware acceleration steps we could reach real-time performance (at least 20–30 FPS) on up-to-date smartphones, such as Samsung Galaxy S4, S5 or Google Nexus 5.

Beyond that, we present some new color features and compare their tracking quality to the HOG features using the KC tracker and show that their tracking quality is mostly superior compared to the HOG features.

If an object gets lost by the tracker which is the case e.g. if the object is totally hidden or outside the viewing range, there should be a possibility to perform a re-detection. In this paper, we show a basic approach to determine the tracking quality and search for the tracking object in the entire images of the subsequent video-frames.

Section snippets

Tracker benchmark

Ways of measuring the tracking performance are explained in  [3], which describes three ways of measuring the tracking accuracy. One method is the OPE (One-Pass Evaluation), which is used in our work to compare tracking performances. This type of precision plot measures the average precision for a video sequence from the beginning to the end. With it, the percentages of frames of a sequence are measured, which do not exceed certain error thresholds. The location error threshold is the Euclidean

Color features

In this paper, we took a closer look into two sorts of color features. With both variants of color features, the image is first divided in square-sized cells and for each cell a histogram is calculated, similar to the HOG features.

Implementation

The KC tracking algorithm is implemented for the Android OS. The KC algorithm and the features (HOG, color) are implemented as a shared library in C/C++ with the NDK for Android. That way, APIs like OpenGL ES and OpenCL and other libraries, that e.g. SoC manufacturers offer, can be used very easily.

Performance comparison of color features to HOG features with MATLAB

Both color feature variants were implemented for MATLAB as built-in C functions, but without the use of vector registers (e.g. SSE) for even shorter runtimes. As the color features were implemented for MATLAB, one can also use the MATLAB implementation of the KC tracker with these features. This makes it also possible to compare the tracking performance using these color features with the HOG features.

The tracking performance (see Section  1.1) with the first variant of color features was

Conclusion

In this paper we showed that the KC tracker, which reaches a very good precision, can be implemented and run on smartphones with at least 20–30 FPS, if the available hardware (CPU, GPU, …) is used appropriately. Furthermore, we presented two approaches to color features, where the second variant reaches a very good performance with the used tracker, which is at least as good or even better as with HOG features. The combination of HOG and color features does significantly improve the precision

Future work

GPUs in smartphones are getting more powerful with every generation. Therefore, a fast GPU implementation for FFT transforms with any sample sizes could accelerate the tracking frame rates significantly. With the new Android camera API for image processing in Android 5.0, maybe higher camera frame rates than 30 FPS could be used in Android applications. Currently, high camera frame rates are only available for video captures (up to 120 FPS) but not within apps. To improve the detection of a

Acknowledgment

The authors would like to thank Prof. Dr. Franz Kurfess for his helpful comments/proofreading.

References (15)

  • J.F. Henriques et al.

    Exploiting the circulant structure of tracking-by-detection with kernels

  • J.F. Henriques et al.

    High-speed tracking with kernelized correlation filters

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2015)
  • Y. Wu et al.

    Online object tracking: A benchmark

  • S. Hare et al.

    Struck: Structured output tracking with Kernels

  • N. Dalal et al.

    Histograms of oriented gradients for human detection

  • P.F. Felzenszwalb et al.

    Object detection with discriminatively trained part based models

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2010)
  • R.B. Girshick

    From rigid templates to grammars: Object detection with structured models

    (2012)
There are more references available in the full text version of this article.

Cited by (0)

View full text