skip to main content
10.1145/2435264.2435344acmconferencesArticle/Chapter ViewAbstractPublication PagesfpgaConference Proceedingsconference-collections
poster

A high-performance, low-energy FPGA accelerator for correntropy-based feature tracking (abstract only)

Published: 11 February 2013 Publication History

Abstract

Computer-vision and signal-processing applications often require feature tracking to identify and track the motion of different objects (features) across a sequence of images. Numerous algorithms have been proposed, but common measures of similarity for real-time usage are either based on correlation, mean-squared error, or sum of absolute differences, which are not robust enough for safety-critical applications. To improve robustness, a recent feature-tracking algorithm called C-Flow uses correntropy from Information Theoretic Learning to significantly improve signal-to-noise ratio. In this paper, we present an FPGA accelerator for C-Flow that is typically 3.6-8.5x faster than a GPU and show that the FPGA is the only device capable of real-time usage for large features. Furthermore, we show the FPGA accelerator is more appropriate for embedded usage, with energy consumption that is 2.5-22x less than the GPU.

References

[1]
S. Asano, T. Maruyama, and Y. Yamaguchi. Performance comparison of fpga, gpu and cpu in image processing. In Field Programmable Logic and Applications, 2009. FPL 2009. International Conference on, pages 126--131, 31 2009-sept. 2 2009.
[2]
Z. Baker, M. Gokhale, and J. Tripp. Matched filter computation on fpga, cell and gpu. In Field-Programmable Custom Computing Machines, 2007. FCCM 2007. 15th Annual IEEE Symposium on, pages 207--218, april 2007.
[3]
F. Barranco, M. Tomasi, J. Diaz, M. Vanegas, and E. Ros. Parallel architecture for hierarchical optical flow estimation based on fpga. Very Large Scale Integration (VLSI) Systems, IEEE Transactions on, 20(6):1058--1067, june 2012.
[4]
S. S. Beauchemin and J. L. Barron. The computation of optical flow. ACM Comput. Surv., 27(3):433--466, Sept. 1995.
[5]
S. Birchfield. "KLT: An Implementation of the Kanade-Lucas-Tomasi Feature Tracker." Available at http://www.ces.clemson.edu/~stb/klt/.
[6]
C. Braillon, C. Pradalier, J. Crowley, and C. Laugier. Real-time moving obstacle detection using optical flow models. In Intelligent Vehicles Symposium, 2006 IEEE, pages 466--471, 0-0 2006.
[7]
T. Camus. Real-time quantized optical flow. In Computer Architectures for Machine Perception, 1995. Proceedings. CAMP '95, pages 126 --131, sep 1995.
[8]
Z. Chai and J. Shi. Improving klt in embedded systems by processing oversampling video sequence in real-time. In Reconfigurable Computing and FPGAs (ReConFig), 2011 International Conference on, pages 297 --302, 30 2011-dec. 2 2011.
[9]
J. Chase, B. Nelson, J. Bodily, Z. Wei, and D.-J. Lee. Real-time optical flow calculations on fpga and gpu architectures: A comparison study. In Field-Programmable Custom Computing Machines, 2008. FCCM '08. 16th International Symposium on, pages 173 --182, april 2008.
[10]
S. Che, J. Li, J. Sheaffer, K. Skadron, and J. Lach. Accelerating compute-intensive applications with gpus and fpgas. In Application Specific Processors, 2008. SASP 2008. Symposium on, pages 101 --107, june 2008.
[11]
B. Cope, P. Cheung, W. Luk, and S. Witt. Have gpus made fpgas redundant in the field of video processing? In Field-Programmable Technology, 2005. Proceedings. 2005 IEEE International Conference on, pages 111 --118, dec. 2005.
[12]
D. Crookes, K. Benkrid, A. Bouridane, K. Alotaibi, and A. Benkrid. Design and implementation of a high level programming environment for fpga-based image processing. Vision, Image and Signal Processing, IEEE Proceedings -, 147(4):377 --384, aug 2000.
[13]
D. Demigny, L. Kessal, R. Bourguiba, and N. Boudouani. How to use high speed reconfigurable fpga for real time image processing? In Computer Architectures for Machine Perception, 2000. Proceedings. Fifth IEEE International Workshop on, pages 240 --246, 2000.
[14]
Y. Dong, Y. Dou, and J. Zhou. Optimized generation of memory structure in compiling window operations onto reconfigurable hardware. In ARC, pages 110--121, 2007.
[15]
J. Fowers, G. Brown, P. Cooke, and G. Stitt. A performance and energy comparison of FPGAs, GPUs, and multicores for sliding-window applications. In FPGA '12: Proceedings of the ACM/SIGDA international symposium on Field Programmable Gate Arrays, FPGA'12, pages 47--56, New York, NY, USA, February 2012. ACM.
[16]
M. Harris. 2007. "Optimizing Parallel Reduction in CUDA," NVIDIA Developer Technology.
[17]
K. Hild, D. Erdogmus, K. Torkkola, and J. Principe. Feature extraction using information theoretic learning. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 28(9):1385 --1392, sept. 2006.
[18]
B. K. Horn and B. G. Schunck. Determining optical flow. Artificial Intelligence, 17(1--3):185 -- 203, 1981.
[19]
P. Huijse, P. Estevez, P. Protopapas, P. Zegers, and J. Principe. An information theoretic algorithm for finding periodicities in stellar light curves. Signal Processing, IEEE Transactions on, PP(99):1, 2012.
[20]
L. Hunt. Fault-aware machine vision in small unmanned systems. In In FCRAR'09: Proceedomgs of the Florida Conference on Recent Advances in Robotics, 2009.
[21]
W. L. Hunt, Jr., "Fault-Aware Matched Filter and Optical Flow," U.S. Patent Application US 2011/0116584, May 19, 2011.
[22]
J.-S. Kim, M. Hwangbo, and T. Kanade. Realtime affine-photometric klt feature tracker on gpu in cuda framework. In Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on, pages 886 --893, 27 2009-oct. 4 2009.
[23]
W. Liu, P. Pokharel, and J. Principe. Correntropy: A localized similarity measure. In Neural Networks, 2006. IJCNN '06. International Joint Conference on, pages 4919 --4924, 0-0 2006.
[24]
W. Liu, P. Pokharel, and J. Principe. Correntropy: Properties and applications in non-gaussian signal processing. Signal Processing, IEEE Transactions on, 55(11):5286 --5298, nov. 2007.
[25]
B. D. Lucas and T. Kanade. An Iterative Image Registration Technique with an Application to Stereo Vision. In IJCAI81, pages 674--679, 1981.
[26]
S. McBader and P. Lee. An fpga implementation of a flexible, parallel image processing architecture suitable for embedded vision systems. In Parallel and Distributed Processing Symposium, 2003. Proceedings. International, page 5 pp., april 2003.
[27]
N.Moore, M. Leeser, and L. S. King. Adaptable two-dimension sliding windows on nvidia gpus with runtime compilation. In Proceedings of the 2011 Symposium on Application Accelerators in High-Performance Computing, SAAHPC '11, pages 103--112, Washington, DC, USA, 2011. IEEE Computer Society.
[28]
J. Ohmer and N. Redding. Gpu-accelerated klt tracking with monte-carlo-based feature reselection. In Computing: Techniques and Applications, 2008. DICTA '08.Digital Image, pages 234 --241, dec. 2008.
[29]
K. Pauwels, M. Tomasi, J. Diaz Alonso, E. Ros, and M. Van Hulle. A comparison of fpga and gpu for real-time phase-based optical flow, stereo, and local image features. Computers, IEEE Transactions on, 61(7):999 --1012, july 2012.
[30]
Principe, J., Fisher III, J., Xu, D. 2000. Information theoretic learning. In S. Haykin (Ed.), Unsupervised adaptive filtering. New York, NY: Wiley.
[31]
The Robotics Institute, Carnegie Mellon. 2012. VIVID Tracking Evaluation Web Site. http://vision.cse.psu.edu/data/vividEval/datasets/datasets.html
[32]
J. Shi and C. Tomasi. Good features to track. In Computer Vision and Pattern Recognition, 1994. Proceedings CVPR '94., 1994 IEEE Computer Society Conference on, pages 593 --600, jun 1994.
[33]
S. Silva, S. Cunha, A. Matos, and N. Cruz. An autonomous boat based synthetic aperture sonar. In OCEANS 2007, pages 1 --7, 29 2007-oct. 4 2007.
[34]
S. Sinha, J.-M. Frahm, M. Pollefeys, and Y. Genc. Feature tracking and matching in video using programmable graphics hardware. Machine Vision and Applications, 22:207--217, 2011. 10.1007/s00138-007-0105-z.
[35]
C. Tomasi and T. Kanade. Detection and tracking of point features. Technical report, International Journal of Computer Vision, 1991.
[36]
K. Underwood and K. Hemmert. Closing the gap: Cpu and fpga trends in sustainable floatingpoint blas performance. In Field-Programmable Custom Computing Machines, 2004. FCCM 2004. 12th Annual IEEE Symposium on, pages 219 -- 228, april 2004.
[37]
I. Uzun, A. Amira, and A. Bouridane. Fpga implementations of fast fourier transforms for real-time signal and image processing. Vision, Image and Signal Processing, IEE Proceedings, 152(3):283 -- 296, june 2005.
[38]
Z. Wei, D.-J. Lee, and B. E. Nelson. Fpga-based real-time optical flow algorithm design and implementation. Journal of Multimedia, 2(5), 2007.
[39]
J.-W. Xu, H. Bakardjian, A. Cichocki, and J. C. Principe. A new nonlinear similarity measure for multichannel signals. Neural Networks, 21(2--3):222 -- 231, 2008.
[40]
J.-W. Xu and J. Principe. A pitch detector based on a generalized correlation function. Audio, Speech, and Language Processing, IEEE Transactions on, 16(8):1420 --1432, nov. 2008.
[41]
H. Yu and M. Leeser. Automatic sliding window operation optimization for fpga-based. In Field-Programmable Custom Computing Machines, 2006. FCCM '06. 14th Annual IEEE Symposium on, pages 76 --88, april 2006.
[42]
X.-T. Yuan and B.-G. Hu. Robust feature extraction via information theoretic learning. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, pages 1193--1200, New York, NY, USA, 2009. ACM.
[43]
C. Zach, D. Gallup, and J.-M. Frahm. Fast gain-adaptive klt tracking on the gpu. In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW '08. IEEE Computer Society Conference on, pages 1 --7, june 2008.
[44]
J. S. Zelek. Towards bayesian real-time optical flow. Image and Vision Computing, 22(12):1051 -- 1069, 2004.

Cited By

View all
  • (2017)A system on chip-based real-time tracking system for amphibious spherical robotsInternational Journal of Advanced Robotic Systems10.1177/172988141771655914:4(172988141771655)Online publication date: 7-Jul-2017

Index Terms

  1. A high-performance, low-energy FPGA accelerator for correntropy-based feature tracking (abstract only)

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      FPGA '13: Proceedings of the ACM/SIGDA international symposium on Field programmable gate arrays
      February 2013
      294 pages
      ISBN:9781450318877
      DOI:10.1145/2435264

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 11 February 2013

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. FPGA
      2. GPU
      3. correntropy
      4. feature tracking
      5. optical flow

      Qualifiers

      • Poster

      Conference

      FPGA '13
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 125 of 627 submissions, 20%

      Upcoming Conference

      FPGA '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)0
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 18 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2017)A system on chip-based real-time tracking system for amphibious spherical robotsInternational Journal of Advanced Robotic Systems10.1177/172988141771655914:4(172988141771655)Online publication date: 7-Jul-2017

      View Options

      View options

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media