Skip to main content
SpringerLink
Log in

FPGA-based architecture of a real-time SIFT matcher and RANSAC algorithm for robotic vision applications

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A fundamental problem in computer vision is finding correspondences between features in pairs of similar images. By comparing feature descriptors instead of pixel intensities, the matching capability is significantly increased. Keypoints extracted by Scale-Invariant Feature Transform (SIFT) provide superior matching ability, however, a small proportion of false corresponcences is always inevitable. The exemption of false matches is achieved using robust fitting algorithms, with RANSAC (random sample consensus) being a popular one. SIFT and RANSAC are computationally demanding and time consuming algorithms. When the target application operates in real-time, conventional approaches based on personal computers usually fail to meet the requirements. In this paper, an FPGA-based architecture for real-time SIFT matching and RANSAC algorithm is presented. The proposed scheme is applied to identify the correspondences between point features across consecutive video frames and reject the false matches. The architecture is verified using the DE2i-150 development board. Using Cyclone IV technology, the system supports a processing rate of 40fps for VGA resolution and therefore meets real-time requirements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Alahi A, Ortiz R, Vandergheynst P (2012) FREAK: Fast Retina Keypoint. In: 2012 I.E. Conf. Comput. Vis. Pattern Recognit. IEEE, pp 510–517

  2. Bay H, Tuytelaars T, Van Gool L (2006) SURF: Speeded Up Robust Features. In: Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). Berlin, Heidelberg, pp 404–417

  3. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-Up Robust Features (SURF). Comput Vis Image Underst 110:346–359. doi:10.1016/j.cviu.2007.09.014

    Article  Google Scholar 

  4. Boulekchour M, Aouf N, Richardson M (2014) Robust L∞ convex optimisation for monocular visual odometry trajectory estimation. Robotica:1–20. doi:10.1017/S0263574714001829

  5. Calonder M, Lepetit V, Ozuysal M et al (2011) BRIEF: Computing a Local Binary Descriptor very Fast. IEEE Trans Pattern Anal Mach Intell 34:1281–1298. doi:10.1109/TPAMI.2011.222

    Article  Google Scholar 

  6. Condello G, Pasteris P, Pau D, Sami M (2013) An OpenCL-based feature matcher. Signal Process Image Commun 28:345–350. doi:10.1016/j.image.2012.06.002

    Article  Google Scholar 

  7. Di Carlo S, Gambardella G, Prinetto P et al (2015) SA-FEMIP: A Self-Adaptive Features Extractor and Matcher IP-Core Based on Partially Reconfigurable FPGAs for Space Applications. IEEE Trans Very Large Scale Integr Syst 23:2198–2208. doi:10.1109/TVLSI.2014.2357181

    Article  Google Scholar 

  8. Dohi K, Hatanaka Y, Negi K, et al (2012) Deep-pipelined FPGA implementation of ellipse estimation for eye tracking. In: 22nd Int. Conf. F. Program. Log. Appl. IEEE, pp 458–463

  9. Dung L-R, Huang C-M, Wu Y-Y (2013) Implementation of RANSAC Algorithm for Feature-Based Image Registration. J Comput Commun 1:46–50. doi:10.4236/jcc.2013.16009

    Article  Google Scholar 

  10. Fassold H, Rosner J (2015) A real-time GPU implementation of the SIFT algorithm for large-scale video analysis tasks. Proc SPIE - Int Soc Opt Eng. doi:10.1117/12.2083201

  11. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24:381–395. doi:10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  12. Fürntratt H, Rosner J, Stiegler H, Fassold H (2013) GPU-accelerated SIFT descriptor matching. GPU Technol. Conf

  13. Gentle JE (2007) Matrix Transformations and Factorizations. Matrix Algebr Theory, Comput Appl Stat. doi:10.1007/978-0-387-70873-7

  14. Haiyang L, Hongzhou H, Yongge W (2013) A Fast Image Matching Algorithm Based on GPU Parallel Computing. Inf Technol J 12:1449–1453. doi:10.3923/itj.2013.1449.1453

    Article  Google Scholar 

  15. Hartley R, Zisserman A (2004) Estimation - 2D Projective Transformations. In: Mult. View Geom. Comput. Vis. Cambridge University Press, pp 87–127

  16. Hidalgo-Paniagua A, Vega-Rodríguez MA, Pavón N, Ferruz J (2014) A Comparative Study of Parallel RANSAC Implementations in 3D Space. Int J Parallel Prog 43:703–720. doi:10.1007/s10766-014-0316-7

    Article  Google Scholar 

  17. Jiang J, Li X, Zhang G (2014) SIFT Hardware Implementation for Real-Time Image Feature Extraction. IEEE Trans Circuits Syst Video Technol 24:1209–1220. doi:10.1109/TCSVT.2014.2302535

    Article  Google Scholar 

  18. Kapela R, Gugala K, Sniatala P et al (2015) Embedded platform for local image descriptor based object detection. Appl Math Comput 267:419–426. doi:10.1016/j.amc.2015.02.029

    MathSciNet  Google Scholar 

  19. Ke Y, Sukthankar R (2004) PCA-SIFT: A more distinctive representation for local image descriptors. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2

  20. Leutenegger S, Chli M, Siegwart RY (2011) BRISK: Binary Robust invariant scalable keypoints. In: 2011 Int. Conf. Comput. Vis. IEEE, Barcelona; Spain, pp 2548–2555

  21. Liu H, Shen H (2014) Application of improved SIFT algorithm on stitching of UAV remote sensing image. Bandaoti Guangdian/Semiconductor Optoelectron 35:108–112

    Google Scholar 

  22. Lowe DG (1999) Object recognition from local scale-invariant features. In: Proc. IEEE Int. Conf. Comput. Vis. IEEE, Kerkyra, Greece, pp 1150–1157

  23. Lowe DG (2004) Distinctive Image Features from Scale-Invariant Keypoints. Int J Comput Vis 60:91–110. doi:10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  24. Mikolajczyk K, Schmid C (2005) Performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27:1615–1630. doi:10.1109/TPAMI.2005.188

    Article  Google Scholar 

  25. Scharstein D, Szeliski R (2003) High-accuracy stereo depth maps using structured light. Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 1

  26. Song H, Xiao H, He W, et al (2013) A fast stereovision measurement algorithm based on SIFT keypoints for mobile robot. In: 2013 I.E. Int. Conf. Mechatronics Autom. IEEE, Takamastu, Japan, pp 1743–1748

  27. Tang JW, Shaikh-Husin N, Sheikh UU (2013) FPGA implementation of RANSAC algorithm for real-time image geometry estimation. In: 2013 I.E. Student Conf. Res. Dev. IEEE, pp 290–294

  28. Vourvoulakis J, Kalomiros J, Lygouras J (2016) Fully pipelined FPGA-based architecture for real-time SIFT extraction. Microprocess Microsyst 40:53–73. doi:10.1016/j.micpro.2015.11.013

    Article  Google Scholar 

  29. Vourvoulakis J, Lygouras J, Kalomiros J (2016) Acceleration of RANSAC algorithm for images with affine transformation. In: 2016 I.E. Int. Conf. Imaging Syst. Tech. IEEE, pp 60–65

  30. Vourvoulakis J, Kalomiros J, Lygouras J (2017) FPGA accelerator for real-time SIFT matching with RANSAC support. Microprocess Microsyst 49:105–116. doi:10.1016/j.micpro.2016.11.011

    Article  Google Scholar 

  31. Wang J, Zhong S, Yan L, Cao Z (2014) An Embedded System-on-Chip Architecture for Real-time Visual Detection and Matching. IEEE Trans Circuits Syst Video Technol 24:525–538. doi:10.1109/TCSVT.2013.2280040

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Section of Electronics & Information Systems Technology, Department of Electrical & Computer Engineering, Polytechnic school of Xanthi, Democritus University of Thrace, 67100, Xanthi, Greece

    John Vourvoulakis & John Lygouras

  2. Department of Informatics Engineering, Technological and Educational Institute of Central Macedonia, Terma Magnisias, 62124, Serres, Greece

    John Kalomiros

Authors
  1. John Vourvoulakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Kalomiros
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John Lygouras
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John Vourvoulakis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vourvoulakis, J., Kalomiros, J. & Lygouras, J. FPGA-based architecture of a real-time SIFT matcher and RANSAC algorithm for robotic vision applications. Multimed Tools Appl 77, 9393–9415 (2018). https://doi.org/10.1007/s11042-017-5042-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5042-x

Keywords

Search

Navigation