Skip to main content
Springer Nature Link
Log in

A fully pipelined and parallel hardware architecture for real-time BRISK salient point extraction

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Scale and rotation invariant salient point detection and matching algorithms are variously used in computer vision applications such as image matching, 3D localization and pose estimation. Recently, hardware implementation of image and video processing algorithms has emerged as a viable solution to handle the high computational complexity of applications like 3D pose estimation with several processing stages. The hardware implementation of various stages of theses algorithms can be executed in a pipelined manner to ensure the reality of time. In this paper, a new and fully pipelined hardware architecture is proposed for salient point detection using Binary Robust Invariant Scalable Keypoints (BRISK) algorithm. BRISK algorithm is a binary keypoint extractor that detects salient points by constructing a scale-space pyramid; therefore, its fixed-point hardware implementation in a pipelined manner is challenging because of the required synchronization for various layers in scale domain. The proposed hardware architecture was implemented using Verilog Hardware Description Language, and the functionality of the design was validated through several experiments. The proposed design was synthesized by using an ASIC digital design flow utilizing 180 nm CMOS technology as well as a Virtex-4 FPGA. The design is clocked at 90.91 MHz in ASIC implementation and achieves processing rate of 169.29 frames/s while running on input images with 800 × 600 resolution. The throughput of FPGA implementation is 180.44 frames/s with 96.89 MHz clock frequency for the same input image resolution. Experimental results confirm the efficiency of the proposed hardware architecture in comparison with software implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of the Fourth Alvey Vision Conference 1988, pp. 147–151

  2. Moravec, H.P.: Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover. Stanford University, Stanford (1980)

    Google Scholar 

  3. Shi, J., Tomasi, C.: Good features to track. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 593–600. IEEE (1994)

  4. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  5. Yao, L., Feng, H., Zhu, Y., Jiang, Z., Zhao, D., Feng, W.: An architecture of optimised SIFT feature detection for an FPGA implementation of an image matcher. In: International Conference on Field-Programmable Technology (FPT), pp. 30–37. IEEE (2009)

  6. Grabner, M., Grabner, H., Bischof, H.: Fast approximated SIFT. In: 7th Asian Conference on Computer Vision (ACCV), Hyderabad, India, pp. 918–927. Springer, Berlin (2006)

    Chapter  Google Scholar 

  7. Ke, Y., Sukthankar, R.: PCA-SIFT: A more distinctive representation for local image descriptors. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, Vol. 502, pp. II-506–II-513. IEEE (2004)

  8. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust features. In: 9th European Conference on Computer vision (ECCV), Graz, Austria, pp. 404–417. Springer, Berlin (2006)

    Chapter  Google Scholar 

  9. Rosten, E., Drummond, T.: Fusing points and lines for high performance tracking. In: Tenth IEEE International Conference on Computer Vision (ICCV) Beijing, China, pp. 1508–1515. IEEE (2005)

  10. Mair, E., Hager, G.D., Burschka, D., Suppa, M., Hirzinger, G.: Adaptive and generic corner detection based on the accelerated segment test. In: 11th European Conference on Computer Vision (ECCV), Heraklion, Crete, Greece, pp. 183–196. Springer, Berlin (2010)

    Chapter  Google Scholar 

  11. Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: Binary robust invariant scalable keypoints. In: IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, pp. 2548–2555. IEEE (2011)

  12. Pedre, S., Krajník, T., Todorovich, E., Borensztejn, P.: Accelerating embedded image processing for real time: a case study. J. Real-Time Image Proc. 11(2), 349–374 (2016)

    Article  Google Scholar 

  13. Velez, G., Cortés, A., Nieto, M., Vélez, I., Otaegui, O.: A reconfigurable embedded vision system for advanced driver assistance. J. Real-Time Image Proc. 10(4), 725–739 (2015)

    Article  Google Scholar 

  14. Wang, W., Yan, J., Xu, N., Wang, Y., Hsu, F.-H.: Real-time high-quality stereo vision system in FPGA. IEEE Trans. Circuits Syst. Video Technol. 25(10), 1696–1708 (2015)

    Article  Google Scholar 

  15. Sivanantham, S., Paul, N.N., Iyer, R.S.: Object tracking algorithm implementation for security applications. Far East J. Electron. Commun. 16(1), 1–13 (2016)

    Article  Google Scholar 

  16. Azizabadi, M., Behrad, A., Ghaznavi-Ghoushchi, M.: VLSI implementation of star detection and centroid calculation algorithms for star tracking applications. J. Real-Time Image Proc. 9(1), 127–140 (2014)

    Article  Google Scholar 

  17. Araneda, L., Figueroa, M.: A compact hardware architecture for digital image stabilization using integral projections. Microprocess. Microsyst. 39(8), 987–997 (2015)

    Article  Google Scholar 

  18. Bonato, V., Marques, E., Constantinides, G.: A parallel hardware architecture for scale and rotation invariant feature detection. IEEE Trans. Circuits Syst. Video Technol. 18(12), 1703–1712 (2008)

    Article  Google Scholar 

  19. Huang, F.-C., Huang, S.-Y., Ker, J.-W., Chen, Y.-C.: High-performance SIFT hardware accelerator for real-time image feature extraction. IEEE Trans. Circuits Syst. Video Technol. 22(3), 340–351 (2012)

    Article  Google Scholar 

  20. Suzuki, T., Ikenaga, T.: SIFT-based low complexity keypoint extraction and its real-time hardware implementation for full-HD video. In: 2012 Asia-Pacific Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC), California, USA, pp. 1–6. IEEE (2012)

  21. Chang, L., Hernández-Palancar, J., Sucar, L.E., Arias-Estrada, M.: FPGA-based detection of SIFT interest keypoints. Mach. Vis. Appl. 24(2), 371–392 (2013)

    Article  Google Scholar 

  22. Zhong, S., Wang, J., Yan, L., Kang, L., Cao, Z.: A real-time embedded architecture for SIFT. J. Syst. Architect. 59(1), 16–29 (2013)

    Article  Google Scholar 

  23. Chiu, L.-C., Chang, T.-S., Chen, J.-Y., Chang, N.Y.-C.: Fast SIFT design for real-time visual feature extraction. IEEE Trans. Image Process. 22(8), 3158–3167 (2013)

    Article  Google Scholar 

  24. Jiang, J., Li, X., Zhang, G.: SIFT hardware implementation for real-time image feature extraction. IEEE Trans. Circuits Syst. Video Technol. 24(7), 1209–1220 (2014)

    Article  MathSciNet  Google Scholar 

  25. Sledevič, T., Serackis, A.: SURF algorithm implementation on FPGA. In: 13th Biennial Baltic Electronics Conference (BEC), Tallinn, Estonia, pp. 291–294. IEEE (2012)

  26. Wilson, C., Zicari, P., Craciun, S., Gauvin, P., Carlisle, E., George, A., Lam, H.: A power-efficient real-time architecture for SURF feature extraction. In: International Conference on ReConFigurable Computing and FPGAs (ReConFig), pp. 1–8. IEEE (2014)

  27. Zhang, W., Liu, L., Yin, S., Zhou, R., Cai, S., Wei, S.: An efficient VLSI architecture of speeded-up robust feature extraction for high resolution and high frame rate video. Sci. China Inf. Sci. 56(7), 1–14 (2013)

    MathSciNet  Google Scholar 

  28. Battezzati, N., Colazzo, S., Maffione, M., Senepa, L.: SURF algorithm in FPGA: a novel architecture for high demanding industrial applications. In: Design, Automation & Test in Europe Conference & Exhibition (DATE), Dresden, pp. 161–162 (2012)

  29. Krajník, T., Šváb, J., Pedre, S., ížek P, P., Peuil, L.: FPGA-based module for SURF extraction. Mach. Vis. Appl. 25(3), 787–800 (2014)

    Article  Google Scholar 

  30. Heo, H., Lee, J.-Y., Lee, K.-Y., Lee, C.-H.: FPGA based implementation of FAST and BRIEF algorithm for object recognition. In: TENCON 2013–2013 IEEE Region 10 Conference (31194), pp. 1–4. IEEE (2013)

  31. Park, J.-S., Kim, L.-S.: Hardware accelerator for feature point detection and matching. In: Jaeseok Kim, H.S. (ed.) Algorithm & SoC Design for Automotive Vision Systems, pp. 197–230. Springer, New York (2014)

    Google Scholar 

  32. Heinly, J., Dunn, E., Frahm, J.-M.: Comparative evaluation of binary features. In: Computer Vision–ECCV 2012, pp. 759–773. Springer, NewYork (2012)

    Chapter  Google Scholar 

  33. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571. IEEE (2011)

  34. Bekele, D., Teutsch, M., Schuchert, T.: Evaluation of binary keypoint descriptors. In: 20th IEEE International Conference on Image Processing (ICIP), pp. 3652–3656. IEEE (2013)

  35. Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: Fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 510–517. IEEE (2012)

  36. Canclini, A., Cesana, M., Redondi, A., Tagliasacchi, M., Ascenso, J., Cilla, R.: Evaluation of low-complexity visual feature detectors and descriptors. In: 18th International Conference on Digital Signal Processing (DSP), pp. 1–7. IEEE (2013)

  37. Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)

    Google Scholar 

  38. Oxford Image Dataset. http://www.robots.ox.ac.uk/~vgg/data/data-aff.html. Accessed Feb 2017

  39. Bouris, D., Nikitakis, A., Papaefstathiou, I.: Fast and efficient FPGA-based feature detection employing the SURF algorithm. In: 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), pp. 3–10. IEEE (2010)

  40. Schaeferling, M., Kiefer, G.: Flex-SURF: A flexible architecture for FPGA-based robust feature extraction for optical tracking systems. In: 2010 International Conference on Reconfigurable Computing and FPGAs (ReConFig), pp. 458–463. IEEE (2010)

  41. Lee, S.-S., Jang, S.-J., Kim, J., Hwang, Y., Choi, B.: Memory-efficient SURF architecture for ASIC implementation. Electron. Lett. 50(15), 1058–1059 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Ehsan Azimi

  2. Electrical Engineering Department, Shahed University, Tehran, Iran

    Alireza Behrad & Mohammad Bagher Ghaznavi-Ghoushchi

  3. Department of Computer Engineering, Kharazmi University, Tehran, Iran

    Jamshid Shanbehzadeh

Authors
  1. Ehsan Azimi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Alireza Behrad
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Mohammad Bagher Ghaznavi-Ghoushchi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jamshid Shanbehzadeh
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Alireza Behrad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azimi, E., Behrad, A., Ghaznavi-Ghoushchi, M.B. et al. A fully pipelined and parallel hardware architecture for real-time BRISK salient point extraction. J Real-Time Image Proc 16, 1859–1879 (2019). https://doi.org/10.1007/s11554-017-0693-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-017-0693-4

Keywords