Skip to main content
SpringerLink
Log in

FPGA implementation of HOOFR bucketing extractor-based real-time embedded SLAM applications

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Feature extraction is an important vision task in many applications like simultaneous localization and mapping (SLAM). In the recent computing systems, FPGA-based acceleration have presented a strong competition to GPU-based acceleration due to its high computation capabilities and lower energy consumption. In this paper, we present a high-level synthesis implementation on a SoC-FPGA of a feature extraction algorithm dedicated for SLAM applications. We choose HOOFR extraction algorithm which provides a robust performance but requires a significant computation on embedded CPU. Our system is dedicated for SLAM applications so that we also integrated bucketing detection method in order to have a homogeneous distribution of keypoints in the image. Moreover, instead of optimizing performance by simplifying the original algorithm as in many other researches, we respected the complexity of HOOFR extractor and have parallelized the processing operations. The design has been validated on an Intel Arria 10 SoC-FPGA with a throughput of 54 fps at \(1226 \times 370\) pixels (handling 1750 features) or 14 fps at \(1920 \times 1080\) pixels (handling 6929 features).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras (2016). arXiv preprint arXiv:1610.06475

  2. Mei, C., Sibley, G., Cummins, M., Newman, P.M., Reid, I.D.: A constant-time efficient stereo slam system. In: BMVC 2009, pp. 1–11 (2009)

  3. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  4. Bay, H., Tuytelaars, T., Van Gool, L., Surf: Speeded up robust features. In: Computer Vision–ECCV 2006. Springer, Berlin, pp. 404–417 (2006)

  5. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In 2011 IEEE International Conference on Computer Vision (ICCV). IEEE, pp. 2564–2571 (2011)

  6. Aldegheri, S., Bombieri, N., Bloisi, D.D., Farinelli, A.: Data flow orb-slam for real-time performance on embedded gpu boards. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5370–5375 (2019)

  7. Nguyen, D.-D., El Ouardi, A., Aldea, E., Bouaziz, S.: Hoofr: an enhanced bio-inspired feature extractor. In 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, pp. 2977–2982 (2016)

  8. Pereira, K., Athanas, P., Lin, H., Feng, W.: Spectral method characterization on FPGA and GPU accelerators. In: 2011 International Conference on Reconfigurable Computing and FPGAs (ReConFig). IEEE, pp. 487–492 (2011)

  9. Weber, R., Gothandaraman, A., Hinde, R.J., Peterson, G.D.: Comparing hardware accelerators in scientific applications: a case study. IEEE Trans. Parallel Distrib. Syst. 22(1), 58–68 (2011)

    Article  Google Scholar 

  10. De Schryver, C., Shcherbakov, I., Kienle, F., Wehn, N., Marxen, H., Kostiuk, A., Korn, R.: An energy efficient fpga accelerator for monte carlo option pricing with the Beston model. In: 2011 International Conference on Reconfigurable Computing and FPGAs (ReConFig). IEEE, pp. 468–474 (2011)

  11. Pauwels, K., Tomasi, M., Alonso, J.D., Ros, E., Van Hulle, M.M.: A comparison of fpga and gpu for real-time phase-based optical flow, stereo, and local image features. IEEE Trans. Comput 61(7), 999–1012 (2012)

    Article  MathSciNet  Google Scholar 

  12. Morales, V.M., Horrein, P.-H., Baghdadi, A., Hochapfel, E., Vaton, S.: Energy-efficient fpga implementation for binomial option pricing using openCL. In: Design, Automation and Test in Europe Conference and Exhibition (DATE), 2014. IEEE, pp. 1–6 (2014)

  13. Helali, A., Ameur, H., Górriz, J., Ramírez, J., Maaref, H.: Hardware implementation of real-time pedestrian detection system. Neural Comput. Appl. (2020). https://doi.org/10.1007/s00521-020-04731-y

    Article  Google Scholar 

  14. Mami, S., Lahbib, Y., Mami, A.: A new HLS allocation algorithm for efficient DSP utilization in FPGAs. J. Signal Process. Syst. 92, 153–171 (2019)

    Article  Google Scholar 

  15. Intel FPGA SDK for OpenCL Standard Edition: Programming Guide. INTEL (2014)

  16. Jelodari, P.T., Kordasiabi, M.P., Sheikhaei, S., Forouzandeh, B.: Fpga implementation of an adaptive window size image impulse noise suppression system. J. Real-Time Image Process. 16, 2015–2026 (2017)

    Article  Google Scholar 

  17. Marin, Y., Mitéran, J., Dubois, J., Heyrman, B., Ginhac, D.: An FPGA-based design for real-time super resolution reconstruction. In: Proceedings of the 12th International Conference on Distributed Smart Cameras, pp. 1–2 (2018)

  18. Yao, L., Feng, H., Zhu, Y., Jiang, Z., Zhao, D., Feng, W.: An architecture of optimised sift feature detection for an FPGA implementation of an image matcher. In: International Conference on Field-Programmable Technology, 2009. FPT 2009. IEEE, pp. 30–37 (2009)

  19. Bouris, D., Nikitakis, A., Papaefstathiou, I.: Fast and efficient FPGA-based feature detection employing the surf algorithm. In: 2010 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, pp. 3–10 (2010)

  20. Chiu, L.-C., Chang, T.-S., Chen, J.-Y., Chang, N.Y.-C.: Fast sift design for real-time visual feature extraction. IEEE Trans. Image Process. 22(8), 3158–3167 (2013)

    Article  Google Scholar 

  21. Lee, K.: A design of an optimized orb accelerator for real-time feature detection. Int. J. Control Autom. 7(3), 213–218 (2014)

    Article  Google Scholar 

  22. Weberruss, J., Kleeman, L., Boland, D., Drummond, T.: FPGA acceleration of multilevel orb feature extraction for computer vision. In: 2017 27th International Conference on Field Programmable Logic and Applications (FPL). IEEE, pp 1–8 (2017)

  23. Sun, R., Liu, P., Wang, J., Accetti, C., Naqvi, A.A.: A 42fps full-hd ORB feature extraction accelerator with reduced memory overhead. In: 2017 International Conference on Field Programmable Technology (ICFPT). IEEE, 2017, pp. 183–190 (2017)

  24. Pu, Y., Peng, J., Huang, L., Chen, J.: An efficient knn algorithm implemented on FPGA based heterogeneous computing system using opencl. In: 2015 IEEE 23rd Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, pp. 167–170 (2015)

  25. Muslim, F.B., Ma, L., Roozmeh, M., Lavagno, L.: Efficient fpga implementation of opencl high-performance computing applications via high-level synthesis. IEEE Access 5, 2747–2762 (2017)

    Article  Google Scholar 

  26. Luo, L., Wu, Y., Qiao, F., Yang, Y., Wei, Q., Zhou, X., Fan, Y., Xu, S., Liu, X., Yang, H.: Design of FPGA-based accelerator for convolutional neural network under heterogeneous computing framework with openCL. Int. J. Reconfig. Comput. (2018). https://doi.org/10.1155/2018/1785892

    Article  Google Scholar 

  27. Zhang, S., Wu, Y., Men, C., He, H., Liang, K.: Research on opencl optimization for fpga deep learning application. PloS ONE (2019). https://doi.org/10.1371/journal.pone.0222984

    Article  Google Scholar 

  28. Pire, T., Fischer, T., Civera, J., De Cristóforis, P., Berlles, J.J.: Stereo parallel tracking and mapping for robot localization. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IEEE, pp. 1373–1378 (2015)

  29. Konolige, K., Agrawal, M.: Frameslam: from bundle adjustment to real-time visual mapping. IEEE Trans. Robot. 24(5), 1066–1077 (2008)

    Article  Google Scholar 

  30. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Computer Vision—ECCV 2006. Springer, Berlin, pp. 430–443 (2006)

  31. Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2012, pp. 510–517 (2012)

  32. Tola, E., Lepetit, V., Fua, P.: Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 815–830 (2010)

    Article  Google Scholar 

  33. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)

  34. Bianconi, F., Fernández, A.: Evaluation of the effects of gabor filter parameters on texture classification. Pattern Recognit. 40(12), 3325–3335 (2007)

    Article  Google Scholar 

  35. Fularz, M., Kraft, M., Schmidt, A., Kasinski, A.: A high-performance fpga-based image feature detector and matcher based on the fast and brief algorithms. Int. J. Adv. Robot. Syst. 12(10), 141 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Hanoi University of Science and Technology, Hanoï, Vietnam

    Dai Duong Nguyen

  2. Systèmes et Applications des Technologies de l’Information et de l’Energie, CNRS, ENS Paris-Saclay, Université Paris-Saclay, 91405, Orsay, France

    Abdelhafid El Ouardi, Sergio Rodriguez & Samir Bouaziz

Authors
  1. Dai Duong Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelhafid El Ouardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sergio Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samir Bouaziz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdelhafid El Ouardi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, D.D., El Ouardi, A., Rodriguez, S. et al. FPGA implementation of HOOFR bucketing extractor-based real-time embedded SLAM applications. J Real-Time Image Proc 18, 525–538 (2021). https://doi.org/10.1007/s11554-020-00986-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00986-9

Keywords

Search

Navigation