research-article

eSLAM: An Energy-Efficient Accelerator for Real-Time ORB-SLAM on FPGA Platform

Authors:

Runze Liu,

Jianlei Yang,

Yiran Chen,

Weisheng ZhaoAuthors Info & Claims

DAC '19: Proceedings of the 56th Annual Design Automation Conference 2019

Article No.: 193, Pages 1 - 6

https://doi.org/10.1145/3316781.3317820

Published: 02 June 2019 Publication History

Get Access

Abstract

Simultaneous Localization and Mapping (SLAM) is a critical task for autonomous navigation. However, due to the computational complexity of SLAM algorithms, it is very difficult to achieve real-time implementation on low-power platforms. We propose an energy-efficient architecture for real-time ORB (Oriented-FAST and Rotated-BRIEF) based visual SLAM system by accelerating the most time-consuming stages of feature extraction and matching on FPGA platform. Moreover, the original ORB descriptor pattern is reformed as a rotational symmetric manner which is much more hardware friendly. Optimizations including rescheduling and parallelizing are further utilized to improve the throughput and reduce the memory footprint. Compared with Intel i7 and ARM Cortex-A9 CPUs on TUM dataset, our FPGA realization achieves up to 3× and 31× frame rate improvement, as well as up to 71× and 25× energy efficiency improvement, respectively.

References

[1]

Michael Calonder, Vincent Lepetit, Christoph Strecha, and Pascal Fua. 2010. BRIEF: binary robust independent elementary features. In Proceedings of European Conference on Computer Vision (ECCV). 778--792.

Digital Library

Google Scholar

[2]

J. Cong, B. Grigorian, G. Reinman, and M. Vitanza. 2011. Accelerating vision and navigation applications on a customizable platform. In Proceedings of IEEE International Conference on Application-specific Systems, Architectures and Processors (ASAP). 25--32.

Digital Library

Google Scholar

[3]

Hugh Durrant-Whyte and Tim Bailey. 2006. Simultaneous localization and mapping: part I. IEEE robotics & automation magazine 13, 2 (2006), 99--110.

Google Scholar

[4]

Weikang Fang, Yanjun Zhang, Bo Yu, and Shaoshan Liu. 2017. FPGA-based ORB Feature Extraction for Real-Time Visual SLAM. In Proceedings of International Conference on Field Programmable Technology (FPT). 275--278.

Crossref

Google Scholar

[5]

David Fleet and Yair Weiss. 2006. Optical flow estimation. In Handbook of mathematical models in computer vision. Springer, 237--257.

Google Scholar

[6]

Mengyuan Gu, Kaiyuan Guo, Wenqiang Wang, Yu Wang, and Huazhong Yang. 2015. An FPGA-based real-time simultaneous localization and mapping system. In Proceedings of International Conference on Field Programmable Technology (FPT). 200--203.

Crossref

Google Scholar

[7]

Jorge J. Moré. 1978. The Levenberg-Marquardt algorithm: Implementation and theory. Lecture Notes in Mathematics 630 (1978), 105--116.

Crossref

Google Scholar

[8]

Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary R. Bradski. 2012. ORB: an efficient alternative to SIFT or SURF. In Proceedings of International Conference on Computer Vision (ICCV). 2564--2571.

Digital Library

Google Scholar

[9]

Intel Chip's Specifications. 2017. Intel Core i7-4700MQ Processor, 22nm. (2017). https://ark.intel.com/content/www/us/en/ark/products/75117/intel-core-i7-4700mq-processor-6m-cache-up-to-3-40-ghz.html.

Google Scholar

[10]

Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. 2012. A benchmark for the evaluation of RGB-D SLAM systems. In IEEE/RSJ International Conference on Intelligent Robots and Systems. 573--580.

Crossref

Google Scholar

[11]

Amr Suleiman, Zhengdong Zhang, Luca Carlone, Sertac Karaman, and Vivienne Sze. 2019. Navion: A 2mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones. IEEE Journal of Solid-State Circuits (JSSC) (2019), 1--14.

Google Scholar

[12]

Xilinx. 2018. Zynq-7000 SoC. (2018). https://www.xilinx.com/products/silicon-devices/soc/zynq-7000.html.

Google Scholar

Cited By

View all

Nieto RMachado FFernández-Conde JLobato DCañas J(2025)Open-source ROS-based simulation for verification of FPGA robotics applicationsMicroprocessors and Microsystems10.1016/j.micpro.2025.105143113(105143)Online publication date: Mar-2025
https://doi.org/10.1016/j.micpro.2025.105143
El Bouazzaoui IRodríguez Flórez SEl Ouardi A(2025)FPGA architecture-based front-end processing for SLAM applicationsJournal of Real-Time Image Processing10.1007/s11554-025-01650-w22:2Online publication date: 6-Mar-2025
https://doi.org/10.1007/s11554-025-01650-w
Tao SMinami YIshikawa M(2025)Performance evaluation of ORB-SLAM3 with quantized imagesArtificial Life and Robotics10.1007/s10015-025-01006-4Online publication date: 7-Feb-2025
https://doi.org/10.1007/s10015-025-01006-4
Show More Cited By

Recommendations

Portable and scalable FPGA-based acceleration of a direct linear system solver

FPGAs have the potential to serve as a platform for accelerating many computations including scientific applications. However, the large development cost and short life span for FPGA designs have limited their adoption by the scientific computing ...
SIFT Keypoint Descriptor Matching Algorithm: A Fully Pipelined Accelerator on FPGA(Abstract Only)
FPGA '18: Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

Scale Invariant Feature Transform (SIFT) algorithm is one of the classical feature extraction algorithms that is well known in Computer Vision. It consists of two stages: keypoint descriptor extraction and descriptor matching. SIFT descriptor matching ...
Accelerating Big Data Analytics Using FPGAs
FCCM '15: Proceedings of the 2015 IEEE 23rd Annual International Symposium on Field-Programmable Custom Computing Machines

Emerging big data analytics applications require a significant amount of server computational power. As chips are hitting power limits, computing systems are moving away from general-purpose designs and toward greater specialization. Hardware ...

Comments

Information & Contributors

Information

Published In

DAC '19: Proceedings of the 56th Annual Design Automation Conference 2019

June 2019

1378 pages

ISBN:9781450367257

DOI:10.1145/3316781

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Key Technology Program of China
State Key Laboratory of Software Development Environment
National Natural Science Foundation of China
CCF-Tencent

Conference

DAC '19

Sponsor:

SIGDA

DAC '19: The 56th Annual Design Automation Conference 2019

June 2 - 6, 2019

NV, Las Vegas, USA

Acceptance Rates

Overall Acceptance Rate 1,770 of 5,499 submissions, 32%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

61
Total Citations
View Citations
978
Total Downloads

Downloads (Last 12 months)129
Downloads (Last 6 weeks)17

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Nieto RMachado FFernández-Conde JLobato DCañas J(2025)Open-source ROS-based simulation for verification of FPGA robotics applicationsMicroprocessors and Microsystems10.1016/j.micpro.2025.105143113(105143)Online publication date: Mar-2025
https://doi.org/10.1016/j.micpro.2025.105143
El Bouazzaoui IRodríguez Flórez SEl Ouardi A(2025)FPGA architecture-based front-end processing for SLAM applicationsJournal of Real-Time Image Processing10.1007/s11554-025-01650-w22:2Online publication date: 6-Mar-2025
https://doi.org/10.1007/s11554-025-01650-w
Tao SMinami YIshikawa M(2025)Performance evaluation of ORB-SLAM3 with quantized imagesArtificial Life and Robotics10.1007/s10015-025-01006-4Online publication date: 7-Feb-2025
https://doi.org/10.1007/s10015-025-01006-4
Taranco RArnau JGonzález A(2024)SLIDEX: A Novel Architecture for Sliding Window ProcessingProceedings of the 38th ACM International Conference on Supercomputing10.1145/3650200.3656613(312-323)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3650200.3656613
Li DZhao YXu JZhang SShangguan LMa QDing XYang Z(2024)Reshaping Edge-Assisted Visual SLAM by Embracing On-Chip IntelligenceIEEE Transactions on Mobile Computing10.1109/TMC.2024.342445223:12(12983-12997)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3424452
Qi XLiu YHao SLiu ZHuang KYang MZhou LZhou J(2024)A High-Performance ORB Accelerator with Algorithm and Hardware Co-design for Visual Localization2024 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS58744.2024.10558318(1-5)Online publication date: 19-May-2024
https://doi.org/10.1109/ISCAS58744.2024.10558318
Li DZhao YXu JZhang SShangguan LYang Z(2024)edgeSLAM2: Rethinking Edge-Assisted Visual SLAM with On-Chip IntelligenceIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621278(1481-1490)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621278
Li MZhu HZhang YHe SLiu QZeng XChen C(2024)A Real-Time Optical-Flow-based SLAM FPGA Accelerator with Inter-Frame Similarity Exploitation and Correlation-Guided Mixed-Precision Flow Update2024 IEEE Asian Solid-State Circuits Conference (A-SSCC)10.1109/A-SSCC60305.2024.10848536(1-3)Online publication date: 18-Nov-2024
https://doi.org/10.1109/A-SSCC60305.2024.10848536
Huang QZhang YZheng JShang GChen G(2024)A CNN-Based Real-Time Dense Stereo SLAM System on Embedded FPGAArtificial Intelligence10.1007/978-981-99-9119-8_53(569-574)Online publication date: 3-Feb-2024
https://doi.org/10.1007/978-981-99-9119-8_53
Cuero JCardenas PTrespalacios R(2024)FPSoC Integration in ROS-Enabled Mobile Robots for Enhanced FunctionalityAdvances in Automation and Robotics Research10.1007/978-3-031-54763-8_11(104-112)Online publication date: 31-Mar-2024
https://doi.org/10.1007/978-3-031-54763-8_11
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Portable and scalable FPGA-based acceleration of a direct linear system solver

SIFT Keypoint Descriptor Matching Algorithm: A Fully Pipelined Accelerator on FPGA(Abstract Only)

Accelerating Big Data Analytics Using FPGAs

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations