skip to main content
10.1145/3508352.3561110acmconferencesArticle/Chapter ViewAbstractPublication PagesiccadConference Proceedingsconference-collections
invited-talk
Public Access

A Reconfigurable Hardware Library for Robot Scene Perception

Published: 22 December 2022 Publication History

Abstract

Perceiving the position and orientation of objects (i.e., pose estimation) is a crucial prerequisite for robots acting within their natural environment. We present a hardware acceleration approach to enable real-time and energy efficient articulated pose estimation for robots operating in unstructured environments. Our hardware accelerator implements Nonparametric Belief Propagation (NBP) to infer the belief distribution of articulated object poses. Our approach is on average, 26× more energy efficient than a high-end GPU and 11× faster than an embedded low-power GPU implementation. Moreover, we present a Monte-Carlo Perception Library generated from high-level synthesis to enable reconfigurable hardware designs on FPGA fabrics that are better tuned to user-specified scene, resource, and performance constraints.

References

[1]
[n.d.]. Xilinx Vitis Library. https://github.com/Xilinx/Vitis_Libraries/tree/master/vision.
[2]
X. Chen, R. Chen, Z. Sui, Z. Ye, Y. Liu, R. I. Bahar, and O. C. Jenkins. 2019. GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 3988--3995.
[3]
Karthik Desingh, Shiyang Lu, Anthony Opipari, and Odest Chadwicke Jenkins. 2019. Efficient nonparametric belief propagation for pose estimation and manipulation of articulated objects. Science Robotics 4, 30 (2019), eaaw4523. arXiv:https://www.science.org/doi/pdf/10.1126/scirobotics.aaw4523
[4]
Karthik Desingh, Anthony Opipari, and Odest Chadwicke Jenkins. 2018. Pull Message Passing for Nonparametric Belief Propagation. arXiv:1807.10487 [cs] (2018). arXiv:1807.10487 http://arxiv.org/abs/1807.10487
[5]
Marek Gorgon and Ryszard Tadeusiewicz. 2000. Hardware-based image processing library for Virtex FPGA. In Reconfigurable Technology: FPGAs for Computing and Applications II, Vol. 4212. SPIE, 1--10.
[6]
Lester Kalms, Ariel Podlubne, and Diana Göhringer. 2019. Hiflipvx: An open source high-level synthesis fpga library for image processing. In International Symposium on Applied Reconfigurable Computing. Springer, 149--164.
[7]
Atsutake Kosuge, Keisuke Yamamoto, Yukinori Akamine, and Takashi Oshima. 2021. An SoC-FPGA-Based Iterative-Closest-Point Accelerator Enabling Faster Picking Robots. IEEE Transactions on Industrial Electronics 68, 4 (2021), 3567--3576.
[8]
Y. Liu, G. Calderoni, and R. I. Bahar. 2020. Hardware Acceleration of Monte-Carlo Sampling for Energy Efficient Robust Robot Manipulation. In IEEE/ACM International Conference on Field Programmable Logic and Applications (FPL).
[9]
Yanqi Liu, Alessandro Costantini, R Bahar, Zhiqiang Sui, Zhefan Ye, Shiyang Lu, and Odest Chadwicke Jenkins. 2018. Robust object estimation using generative-discriminative inference for secure robotics applications. In Proceedings of the International Conference on Computer-Aided Design. ACM, 75.
[10]
Yanqi Liu, Can Eren Derman, Giuseppe Calderoni, and R. Iris Bahar. 2020. Hardware Acceleration of Robot Scene Perception Algorithms. In 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD). 1--8.
[11]
Anthony Opipari, Chao Chen, Shoutian Wang, Jana Pavlasek, Karthik Desingh, and Odest Chadwicke Jenkins. 2021. Differentiable Nonparametric Belief Propagation. arXiv preprint arXiv:2101.05948 (2021).
[12]
Jana Pavlasek, Stanley Lewis, Karthik Desingh, and Odest Chadwicke Jenkins. 2020. Parts-Based Articulated Object Localization in Clutter Using Belief Propagation. arXiv:2008.02881 [cs] (2020). arXiv:2008.02881 http://arxiv.org/abs/2008.02881
[13]
Judea Pearl. 1982. Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach. In Proceedings of the Second AAAI Conference on Artificial Intelligence (Pittsburgh, Pennsylvania) (AAAI'82). AAAI Press, 133--136.
[14]
Michael Schaeferling, Ulrich Hornung, and Gundolf Kiefer. 2012. Object recognition and pose estimation on embedded hardware: SURF-based system designs accelerated by FPGA logic. International Journal of Reconfigurable Computing 2012 (2012), 6.
[15]
László Schäffer, Zoltán Kincses, and Szilveszter Pletl. 2018. A Real-Time Pose Estimation Algorithm Based on FPGA and Sensor Fusion. In International Symposium on Intelligent Systems and Informatics (SISY).
[16]
Zhiqiang Sui, Zhefan Ye, and Odest Chadwicke Jenkins. 2018. Never Mind the Bounding Boxes, Here's the SAND Filters. arXiv:1808.04969 (2018).
[17]
Zhiqiang Sui, Zhefan Ye, and Odest Chadwicke Jenkins. 2018. Never Mind the Bounding Boxes, Here's the SAND Filters. CoRR abs/1808.04969 (2018). arXiv:1808.04969 http://arxiv.org/abs/1808.04969
[18]
Andreas Ten Pas and Robert Platt. 2016. Localizing handle-like grasp affordances in 3d point clouds. In Experimental Robotics. Springer, 623--638.
[19]
Jeong Woo, Young-Joong Kim, Jeong-on Lee, and Myo-Taeg Lim. 2006. Localization of mobile robot using particle filter. In 2006 SICE-ICASE International Joint Conference. IEEE, 3031--3034.

Cited By

View all
  • (2024)DNBP: Differentiable Nonparametric Belief PropagationACM / IMS Journal of Data Science10.1145/35927621:1(1-24)Online publication date: 16-Jan-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICCAD '22: Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design
October 2022
1467 pages
ISBN:9781450392174
DOI:10.1145/3508352
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

In-Cooperation

  • IEEE-EDS: Electronic Devices Society
  • IEEE CAS
  • IEEE CEDA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 December 2022

Check for updates

Author Tags

  1. belief propagation
  2. energy-efficient
  3. hardware acceleration
  4. robotics

Qualifiers

  • Invited-talk

Funding Sources

Conference

ICCAD '22
Sponsor:
ICCAD '22: IEEE/ACM International Conference on Computer-Aided Design
October 30 - November 3, 2022
California, San Diego

Acceptance Rates

Overall Acceptance Rate 457 of 1,762 submissions, 26%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)115
  • Downloads (Last 6 weeks)27
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)DNBP: Differentiable Nonparametric Belief PropagationACM / IMS Journal of Data Science10.1145/35927621:1(1-24)Online publication date: 16-Jan-2024

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media