Yuhui Hao
ABSTRACT
Factor graph is a graph representing the factorization of a probability distribution function, and has been utilized in many autonomous machine computing tasks, such as localization, tracking, planning and control etc. We are developing an architecture with the goal of using factor graph as a common abstraction for most, if not, all autonomous machine computing tasks. If successful, the architecture would provide a very simple interface of mapping autonomous machine functions to the underlying compute hardware. As a first step of such an attempt, this paper presents our most recent work of developing a factor graph accelerator for LiDAR-Inertial Odometry (LIO), an essential task in many autonomous machines, such as autonomous vehicles and mobile robots. By modeling LIO as a factor graph, the proposed accelerator not only supports multi-sensor fusion such as LiDAR, inertial measurement unit (IMU), GPS, etc., but solves the global optimization problem of robot navigation in batch or incremental modes. Our evaluation demonstrates that the proposed design significantly improves the real-time performance and energy efficiency of autonomous machine navigation systems. The initial success suggests the potential of generalizing the factor graph architecture as a common abstraction for autonomous machine computing, including tracking, planning, and control etc.
- Shaoshan Liu and Jean-Luc Gaudiot. Rise of the autonomous machines. Computer, 55, 2022.Google Scholar
- Shaoshan Liu, Zishen Wan, Bo Yu, and Yu Wang. Robotic computing on fpgas. Synthesis Lectures on Computer Architecture, 16(1):1--218, 2021.Google ScholarCross Ref
- Shaoshan Liu, Yuhao Zhu, Bo Yu, Jean-Luc Gaudiot, and Guang R Gao. Dataflow accelerator architecture for autonomous machine computing. arXiv preprint arXiv:2109.07047, 2021.Google Scholar
- Bo Yu, Wei Hu, Leimeng Xu, Jie Tang, Shaoshan Liu, and Yuhao Zhu. Building the computing system for autonomous micromobility vehicles: Design constraints and architectural optimizations. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 1067--1081. IEEE, 2020.Google ScholarCross Ref
- Shaoshan Liu, Jie Tang, Zhe Zhang, and Jean-Luc Gaudiot. Computer architectures for autonomous driving. Computer, 50(8):18--25, 2017.Google ScholarDigital Library
- Weizhuang Liu, Bo Yu, Yiming Gan, Qiang Liu, Jie Tang, Shaoshan Liu, and Yuhao Zhu. Archytas: A framework for synthesizing and dynamically optimizing accelerators for robotic localization. In 2021 54th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 2021.Google ScholarDigital Library
- Yiming Gan, Yu Bo, Boyuan Tian, Leimeng Xu, Wei Hu, Shaoshan Liu, Qiang Liu, Yanjun Zhang, Jie Tang, and Yuhao Zhu. Eudoxus: Characterizing and accelerating localization in autonomous machines industry track paper. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 827--840. IEEE, 2021.Google ScholarCross Ref
- Frank Dellaert, Michael Kaess, et al. Factor graphs for robot perception. Foundations and Trends® in Robotics, 6(1--2):1--139, 2017.Google ScholarCross Ref
- Jing Dong, Mustafa Mukadam, Frank Dellaert, and Byron Boots. Motion planning as probabilistic inference using gaussian processes and factor graphs. In Robotics: Science and Systems, volume 12, 2016.Google ScholarCross Ref
- Frank Dellaert. Factor graphs: Exploiting structure in robotics. Annual Review of Control, Robotics, and Autonomous Systems, 4:141--166, 2021.Google Scholar
- Frank Dellaert. Factor graphs and gtsam: A hands-on introduction. Technical report, Georgia Institute of Technology, 2012.Google Scholar
- Vadim Indelman, Stephen Williams, Michael Kaess, and Frank Dellaert. Factor graph based incremental smoothing in inertial navigation systems. In 2012 15th International Conference on Information Fusion, pages 2154--2161. IEEE, 2012.Google Scholar
- Han-Pang Chiu, Stephen Williams, Frank Dellaert, Supun Samarasekera, and Rakesh Kumar. Robust vision-aided navigation using sliding-window factor graphs. In 2013 IEEE International Conference on Robotics and Automation, pages 46--53. IEEE, 2013.Google ScholarCross Ref
- Rainer Kümmerle, Giorgio Grisetti, Hauke Strasdat, Kurt Konolige, and Wolfram Burgard. g 2 o: A general framework for graph optimization. In 2011 IEEE International Conference on Robotics and Automation, pages 3607--3613. IEEE, 2011.Google ScholarCross Ref
- Michael Kaess, Hordur Johannsson, Richard Roberts, Viorela Ila, John J Leonard, and Frank Dellaert. isam2: Incremental smoothing and mapping using the bayes tree. The International Journal of Robotics Research, 31(2):216--235, 2012.Google ScholarDigital Library
- Tixiao Shan, Brendan Englot, Drew Meyers, Wei Wang, Carlo Ratti, and Daniela Rus. Lio-sam: Tightly-coupled lidar inertial odometry via smoothing and mapping. In 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 5135--5142. IEEE, 2020.Google ScholarDigital Library
- Frank Dellaert Michael Kaess, Ananth Ranganathan. iSAM: Incremental Smoothing and Mapping. IEEE Transactions on Robotics, 24:1365 -- 1378, 2008.Google Scholar
- Shan, Tixiao. the Walking dataset. https://github.com/TixiaoShan/LIO-SAM. Accessed: 2022-07-21.Google Scholar
- Shan, Tixiao. the Park dataset. https://github.com/TixiaoShan/LIO-SAM. Accessed: 2022-07-21.Google Scholar
- Georgia Institute of Technology. GTSAM. https://github.com/borglab/gtsam. Accessed: 2022-07-21.Google Scholar
- NVIDIA. TX1 datasheet. http://images.nvidia.com/content/tegra/embedded-systems/pdf/JTX1-Module-Product-sheet.pdf. Accessed: 2022-07-21.Google Scholar
Index Terms
- Factor Graph Accelerator for LiDAR-Inertial Odometry (Invited Paper)
Recommendations
A model-based approach to robot kinematics and control using discrete factor graphs with belief propagation
Much of recent researches in robotics have shifted the focus from traditionally-specific industrial tasks to investigations of new types of robots with alternative ways of controlling them. In this paper, we describe the development of a generic method ...
Improving Inertial Odometry Through Particle Swarm Optimization in the RoboCup Small Size League
RoboCup 2023: Robot World Cup XXVIAbstractSmall Size League (SSL) robots require mobile navigation to interact with their surroundings. Therefore, robots may rely on odometry to track their movement from the actuator’s data and navigate. The odometry is based on the robot’s kinematic ...
3D Visibility Graph based Motion Planning and Control
ICRAI '19: Proceedings of the 5th International Conference on Robotics and Artificial IntelligenceThis paper proposed a motion planning and control approach for real-time collision avoidance of high degree-of-freedom (DoF) robots. It constructs a 3D visibility graph by representing external obstacles with discrete polyhedrons and searches for the ...
Comments