Armand Behroozi
ABSTRACT
Simultaneous localization and mapping (SLAM) algorithms track an agent's movements through an unknown environment. SLAM must be fast and accurate to avoid adverse effects such as motion sickness in AR/VR headsets and navigation errors in autonomous robots and drones. However, accurate SLAM is computationally expensive and target platforms are often highly constrained. Therefore, to maintain real-time functionality, designers must either pay a large up-front cost to design specialized accelerators or reduce the algorithm's functionality, resulting in poor pose estimation.
We find that most SLAM algorithms are statically configured for the worst-case input and mismanage their computation budget. Thus, we present SlimSLAM, a domain-specific runtime scheduler which adapts SLAM algorithmic parameters based on input needs, minimizing computation while maintaining accuracy. SlimSLAM exploits information from a SLAM algorithm's state to detect and adjust over-provisioned parameters in real-time. We demonstrate SlimSLAM on the state-of-the-art real-time visual-inertial SLAM algorithm: HybVIO. We show that SlimSLAM is able to provide speedups up to 5.4× on the EuRoC, TUM-VI Room, and Monado VR datasets and outperforms other adaptive approaches on average by 2.3×-1.3× with iso-accuracy. This enables more accurate SLAM on constrained computation platforms such as the Raspberry Pi 4 (RPi4) where SlimSLAM is faster and more accurate than both HybVIO's static RPi4 configuration as well as other SLAM algorithms.
- Islam Ali and Hong Zhang. Are we ready for robust and resilient slam? a framework for quantitative characterization of slam datasets. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2810--2816, 2022.Google Scholar
Cross Ref
- Bahar Asgari, Ramyad Hadidi, Nima Shoghi Ghaleshahi, and Hyesoon Kim. Pisces: Power-aware implementation of slam by customizing efficient sparse algebra. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1--6, 2020.Google ScholarCross Ref
- Antoine Billy, Sébastien Pouteau, Pascal Desbarats, Serge Chaumette, and Jean-Philippe Domenger. Adaptive slam with synthetic stereo dataset generation for real-time dense 3d reconstruction. In VISIGRAPP 2019: 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, 2019.Google Scholar
- Willem Bles, Jelte E Bos, Bernd De Graaf, Eric Groen, and Alexander H Wertheim. Motion sickness: only one provocative conflict? Brain research bulletin, 47(5):481--487, 1998.Google Scholar
- Jelte E Bos, Willem Bles, and Eric L Groen. A theory on visually induced motion sickness. Displays, 29(2):47--57, 2008.Google ScholarCross Ref
- Tiancong Bu, Kaige Yan, and Jingweijia Tan. Towards fine-grained online adaptive approximation control for dense slam on embedded gpus. ACM Transactions on Design Automation of Electronic Systems (TODAES), 27(2):1--19, 2021.Google Scholar
- Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus W Achtelik, and Roland Siegwart. The euroc micro aerial vehicle datasets. The International Journal of Robotics Research, 2016.Google ScholarDigital Library
- Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, José Neira, Ian Reid, and John J. Leonard. Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age. IEEE Transactions on Robotics, 32(6):1309--1332, 2016.Google ScholarDigital Library
- Carlos Campos, Richard Elvira, Juan J Gómez Rodríguez, José MM Montiel, and Juan D Tardós. Orb-slam3: An accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Transactions on Robotics, 37(6):1874--1890, 2021.Google ScholarCross Ref
- Siddharth Choudhary, Vadim Indelman, Henrik I Christensen, and Frank Dellaert. Information-based reduced landmark slam. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 4620--4627. IEEE, 2015.Google ScholarCross Ref
- Andrew Courtney. Why is oculus quest 2 not charging while playing? (solved). https://vrlowdown.com/why-is-oculus-quest-2-not-charging-while-playing/.Google Scholar
- Mateo De Mayo. Monado slam datasets. https://huggingface.co/datasets/collabora/monado-slam-datasets, 2023.Google Scholar
- Guoguang Du, Kai Wang, Shiguo Lian, and Kaiyong Zhao. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review. Artificial Intelligence Review, 54(3):1677--1734, 2021.Google ScholarDigital Library
- Alejandro Fontan, Javier Civera, and Rudolph Triebel. Information-driven direct rgb-d odometry. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4929--4937, 2020.Google ScholarCross Ref
- Christian Forster, Luca Carlone, Frank Dellaert, and Davide Scaramuzza. On-manifold preintegration for real-time visual-inertial odometry. IEEE Transactions on Robotics, 33(1):1--21, 2016.Google ScholarDigital Library
- Andrei Frumusanu and Ryan Smith. The snapdragon 845 performance preview: Setting the stage for flagship android 2018. https://www.anandtech.com/show/12420/snapdragon-845-performance-preview/4.Google Scholar
- Yiming Gan, Yu Bo, Boyuan Tian, Leimeng Xu, Wei Hu, Shaoshan Liu, Qiang Liu, Yanjun Zhang, Jie Tang, and Yuhao Zhu. Eudoxus: Characterizing and accelerating localization in autonomous machines industry track paper. In 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 827--840. IEEE, 2021.Google ScholarCross Ref
- Quentin Gautier, Alric Althoff, and Ryan Kastner. Fpga architectures for real-time dense slam. In 2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP), volume 2160-052X, pages 83--90, 2019.Google ScholarCross Ref
- Patrick Geneva, Kevin Eckenhoff, Woosik Lee, Yulin Yang, and Guoquan Huang. Openvins: A research platform for visual-inertial estimation. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 4666--4672. IEEE, 2020.Google ScholarCross Ref
- Maria Rafaela Gkeka, Alexandros Patras, Christos D. Antonopoulos, Spyros Lalis, and Nikolaos Bellas. Fpga architectures for approximate dense slam computing. In 2021 Design, Automation and Test in Europe Conference and Exhibition (DATE), pages 828--833, 2021.Google Scholar
- Sandra Hernández, José Araujo, Patric Jensfelt, Ioannis Karagiannis, Ananya Muddukrishna, and Bryan Donyanavard. Cross-layer configuration optimization for localization on resource-constrained devices. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2282--2288. IEEE, 2021.Google ScholarDigital Library
- Joel Hesch, Anna Kozminski, and Oskar Linde. Powered by ai: Oculus insight. https://ai.meta.com/blog/powered-by-ai-oculus-insight/.Google Scholar
- Stefan Hinterstoisser, Vincent Lepetit, Slobodan Ilic, Stefan Holzer, Gary Bradski, Kurt Konolige, and Nassir Navab. Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In Computer Vision-ACCV 2012: 11th Asian Conference on Computer Vision, Daejeon, Korea, November 5-9, 2012, Revised Selected Papers, Part I 11, pages 548--562. Springer, 2013.Google ScholarDigital Library
- Klaus Hinum. Qualcomm snapdragon 865. https://www.notebookcheck.net/Qualcomm-Snapdragon-865-SoC-Benchmarks-and-Specs.448194.0.html.Google Scholar
- Zheng Huai and Guoquan Huang. Robocentric visual-inertial odometry. The International Journal of Robotics Research, 41(7):667--689, 2022.Google ScholarDigital Library
- Muhammad Huzaifa, Rishi Desai, Samuel Grayson, Xutao Jiang, Ying Jing, Jae Lee, Fang Lu, Yihan Pang, Joseph Ravichandran, Finn Sinclair, et al. Illixr: Enabling end-to-end extended reality research. In 2021 IEEE International Symposium on Workload Characterization (IISWC), pages 24--38. IEEE, 2021.Google ScholarCross Ref
- Abdullah Khalufa, Graham Riley, and Mikel Luján. A dynamic adaptation strategy for energy-efficient keyframe-based visual slam. In Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA), pages 3--10. The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 2019.Google Scholar
- Jouko Kinnari, Francesco Verdoja, and Ville Kyrki. Gnss-denied geolocalization of uavs by visual matching of onboard camera images with orthophotos. In 2021 20th International Conference on Advanced Robotics (ICAR), pages 555--562, 2021.Google ScholarCross Ref
- Andreas Koch, Ingolf Cascorbi, Martin Westhofen, Manuel Dafotakis, Sebastian Klapa, and Johann Peter Kuhtz-Buschbeck. The neuro-physiology and treatment of motion sickness. Deutsches Ärzteblatt International, 115(41):687, 2018.Google Scholar
- Mathieu Labbé and François Michaud. Rtab-map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. Journal of Field Robotics, 36(2):416--446, 2019.Google ScholarCross Ref
- Stefan Leutenegger, Paul Furgale, Vincent Rabaud, Margarita Chli, Kurt Konolige, and Roland Siegwart. Keyframe-based visual-inertial slam using nonlinear optimization. Proceedings of Robotis Science and Systems (RSS) 2013, 2013.Google ScholarCross Ref
- Stefan Leutenegger, Paul Furgale, Vincent Rabaud, Margarita Chli, Kurt Konolige, and Roland Siegwart. Keyframe-based visual-inertial slam using nonlinear optimization. Proceedings of Robotis Science and Systems (RSS) 2013, 2013.Google ScholarCross Ref
- Qiang Liu, Yuhui Hao, Weizhuang Liu, Bo Yu, Yiming Gan, Jie Tang, Shao-Shan Liu, and Yuhao Zhu. An energy efficient and runtime reconfigurable accelerator for robotic localization. IEEE Transactions on Computers, 72(7):1943--1957, 2023.Google ScholarDigital Library
- Runze Liu, Jianlei Yang, Yiran Chen, and Weisheng Zhao. Eslam: An energy-efficient accelerator for real-time orb-slam on fpga platform. In Proceedings of the 56th Annual Design Automation Conference 2019, DAC '19, New York, NY, USA, 2019. Association for Computing Machinery.Google ScholarDigital Library
- Weizhuang Liu, Bo Yu, Yiming Gan, Qiang Liu, Jie Tang, Shaoshan Liu, and Yuhao Zhu. Archytas: A framework for synthesizing and dynamically optimizing accelerators for robotic localization. In MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO '21, page 479--493, New York, NY, USA, 2021. Association for Computing Machinery.Google ScholarDigital Library
- Dipan Kumar Mandal, Srivatsava Jandhyala, Om J Omer, Gurpreet S Kalsi, Biji George, Gopi Neela, Santhosh Kumar Rethinagiri, Sreenivas Subramoney, Lance Hacking, Jim Radford, Eagle Jones, Belliappa Kuttanna, and Hong Wang. Visual inertial odometry at the edge: A hardware-software co-design approach for ultra-low latency and power. In 2019 Design, Automation and Test in Europe Conference and Exhibition (DATE), pages 960--963, 2019.Google Scholar
- Pranay Mathur, Nikhil Khedekar, and Kostas Alexis. Resource-aware online parameter adaptation for computationally-constrained visual-inertial navigation systems. In 2021 20th International Conference on Advanced Robotics (ICAR), pages 842--848. IEEE, 2021.Google ScholarCross Ref
- Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. Orb-slam: a versatile and accurate monocular slam system. IEEE transactions on robotics, 31(5):1147--1163, 2015.Google Scholar
- Luigi Nardi, Bruno Bodin, Sajad Saeedi, Emanuele Vespa, Andrew J Davison, and Paul HJ Kelly. Algorithmic performance-accuracy trade-off in 3d vision applications using hypermapper. In 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pages 1434--1443. IEEE, 2017.Google ScholarCross Ref
- Jinwook Oh, Jungwook Choi, Guilherme C Januario, and Kailash Gopalakrishnan. Energy-efficient simultaneous localization and mapping via compounded approximate computing. In 2016 IEEE International Workshop on Signal Processing Systems (SiPS), pages 51--56. IEEE, 2016.Google ScholarCross Ref
- Oscar Palomar, Andy Nisbet, John Mawer, Graham Riley, and Mikel Lujan. Reduced precision applicability and trade-offs for slam algorithms. In Third Workshop on Approximate Computing (WACAS), 2017.Google Scholar
- Sunghyun Park, Salar Latifi, Yongjun Park, Armand Behroozi, Byungsoo Jeon, and Scott Mahlke. Srtuner: Effective compiler optimization customization by exposing synergistic relations. In 2022 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), pages 118--130, 2022.Google ScholarDigital Library
- Yeonsoo Park and Soohyun Bae. Keeping less is more: Point sparsification for visual slam. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7936--7943, 2022.Google ScholarCross Ref
- Lina M. Paz, Pedro PiniÉs, Juan D. TardÓs, and JosÉ Neira. Large-scale 6-dof slam with stereo-in-hand. IEEE Transactions on Robotics, 24(5):946--957, 2008.Google ScholarDigital Library
- Yan Pei, Swarnendu Biswas, Donald S Fussell, and Keshav Pingali. Slambooster: An application-aware online controller for approximation in dense slam. In 2019 28th International Conference on Parallel Architectures and Compilation Techniques (PACT), pages 296--310. IEEE, 2019.Google ScholarCross Ref
- Yan Pei, Swarnendu Biswas, Donald S Fussell, and Keshav Pingali. A methodology for principled approximation in visual slam. In Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques, pages 373--386, 2020.Google ScholarDigital Library
- Jin-Chun Piao and Shin-Dug Kim. Adaptive monocular visual-inertial slam for real-time augmented reality applications in mobile devices. Sensors, 17(11):2567, 2017.Google ScholarCross Ref
- Jin-Chun Piao and Shin-Dug Kim. Real-time visual-inertial slam based on adaptive keyframe selection for mobile ar applications. IEEE Transactions on Multimedia, 21(11):2827--2836, 2019.Google ScholarCross Ref
- Tong Qin, Sankalp Dayal, Joydeep Biswas, Varada Gopalakrishnan, Adam Fineberg, and Rahul Bakshi. Low-precision arithmetic makes robot localization more efficient, 2023.Google Scholar
- Xiaochen QIU, Hai ZHANG, and Wenxing FU. Lightweight hybrid visual-inertial odometry with closed-form zero velocity update. Chinese Journal of Aeronautics, 33(12):3344--3359, 2020.Google ScholarCross Ref
- Marco Recenti, Carlo Ricciardi, Romain Aubonnet, Ilaria Picone, Deborah Jacob, Halldór ÁR Svansson, Sólveig Agnarsdóttir, Gunnar H Karlsson, Valdís Baeringsdóttir, Hannes Petersen, et al. Toward predicting motion sickness using virtual reality and a moving platform assessing brain, muscles, and heart signals. Frontiers in Bioengineering and Biotechnology, 9:635661, 2021.Google ScholarCross Ref
- Antoni Rosinol, Marcus Abate, Yun Chang, and Luca Carlone. Kimera: an open-source library for real-time metric-semantic localization and mapping. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 1689--1696. IEEE, 2020.Google ScholarCross Ref
- Sajad Saeedi, Luigi Nardi, Edward Johns, Bruno Bodin, Paul H. J. Kelly, and Andrew J. Davison. Application-oriented design space exploration for slam algorithms. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 5716--5723, 2017.Google ScholarDigital Library
- David Schubert, Thore Goll, Nikolaus Demmel, Vladyslav Usenko, Jörg Stückler, and Daniel Cremers. The tum vi benchmark for evaluating visual-inertial odometry. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1680--1687. IEEE, 2018.Google ScholarDigital Library
- Otto Seiskari, Pekka Rantalankila, Juho Kannala, Jerry Ylilammi, Esa Rahtu, and Arno Solin. Hybvio: Pushing the limits of real-time visual-inertial odometry. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 701--710, 2022.Google ScholarCross Ref
- Tixiao Shan, Brendan Englot, Carlo Ratti, and Rus Daniela. Lvi-sam: Tightly-coupled lidar-visual-inertial odometry via smoothing and mapping. In IEEE International Conference on Robotics and Automation (ICRA), pages 5692--5698. IEEE, 2021.Google ScholarDigital Library
- Xuesong Shi, Dongjiang Li, Pengpeng Zhao, Qinbin Tian, Yuxin Tian, Qiwei Long, Chunhao Zhu, Jingwei Song, Fei Qiao, Le Song, Yangquan Guo, Zhigang Wang, Yimin Zhang, Baoxing Qin, Wei Yang, Fangshi Wang, Rosa H. M. Chan, and Qi She. Are we ready for service robots? the OpenLORIS-Scene datasets for lifelong SLAM. In 2020 International Conference on Robotics and Automation (ICRA), pages 3139--3145, 2020.Google ScholarCross Ref
- Arno Solin, Santiago Cortes, Esa Rahtu, and Juho Kannala. Pivo: Probabilistic inertial-visual odometry for occlusion-robust navigation. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 616--625. IEEE, 2018.Google ScholarCross Ref
- J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers. A benchmark for the evaluation of rgb-d slam systems. In Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.Google ScholarCross Ref
- Amr Suleiman, Zhengdong Zhang, Luca Carlone, Sertac Karaman, and Vivienne Sze. Navion: A 2-mw fully integrated real-time visual-inertial odometry accelerator for autonomous navigation of nano drones. IEEE Journal of Solid-State Circuits, 54(4):1106--1119, 2019.Google ScholarCross Ref
- Vladyslav Usenko, Nikolaus Demmel, David Schubert, Jörg Stückler, and Daniel Cremers. Visual-inertial mapping with non-linear factor recovery. IEEE Robotics and Automation Letters, 5(2):422--429, 2019.Google ScholarCross Ref
- Lukas Von Stumberg and Daniel Cremers. Dm-vio: Delayed marginalization visual-inertial odometry. IEEE Robotics and Automation Letters, 7(2):1408--1415, 2022.Google ScholarCross Ref
- Kejian J Wu, Tien Do, Luis C Carrillo-Arce, and Stergios I Roumeliotis. On the vins resource-allocation problem for a dual-camera, small-size quadrotor. In International Symposium on Experimental Robotics, pages 538--549. Springer, 2016.Google Scholar
- Tianguang Zhang, Xiaodong Liu, Kolja Kühnlenz, and Martin Buss. Visual odometry for the autonomous city explorer. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 3513--3518, 2009.Google ScholarDigital Library
- Zhengdong Zhang, Amr AbdulZahir Suleiman, Luca Carlone, Vivienne Sze, and Sertac Karaman. Visual-inertial odometry on chip: An algorithm-and-hardware co-design approach. Tech. Rep., 2017.Google Scholar
- Yipu Zhao and Patricio A Vela. Good feature selection for least squares pose optimization in vo/vslam. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1183--1189. IEEE, 2018.Google ScholarDigital Library
- Yipu Zhao and Patricio A Vela. Good feature matching: Toward accurate, robust vo/vslam with low latency. IEEE Transactions on Robotics, 36(3):657--675, 2020.Google ScholarCross Ref
- Danping Zou, Yuanxin Wu, Ling Pei, Haibin Ling, and Wenxian Yu. Structvio: Visual-inertial odometry with structural regularity of man-made environments. IEEE Transactions on Robotics, 35(4):999--1013, 2019.Google ScholarDigital Library
Index Terms
- SlimSLAM: An Adaptive Runtime for Visual-Inertial Simultaneous Localization and Mapping
Recommendations
Combined visual odometry and visual compass for off-road mobile robots localization
In this paper, we present the work related to the application of a visual odometry approach to estimate the location of mobile robots operating in off-road conditions. The visual odometry approach is based on template matching, which deals with ...
Multiple-Robot Simultaneous Localization and Mapping: A Review
Simultaneous localization and mapping SLAM in unknown GPS-denied environments is a major challenge for researchers in the field of mobile robotics. Many solutions for single-robot SLAM exist; however, moving to a platform of multiple robots adds many ...
Comments