research-article

Semi-Direct Monocular Visual-Inertial Odometry Using Point and Line Features for IoV

Authors:

Honglong ChenAuthors Info & Claims

ACM Transactions on Internet Technology (TOIT), Volume 22, Issue 1

Article No.: 5, Pages 1 - 23

https://doi.org/10.1145/3432248

Published: 28 September 2021 Publication History

Abstract

The precise measuring of vehicle location has been a critical task in enhancing the autonomous driving in terms of intelligent decision making and safe transportation. Internet of Vehicles (IoV) is an important infrastructure in support of autonomous driving, allowing real-time road information exchanging and sharing for localizing vehicles. Global positioning System (GPS) is widely used in the traditional IoV system. GPS is unable to meet the key application requirements of autonomous driving due to meter level error and signal deterioration. In this article, we propose a novel solution, named Semi-Direct Monocular Visual-Inertial Odometry using Point and Line Features (SDMPL-VIO) for precise vehicle localization. Our SDMPL-VIO model takes advantage of a low-cost Inertial Measurement Unit (IMU) and monocular camera, using them as the sensor to acquire the surrounding environmental information. Visual-Inertial Odometry (VIO), taking into account both point and line features, is proposed, which is able to deal with both weak texture and dynamic environment. We use a semi-direct method to deal with keyframes and non-keyframes, respectively. Dual sliding window mechanisms can effectively fuse point-line and IMU information. To evaluate our SDMPL-VIO system model, we conduct extensive experiments on both an indoor dataset (i.e., EuRoC) and an outdoor dataset (i.e., KITTI) from the real-world applications, respectively. The experimental results show that the accuracy of SDMPL-VIO proposed by us is better than the mainstream VIO system at present. Especially in the weak texture of the datasets, fast-moving datasets, and other challenging datasets, SDMPL-VIO has a relatively high robustness.

References

[1]

Mehiddin Albaali and Roger Fletcher. 1985. Variational methods for non-linear least-squares. Journal of the Operational Research Society 36, 5 (1985), 405–421.

[2]

Guillaume Bresson, Thomas Feraud, Romuald Aufrere, Paul Checchin, and Roland Chapuis. 2015. Real-time monocular slam with low memory requirements. IEEE Transactions on Intelligent Transportation Systems 16, 4 (2015), 1827–1839.

Digital Library

[3]

Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus Achtelik, and Roland Siegwart. 2016. The EuRoC micro aerial vehicle datasets. The International Journal of Robotics Research 35, 10 (2016), 1157–1163.

Digital Library

[4]

Alejo Concha, Giuseppe Loianno, Vijay Kumar, and Javier Civera. 2016. Visual-inertial direct SLAM. In 2016 IEEE International Conference on Robotics and Automation. 1331–1338.

Digital Library

[5]

Christian Forster, Luca Carlone, Frank Dellaert, and Davide Scaramuzza. 2015. On-manifold preintegration for real-time visual-inertial odometry. IEEE Transactions on Robotics 33, 1 (2015), 1–21.

Digital Library

[6]

Christian Forster, Zichao Zhang, Michael Gassner, Manuel Werlberger, and Davide Scaramuzza. 2017. SVO: Semidirect visual odometry for monocular and multicamera systems. IEEE Transactions on Robotics 33, 2 (2017), 249–265.

Digital Library

[7]

Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. 2013. Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research 32, 11 (2013), 1231–1237.

Digital Library

[8]

Ruben Gomezojeda and Javier Gonzalezjimenez. 2016. Robust stereo visual odometry through a probabilistic combination of points and line segments. In 2016 IEEE International Conference on Robotics and Automation. 2521–2526.

[9]

Gottfried Graber, Thomas Pock, and Horst Bischof. 2011. Online 3D reconstruction using convex optimization. In 2011 IEEE International Conference on Computer Vision Workshops. 708–711.

[10]

Chunzhao Guo, Kiyosumi Kidono, Junichi Meguro, Yoshiko Kojima, Masaru Ogawa, and Takashi Naito. 2016. A low-cost solution for automatic lane-level map generation using conventional in-car sensors. IEEE Transactions on Intelligent Transportation Systems 17, 8 (2016), 2355–2366.

Digital Library

[11]

Yijia He, Ji Zhao, Yue Guo, Wenhao He, and Kui Yuan. 2018. PL-VIO: Tightly-coupled monocular visual-inertial odometry using point and line features. Sensors 18, 4 (2018), 1159.

[12]

Manuel Hofer, Michael Maurer, and Horst Bischof. 2015. Line3D: Efficient 3D scene abstraction for the built environment. In German Conference on Pattern Recognition. 237–248.

Digital Library

[13]

Georg Klein and D. W. Murray. 2007. Parallel tracking and mapping for small AR workspaces. In 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. 225–234.

Digital Library

[14]

Xianglong Kong, Wenqi Wu, Lilian Zhang, and Yujie Wang. 2015. Tightly-coupled stereo visual-inertial navigation using point and line features. Sensors 15, 6 (2015), 12816–12833.

[15]

Stefan Leutenegger, Simon Lynen, Michael Bosse, Roland Siegwart, and Paul Furgale. 2014. Keyframe-based visual-inertial odometry using nonlinear optimization. International Journal of Robotics Research 34, 3 (2014), 314–334.

Digital Library

[16]

Xiaohu Lin, Fuhong Wang, Lei Guo, and Wanwei Zhang. 2019. An automatic key-frame selection method for monocular visual odometry of ground vehicle. IEEE Access 7 (2019), 70742–70754.

[17]

Yanqing Liu, Dongdong Yang, Jiamao Li, Yuzhang Gu, Jiatian Pi, and Xiaolin Zhang. 2018. Stereo visual-inertial SLAM with points and lines. IEEE Access 6 (2018), 69381–69392.

[18]

Lupton and Sukkarieh. 2012. Visual-inertial-aided navigation for high-dynamic motion in built environments without initial conditions. IEEE Transactions on Robotics 28, 1 (2012), 61–76.

Digital Library

[19]

Agostino Martinelli. 2012. Vision and IMU Data Fusion: Closed-form solutions for attitude, speed, absolute scale and bias determination. IEEE Transactions on Robotics 28, 1 (2012), 44–60.

Digital Library

[20]

Chen Min, Yuanwen Tian, Giancarlo Fortino, Zhang Jing, and Iztok Humar. 2018. Cognitive internet of vehicles. Computer Communications (2018), 58–76.

Digital Library

[21]

Anastasios I. Mourikis and Stergios I. Roumeliotis. 2007. A multi-state constraint Kalman filter for vision-aided inertial navigation. In 2007 IEEE International Conference on Robotics and Automation. 3565–3572.

[22]

Raul Mur-Artal and Juan Domingo Tardos. 2016. ORB-SLAM2: An open-source SLAM system for monocular, stereo and RGB-D cameras. IEEE Transactions on Robotics PP, 99 (2016), 1255–1262.

[23]

Raul Mur-Artal and Juan Domingo Tardos. 2016. Visual-inertial monocular SLAM with map reuse. IEEE Robotics & Automation Letters 2, 2 (2016), 796–803.

[24]

Raul Murartal, J. M. M. Montiel, and Juan D. Tardos. 2017. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics 31, 5 (2017), 1147–1163.

Digital Library

[25]

Daiduong Nguyen, Abdelhafid Elouardi, Sergio Alberto Rodriguez Florez, and Samir Bouaziz. 2018. HOOFR SLAM System: An embedded vision SLAM algorithm and its hardware-software mapping-based intelligent vehicles applications. IEEE Transactions on Intelligent Transportation Systems (2018), 1–16.

[26]

Taih Pire, Thomas Fischer, Gastn Castro, Pablo Decristforis, Javier Civera, and Julio Jacoboberlles. 2017. S-PTAM: Stereo parallel tracking and mapping. Robotics & Autonomous Systems 93 (2017), 27–42.

Digital Library

[27]

Matia Pizzoli, Christian Forster, and Davide Scaramuzza. 2014. REMODE: Probabilistic, monocular dense reconstruction in real time. In 2014 IEEE International Conference on Robotics & Automation. 2609–2616.

[28]

Pedro F. Proena and Yang Gao. 2018. Probabilistic RGB-D odometry based on points, lines and planes under depth uncertainty. Robotics & Autonomous Systems 104 (2018), 25–39.

[29]

Albert Pumarola, Alexander Vakhitov, Antonio Agudo, Alberto Sanfeliu, and Francese Morenonoguer. 2017. PL-SLAM: Real-time monocular visual SLAM with points and lines. In 2017 IEEE International Conference on Robotics and Automation. 4503–4508.

[30]

Tong Qin, Peiliang Li, and Shaojie Shen. 2017. VINS-Mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics PP, 99 (2017), 1–17.

Digital Library

[31]

Grompone Von Gioi Rafael, Jakubowicz Jĺȩrĺȩmie, Morel Jean-Michel, and Randall Gregory. 2010. LSD: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis & Machine Intelligence 32, 4 (2010), 722–732.

Digital Library

[32]

Milad Ramezani and Kourosh Khoshelham. 2018. Vehicle positioning in GNSS-deprived urban areas by stereo visual-inertial odometry. IEEE Transactions on Intelligent Vehicles 3, 2 (2018), 208–217.

[33]

Jan Sthmer, Stefan Gumhold, and Daniel Cremers. 2010. Real-time dense geometry from a handheld camera. In Dagm Conference on Pattern Recognition. 11–20.

Digital Library

[34]

Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. 2012. A benchmark for the evaluation of RGB-D SLAM systems. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. 573–580.

[35]

Fangchun Yang, Shangguang Wang, Jinglin Li, Zhihan Liu, and Qibo Sun. 2014. An overview of internet of vehicles. China Communications 11, 10 (2014), 1–15.

[36]

Shichao Yang and Sebastian Scherer. 2017. Direct monocular odometry using points and lines. In 2017 IEEE International Conference on Robotics and Automation. 3871–3877.

[37]

Hongsheng Yu and Anastasios I. Mourikis. 2017. Edge-based visual-inertial odometry. In 2017 IEEE/ RSJ International Conference on Intelligent Robots and Systems. 6670–6677.

[38]

Guoxuan Zhang, Han Lee Jin, Jongwoo Lim, and Il Hong Suh. 2015. Building a 3-D line-based map using stereo SLAM. IEEE Transactions on Robotics 31, 6 (2015), 1364–1377.

Digital Library

[39]

Xingxing Zuo, Xiaojia Xie, Yong Liu, and Guoquan Huang. 2017. Robust visual SLAM with point and line features. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. 1775–1782.

Cited By

El-Sofany HEl-Seoud SKaram OBouallegue B(2024)Using machine learning algorithms to enhance IoT system securityScientific Reports10.1038/s41598-024-62861-y14:1Online publication date: 27-May-2024
https://doi.org/10.1038/s41598-024-62861-y
Li YZhong WCai YChen LWang HKrólczyk GLi Z(2024)A new visual sensing system for motion state estimation of lateral localization of intelligent vehiclesMeasurement10.1016/j.measurement.2024.115212237(115212)Online publication date: Sep-2024
https://doi.org/10.1016/j.measurement.2024.115212
Pang WXia CLeong BAhmad FPaek JGovindan RCosta XAl Hassanieh HAsadi ACox LPerino DWidmer JGiustiniano D(2023)UbiPose: Towards Ubiquitous Outdoor AR Pose Tracking using Aerial MeshesProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3613263(1-16)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1145/3570361.3613263
Show More Cited By

Index Terms

Semi-Direct Monocular Visual-Inertial Odometry Using Point and Line Features for IoV
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Tracking
  2. Symbolic and algebraic manipulation
    1. Symbolic and algebraic algorithms
      1. Optimization algorithms
2. Networks
  1. Network services
    1. Location based services

Recommendations

Monocular visual-inertial odometry leveraging point-line features with structural constraints
Abstract
Structural geometry constraints, such as perpendicularity, parallelism and coplanarity, are widely existing in man-made scene, especially in Manhattan scene. By fully exploiting these structural properties, we propose a monocular visual-inertial ...
Direct visual-inertial odometry with semi-dense mapping
Abstract
The paper presents a direct visual-inertial odometry system. In particular, a tightly coupled nonlinear optimization based method is proposed by integrating the recent development in direct dense visual tracking of camera and the ...
ImPL-VIO: An Improved Monocular Visual-Inertial Odometry Using Point and Line Features
Intelligent Robotics and Applications
Abstract
Most of the visual-inertial navigation systems (VINS) that use only point features usually work well in regular environment, but decay in low-texture scenes. Meanwhile, those systems rarely construct environmental map with structural information. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology

ACM Transactions on Internet Technology Volume 22, Issue 1

February 2022

717 pages

ISSN:1533-5399

EISSN:1557-6051

DOI:10.1145/3483347

Editor:
Ling Liu
Georgia Institute of Technology, USA

Issue’s Table of Contents

Copyright © 2021 Association for Computing Machinery.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2021

Accepted: 01 October 2020

Revised: 01 September 2020

Received: 01 June 2020

Published in TOIT Volume 22, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China
Excellent Scientific and Technological Innovation Teams of Jiangxi Province of China
Natural Science Foundation of Jiangxi Province of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
306
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)3

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

El-Sofany HEl-Seoud SKaram OBouallegue B(2024)Using machine learning algorithms to enhance IoT system securityScientific Reports10.1038/s41598-024-62861-y14:1Online publication date: 27-May-2024
https://doi.org/10.1038/s41598-024-62861-y
Li YZhong WCai YChen LWang HKrólczyk GLi Z(2024)A new visual sensing system for motion state estimation of lateral localization of intelligent vehiclesMeasurement10.1016/j.measurement.2024.115212237(115212)Online publication date: Sep-2024
https://doi.org/10.1016/j.measurement.2024.115212
Pang WXia CLeong BAhmad FPaek JGovindan RCosta XAl Hassanieh HAsadi ACox LPerino DWidmer JGiustiniano D(2023)UbiPose: Towards Ubiquitous Outdoor AR Pose Tracking using Aerial MeshesProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3613263(1-16)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1145/3570361.3613263
Liu HTian LDu QXu W(2023)Robust RGB: D-SLAM in highly dynamic environments based on probability observations and clustering optimizationMeasurement Science and Technology10.1088/1361-6501/ad0afd35:3(035405)Online publication date: 11-Dec-2023
https://doi.org/10.1088/1361-6501/ad0afd
Liu SWu FTang JLi B(2023)A Medical Image Segmentation Method Based on Residual Network and Channel Attention MechanismMachine Learning for Cyber Security10.1007/978-3-031-20099-1_5(54-60)Online publication date: 13-Jan-2023
https://doi.org/10.1007/978-3-031-20099-1_5
Song KLi JQiu RYang G(2022)Monocular Visual-Inertial Odometry for Agricultural EnvironmentsIEEE Access10.1109/ACCESS.2022.320918610(103975-103986)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3209186
Yu HLi ZBi CChen H(2022)An effective deep learning method with multi-feature and attention mechanism for recognition of Chinese rice variety informationMultimedia Tools and Applications10.1007/s11042-022-12458-281:11(15725-15745)Online publication date: 1-May-2022
https://dl.acm.org/doi/10.1007/s11042-022-12458-2
Li BLiu SWu FLi GZhong MGuan X(2022)RT‐UnetInternational Journal of Intelligent Systems10.1002/int.2295637:11(8565-8582)Online publication date: 26-Sep-2022
https://dl.acm.org/doi/10.1002/int.22956
Guillen IMontalvo F(2021)Human-Centered Modeling Applications In Intelligent Manufacturing SystemsProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132165129265:1(838-842)Online publication date: 12-Nov-2021
https://doi.org/10.1177/1071181321651292

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents