Skip to main content
SpringerLink
Log in

Enforcing Regularities between Planes Using Key Plane for Monocular Mesh-based VIO

  • Regular paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Enforcing the additional structural constraints to improve localization, the plane feature can express better environment information, and thereby is widely used in Visual Inertial Odometry (VIO) method. To enforce regularities between planes, this paper proposes a novel factor graph formulation based on key plane. Firstly, key plane selection criterion is adopted to filter trivial planes. Then structural regularities between each plane detected in previous steps and key plane are implemented by adding corresponding residual terms into optimization function. Furthermore, in addition to gravity, the normal directions of all key planes continually update actual buildings plane normal direction in the world frame, which is used for detecting plane. Finally, experiments was conducted on public dataset and results show that our proposed method obtains better accuracy than state-of-the-art algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ceres solver. http://ceres-solver.org Accessed April 26, 2021

  2. Evo package. https://github.com/MichaelGrupp/evo Accessed April 26, 2021

  3. Fade2d. http://www.geom.at/fade2d/html/annotated.html/http://www.geom.at/fade2d/html/annotated.html/Accessed April 26, 2021

  4. Ros kinetic. https://www.ros.org/ Accessed April 26, 2021

  5. Ataer-Cansizoglu, E., Taguchi, Y., Ramalingam, S., Garaas, T.: Tracking an rgb-d camera using points and planes. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops (2013)

  6. Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M.W., Siegwart, R.: The euroc micro aerial vehicle datasets. The International Journal of Robotics Research. https://doi.org/10.1177/0278364915620033 (2016)

  7. Civera, J., Davison, A.J., Montiel, J.M.M.: Inverse depth parametrization for monocular slam. IEEE Trans. Robot. 24(5), 932–945 (2008). https://doi.org/10.1109/TRO.2008.2003276

    Article  Google Scholar 

  8. Forster, C., Carlone, L., Dellaert, F., Scaramuzza, D.: On-manifold preintegration for real-time visual–inertial odometry. IEEE Trans. Robot. 33(1), 1–21 (2017). https://doi.org/10.1109/TRO.2016.2597321

    Article  Google Scholar 

  9. Fu, Q., Wang, J., Yu, H., Ali, I., Guo, F., He, Y., Zhang, H.: Pl-vins: Real-time monocular visual-inertial slam with point and line features. arXiv:2009.07462 (2020)

  10. Geneva, P., Eckenhoff, K., Lee, W., Yang, Y., Huang, G.: Openvins: a Research Platform for Visual-Inertial Estimation. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 4666–4672. https://doi.org/10.1109/ICRA40945.2020.9196524 (2020)

  11. Greene, W.N., Roy, N.: Flame: Fast Lightweight Mesh Estimation Using Variational Smoothing on Delaunay Graphs. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4696–4704. https://doi.org/10.1109/ICCV.2017.502 (2017)

  12. Gundogdu, E., Alatan, A.A.: Good features to correlate for visual tracking. IEEE Trans. Image Process. 27(5), 2526–2540 (2018). https://doi.org/10.1109/TIP.2018.2806280

    Article  MathSciNet  Google Scholar 

  13. He, Y., Zhao, J., Guo, Y., He, W., Yuan, K.: Pl-vio: Tightly-coupled monocular visual–inertial odometry using point and line features. Sensors 18(4). https://doi.org/10.3390/s18041159. https://www.mdpi.com/1424-8220/18/4/1159 (2018)

  14. Hsiao, M., Westman, E., Kaess, M.: Dense Planar-Inertial Slam with Structural Constraints. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6521–6528. https://doi.org/10.1109/ICRA.2018.8461094 (2018)

  15. Kaess, M.: Simultaneous Localization and Mapping with Infinite Planes. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 4605–4611. https://doi.org/10.1109/ICRA.2015.7139837 (2015)

  16. Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., Furgale, P.: Keyframe-based visual–inertial odometry using nonlinear optimization. Int. J. Robot. Res. 34(3), 314–334 (2015). https://doi.org/10.1177/0278364914554813

    Article  Google Scholar 

  17. Li, X., He, Y., Lin, J., Liu, X.: Leveraging planar regularities for point line visual-inertial odometry. arXiv:2004.11969 (2021)

  18. Li, X., Li, Y., Örnek, E.P., Lin, J., Tombari, F.: Co-planar parametrization for stereo-slam and visual-inertial odometry. IEEE Robot. Autom. Lett. 5(4), 6972–6979 (2020). https://doi.org/10.1109/LRA.2020.3027230

    Article  Google Scholar 

  19. Ling, Y., Shen, S.: Building Maps for Autonomous Navigation Using Sparse Visual Slam Features. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1374–1381. https://doi.org/10.1109/IROS.2017.8202316 (2017)

  20. Lu, Y., Song, D.: Visual navigation using heterogeneous landmarks and unsupervised geometric constraints. IEEE Trans. Robot. 31(3), 736–749 (2015). https://doi.org/10.1109/TRO.2015.2424032

    Article  Google Scholar 

  21. Ma, L., Kerl, C., Stückler, J., Cremers, D.: Cpa-Slam: Consistent Plane-Model Alignment for Direct Rgb-D Slam. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 1285–1291. https://doi.org/10.1109/ICRA.2016.7487260 (2016)

  22. Mourikis, A.I., Roumeliotis, S.I.: A multi-state constraint kalman filter for vision-aided inertial navigation. In: Proceedings 2007 IEEE International Conference on Robotics and Automation, pp. 3565–3572. https://doi.org/10.1109/ROBOT.2007.364024 (2007)

  23. Qin, T., Li, P., Shen, S.: Vins-mono: a robust and versatile monocular visual-inertial state estimator. IEEE Trans. Robot. 34(4), 1004–1020 (2018). https://doi.org/10.1109/TRO.2018.2853729

    Article  Google Scholar 

  24. Rosinol, A., Sattler, T., Pollefeys, M., Carlone, L.: Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 8220–8226. https://doi.org/10.1109/ICRA.2019.8794456 (2019)

  25. Taguchi, Y., Jian, Y.D., Ramalingam, S., Feng, C.: Point-Plane Slam for Hand-Held 3D Sensors. In: 2013 IEEE International Conference on Robotics and Automation, pp. 5182–5189. https://doi.org/10.1109/ICRA.2013.6631318 (2013)

  26. Teixeira, L., Chli, M.: Real-Time Mesh-Based Scene Estimation for Aerial Inspection. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4863–4869. https://doi.org/10.1109/IROS.2016.7759714 (2016)

  27. Yang, Y., Geneva, P., Eckenhoff, K., Huang, G.: Visual-Inertial Odometry with Point and Line Features. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2447–2454. https://doi.org/10.1109/IROS40897.2019.8967905 (2019)

  28. Yang, Y., Geneva, P., Zuo, X., Eckenhoff, K., Liu, Y., Huang, G.: Tightly-Coupled Aided Inertial Navigation with Point and Plane Features. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 6094–6100. https://doi.org/10.1109/ICRA.2019.8794078 (2019)

  29. Yang, Z., Shen, S.: Monocular visual–inertial state estimation with online initialization and camera–imu extrinsic calibration. IEEE Trans. Autom. Sci. Eng. 14(1), 39–51 (2017). https://doi.org/10.1109/TASE.2016.2550621

    Article  Google Scholar 

  30. Zhang, G., Lee, J.H., Lim, J., Suh, I.H.: Building a 3-d line-based map using stereo slam. IEEE Trans. Robot. 31(6), 1364–1377 (2015). https://doi.org/10.1109/TRO.2015.2489498

    Article  Google Scholar 

  31. Zhang, X., Liao, Z., Qi, X., Wang, W.: Stereo plane slam based on intersecting lines. arXiv:2008.08218 (2020)

  32. Zhang, X., Wang, W., Qi, X., Liao, Z., Wei, R.: Point-plane slam using supposed planes for indoor environments. Sensors 19(17). https://doi.org/10.3390/s19173795. https://www.mdpi.com/1424-8220/19/17/3795(2019)

  33. Zhou, H., Zou, D., Pei, L., Ying, R., Liu, P., Yu, W.: Structslam: Visual slam with building structure lines. IEEE Trans. Veh. Technol. 64(4), 1364–1375 (2015). https://doi.org/10.1109/TVT.2015.2388780

    Article  Google Scholar 

  34. Zou, D., Wu, Y., Pei, L., Ling, H., Yu, W.: Structvio: Visual-inertial odometry with structural regularity of man-made environments. IEEE Trans. Robot. 35(4), 999–1013 (2019). https://doi.org/10.1109/TRO.2019.2915140

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China (Grant No. 41764002).

Author information

Authors and Affiliations

  1. School of Information Engineering, Nanchang University, Nanchang, China

    Jianfeng Wu, Jian Xiong & Hang Guo

Authors
  1. Jianfeng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hang Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, J.W. and J.X.; methodology, J.W.; software, J.W.; validation, J.W., J.X. and H.G.; formal analysis, J.W.; investigation, J.W.; resources, J.W.; data curation, J.W.; writing—original draft preparation, J.W.; writing—review and editing, J.X.; visualization, J.W.; supervision, H.G.; project administration, H.G.; funding acquisition, H.G.

Corresponding author

Correspondence to Jian Xiong.

Ethics declarations

Consent for publication

All authors have read and agreed to the published version of the manuscript.

Conflict of Interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Consent to participate

All authors have given consent to participate.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, J., Xiong, J. & Guo, H. Enforcing Regularities between Planes Using Key Plane for Monocular Mesh-based VIO. J Intell Robot Syst 104, 6 (2022). https://doi.org/10.1007/s10846-021-01529-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-021-01529-5

Keywords

Search

Navigation