Skip to main content
SpringerLink
Log in

Fast and automatic city-scale environment modelling using hard and/or weak constrained bundle adjustments

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

To provide high-quality augmented reality service in a car navigation system, accurate 6 degrees of freedom (DoF) localization is required. To ensure such accuracy, most current vision-based solutions rely on an off-line large-scale modelling of the environment. Nevertheless, while existing solutions to model the environment require expensive equipments and/or a prohibitive computation time, we propose in this paper a complete framework that automatically builds an accurate large-scale database of landmarks using only a standard camera, a low-cost global positioning system (GPS) and a geographic information system (GIS). As illustrated in the experiments, only few minutes are required to model large-scale environments. The resulting databases can then be used by an on-line localization algorithm to ensure high-quality augmented reality experiences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. The DEM used is a geometric model where each road is presented by a plane.

  2. The 3D building models used are a geometric model where each facade is presented by a plane.

  3. GPU is not used.

  4. Several clicks are performed for each building corner. Then, the mean of these clicks is considered as a ground truth.

References

  1. Irschara, A., Zach, C., Frahm, J.M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: CVPR (2009)

  2. Li, Y., Snavely, N., Huttenlocher, D., Fua, P.: Worldwide pose estimation using 3d point clouds. In: ECCV (2012)

  3. Zamir, A.R., Shah, M.: Accurate image localization based on google maps street view. In: ECCV (2010)

  4. Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., Sayd, P.: Real time localization and 3D reconstruction. In: CVPR (2006)

  5. Taneja, A., Ballan, L., Pollefeys, M.: Registration of spherical panoramic images with cadastral 3D models. In: 3DIMPVT (2012)

  6. Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S.M., Szeliski, R.: Building Rome in a day. Commun. ACM 54, 105–112 (2011)

    Article  Google Scholar 

  7. Frahm, J.M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Y. H. Jen, E. Dunn, B. Clipp, S. Lazebnik, M. Pollefeys, Building Rome on a cloudless day. In: ECCV (2010)

  8. Klingner, B., Martin, D., Roseborough, J.: Street view motion-from-structure-from-motion. In: ICCV (2013)

  9. Kaminsky, R.S., Snavely, N., Seitz, S.M., Szeliski, R.: Alignment of 3D point clouds to overhead images. In: CVPR Workshop (2009)

  10. Strecha, C., Pylvänäinen, T., Fua, P.: Dynamic and scalable large scale image reconstruction. In: CVPR (2010)

  11. Pylvänäinen, T., Roimela, K., Vedantham, R., Itäranta, J., Wang, R., Grzeszczuk, R.: Automatic alignment and multi-view segmentation of street view data using 3D shape priors. In: 3DPVT (2010)

  12. Lothe, P., Bourgeois, S., Dekeyser, F., Royer, E., Dhome, M.: Towards geographical referencing of monocular SLAM reconstruction using 3D city models: application to real-time accurate vision-based localization. In: CVPR (2009)

  13. Larnaout, D., Bourgeois, S., Gay-Bellile, V., Dhome, M.: Towards bundle adjustment with GIS constraints for online geo-localization of a vehicle in urban center. In: 3DIMPVT (2012)

  14. Hideyuki, K., Takafumi, T., Tomokazu, S., Naokazu, Y.: Extrinsic camera parameter estimation using video images and GPS considering GPS positioning accuracy. In: ICPR (2010)

  15. Michot, J., Bartoli, A., Gaspard, F.: Bi-objective bundle adjustment with application to multi-sensor SLAM. In: 3DPVT (2010)

  16. Lhuillier, M.: Incremental fusion of structure-from-motion and GPS using constrained bundle adjustments. PAMI 34, 2489–2495 (2012)

  17. Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Indus. Appl. Math. 11, 431–441 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  18. Recky, M., Wendel, A., Leberl, F.: Facade segmentation in a multi-view scenario. In: 3DIMPVT (2011)

  19. Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: ECCV (2008)

  20. Tamaazousti, M., Gay-Bellile, V., Naudet-Collette, S., Bourgeois, S., Dhome, M.: NonLinear refinement of structure from motion reconstruction by taking advantage of a partial knowledge of the environment. In: CVPR (2011)

  21. Laneurit, J., Chapuis, R., Chausse, F.: Accurate vehicle positioning on a numerical map. Int. J. Control Autom. Syst. 3, 15–31 (2005)

    MATH  Google Scholar 

  22. Jo, K., Chu, K., Sunwoo, M.: GPS-bias correction for precise localization of autonomous vehicles. In: IV (2013)

  23. Lee, D.D., Vernaza, P.: Robust GPS/INS-aided localization and mapping via GPS bias estimation. In: ISER (2006)

  24. Arth, C., Wagner, D., Klopschitz, M., Irschara, A., Schmalstieg, D.: Wide area localization on mobile phones. In: ISMAR (2009)

  25. Dong, Z., Zhang, G., Jia, J., Bao, H.: Keyframe-based real-time camera tracking. In: ICCV (2009)

  26. Tong, G., Wu, Z., Weng, N., Hou, W.: An omni-directional vSLAM based on spherical camera model and 3D modeling. in: World Congress on Intelligent Control and Automation (2012)

Download references

Author information

Authors and Affiliations

  1. CEA, LIST, LVIC, Point Courrier 173, 91191, Gif-Sur-Yvette, France

    Dorra Larnaout, Vincent Gay-Bellile & Steve Bourgeois

  2. Institut Pascal, UMR 6602 Université Blaise Pascal/CNRS/IFMA, Clermont-Ferrand, France

    Dorra Larnaout & Michel Dhome

Authors
  1. Dorra Larnaout
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vincent Gay-Bellile
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steve Bourgeois
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michel Dhome
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dorra Larnaout.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Larnaout, D., Gay-Bellile, V., Bourgeois, S. et al. Fast and automatic city-scale environment modelling using hard and/or weak constrained bundle adjustments. Machine Vision and Applications 27, 943–962 (2016). https://doi.org/10.1007/s00138-016-0766-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-016-0766-6

Keywords

Search

Navigation