Skip to main content
SpringerLink
Log in

Photorealistic 3D reconstruction from handheld cameras

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

One of the major challenges in the fields of computer vision and computer graphics is the construction and representation of life-like virtual 3D scenarios within a computer. The VISIRE project attempts to reconstruct photo-realistic 3D models of large scenarios using as input multiple freehand video sequences, while rendering the technology accessible to the non-expert.

VISIRE is application oriented and hence must deal with multiple issues of practical relevance that were commonly overlooked in past experiences. The paper presents both an innovative approach for the integration of previously unrelated experiences, as well as a number of novel contributions, such as: an innovative algorithm to enforce closedness of the trajectories, a new approach to 3D mesh generation from sparse data, novel techniques dealing with partial occlusions and a method for using photo-consistency and visibility constrains to refine the 3D mesh.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Rodríguez, T., Sturm, P., Heyden, A., Menéndez, J.M., et al.: Visire. photorealistic 3d reconstruction from video sequences. In: IEEE International Conference on Image Processing, pp. 705–708. Barcelona, Spain (2003)

  2. Pollefeys, M., Gool, L.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. IJCV 59(3), 207–232 (2004)

    Article  Google Scholar 

  3. Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Proceedings of the 23rd Conference on Computer graphics and Interactive Techniques, pp. 43–54 (1996)

  4. Matusik, W., Pfister, H., Ngan, A., Beardsley, P., Ziegler, R., McMillan, L.: Image-based 3d photography using opacity hulls. In: Proceedings of the ACM SIGGRAPH 2002, p. 427–437 (2002)

  5. Takashi Machida, H.T.: Dense estimation of surface reflectance properties based on inverse global illumination rendering. In: ICPR'04, vol. 2, pp. 895–898. Cambridge, UK (2004)

  6. Shi, J.C.T.: Good features to track. In: CVPR'94, pp. 593–600 (1994)

  7. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge, UK (2000)

    Google Scholar 

  8. Heyden, A., Åström, K.: Euclidean reconstruction from image sequences with varying and unknown focal length and principal point. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 438–443 (1997)

  9. Triggs, W., McLauchlan, P., Hartley, R., Fitzgibbon, A.: Bundle adjustment: A modern synthesis. In: Vision Algorithms: Theory and Practice. Springer, Berlin Heidelberg New York (2000)

    Google Scholar 

  10. Fitzgibbon, A., Zisserman, A.: Automatic camera recovery for closed or open image sequences. In: Proceedings of the European Conference on Computer Vision, vol. I, pp. 311–326. Freiburg, Germany (1998)

  11. Triggs, B., McLauchlan, P., Ri, H., Fitzgibbon, A.: Bundle adjustment—a modern synthesis. In: Vision Algorithms'99, pp. 298–372. in conjunction with ICCV'99, Kerkyra, Greece (1999)

  12. Reid, I., Murray, D.: Active tracking of foveated feature clusrters using affine structure. In: International Journal of Computer Vision, pp. 41–60. Seattle, WA (1996)

  13. Sturm, P., Triggs, B.: A factorization based algorithm for multi-image projective structure and motion. In: Buxton, B., Cipolla, R. (eds.) Computer Vision – ECCV'96, Lecture Notes in Computer Science, vol. 1065, pp. 709–720. Springer, Berlin Heidelberg New York (1996)

    Google Scholar 

  14. Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992)

    Article  Google Scholar 

  15. Guilbert, N., Bartoli, A.: Batch recovery of multiple views with missing data using direct sparse solvers. In: British Machine Vision Conference. Norwich, UK (2003)

  16. Triggs, B.: Linear projective reconstruction from matching tensors. Image Vis. Comput. 15(8), 617–625 (1997)

    Article  Google Scholar 

  17. Hoppe, H.: Surface reconstruction from unorganized points. Ph.D. thesis, Department of Computer Science and Engineering, University of Washington (1994)

  18. Petitjean, S., Boyer, E.: Regular and non-regular point sets: Properties and reconstruction. Comput. Geom.—Theor. Appl. 19 (2001)

  19. Manessis, A., Hilton, A., Palmer, P., McLauchlan, P., Shen, X.: Reconstruction of scene models from sparse 3d structure. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. Hilton Head, USA (2000)

  20. Morris, D., Kanade, T.: Image-consistent surface triangulation. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. Hilton Head, USA (2000)

  21. Huber, P.: Robust Statistics. Wiley, New York (1981)

    Google Scholar 

  22. Press, W., Flannery, B., Teukolsky, S., Vetterling, W.: Numerical Recipes in C. Cambridge University Press, Cambridge, UK (1988)

    Google Scholar 

  23. Strecha, C., Fransens, R., Gool, L.V.: Wide-baseline stereo from multiple views: a probabilistic account. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 552–559 Washington, DC (2004)

Download references

Author information

Authors and Affiliations

  1. Eptron SA. R&D Dpt., Madrid, Spain

    Tomás Rodríguez

  2. INRIA Rhône-Alpes Montbonnot, France

    Peter Sturm & Pau Gargallo

  3. Centre for Mathematical Science, Lund University, Sweden

    Nicolas Guilbert & Anders Heyden

  4. E.T.S.I. Telecomunicaciones, Universidad Politécnica de Madrid, Spain

    Fernando Jauregizar, J. M. Menéndez & J. I. Ronda

Authors
  1. Tomás Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Sturm
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pau Gargallo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicolas Guilbert
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Anders Heyden
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fernando Jauregizar
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. J. M. Menéndez
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. J. I. Ronda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomás Rodríguez.

Additional information

Tomas Rodriguez was born in Madrid in 1961. Bachelor in Physics and Master in Electronics by the Universidad Complutense de Madrid. He started his career in the private R&D sector in 1987, when he specialized in computer vision and parallel processing systems. In the early nineties he participated in the EVA project; one of the most outstanding traffic monitoring system of the time. For more than 10 years, he has been involved in international research projects within the ambit of EUREKA, ESPRIT, V and VI Framework programmes. During this time, he coordinated eight international projects (CAMELOT, CITRONE, ON-LIVE, SAID, VISIRE, EVENTS, ITALES, HOLONICS) and acted as principal investigator in two additional ones (CITRUS and VICTORIA). Since the early days, he had the opportunity to collaborate with some of the most prestigious research institutions in Europe: Franhoufer Inst., INRIA, CNRS, University of Oxford, University of Lund, DFKI, Siemens C-Lab, Philips Research Labs, etc. Evaluator of R&D projects for the Spanish Ministry for Science and reviewer of international scientific journals, he is currently the R&D manager and coordinator for European projects at Eptron SA. His recent interests include: computer vision, real time software, industrial control, parallel processing, iTV, and mobile technologies, etc.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rodríguez, T., Sturm, P., Gargallo, P. et al. Photorealistic 3D reconstruction from handheld cameras. Machine Vision and Applications 16, 246–257 (2005). https://doi.org/10.1007/s00138-005-0179-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-005-0179-4

Keywords

Search

Navigation