Skip to main content
SpringerLink
Log in

Interactive 3D Modeling from Multiple Images Using Scene Regularities

  • Conference paper
  • First Online:
3D Structure from Multiple Images of Large-Scale Environments (SMILE 1998)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1506))

Abstract

We present some recent progress in designing and implementing two interactive image-based 3D modeling systems.

The first system constructs 3D models from a collection of panoramic image mosaics. A panoramic mosaic consists of a set of images taken around the same viewpoint, and a camera matrix associated with each input image. The user first interactively specifies features such as points, lines, and planes. Our system recovers the camera pose for each mosaic from known line directions and reference points. It then constructs the 3D model using all available geometrical constraints.

The second system extracts structure from stereo by representing the scene as a collection of approximately planar layers. The user first interactively segments the images into corresponding planar regions. Our system recovers a composite mosaic for each layer, estimates the plane equation for the layer, and optionally recovers the camera locations as well as out-of-plane displacements.

By taking advantage of known scene regularities, our interactive systems avoid difficult feature correspondence problems that occur in traditional automatic modeling systems. They also shift the interactive high-level structural model specification stage to precede (or intermix with) the 3D geometry recovery. They are thus able to extract accurate wire frame and texture-mapped 3D models from multiple image sequences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. S. Ayer, P. Schroeter, and J. Bigün. Segmentation of moving objects by robust parameter estimation over multiple frames. In 3rd ECCV, pages 316–327, 1994.

    Google Scholar 

  2. A. Azarbayejani and A. P. Pentland. Recursive estimation of motion, structure, and focal length. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(6):562–575, June 1995.

    Google Scholar 

  3. S. Baker, R. Szeliski, and P. Anandan. A layered approach to stereo reconstruction. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’98), pages 434–441, Santa Barbara, June 1998.

    Google Scholar 

  4. P. Beardsley, P. Torr, and A. Zisserman. 3D model acquisition from extended image sequences. In Fourth European Conference on Computer Vision (ECCV’96), volume 2, pages 683–695, Cambridge, England, April 1996. Springer-Verlag.

    Google Scholar 

  5. J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani. Hierarchical modelbased motion estimation. In 2nd ECCV, pages 237–252, 1992.

    Google Scholar 

  6. M. J. Black and A. D. Jepson. Estimating optical flow in segmented images using variable-order parametric models with local deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(10):972–986, October 1996.

    Google Scholar 

  7. J. F. Blinn. Jim Blinn’s corner: Compositing, part 1: Theory. IEEE Computer Graphics and Applications, 14(5):83–87, September 1994.

    Google Scholar 

  8. S. Bougnoux and L. Robert. Totalcalib: a fast and reliable system for off-line calibration of image sequences. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’97), June 1997. The Demo Session.

    Google Scholar 

  9. R. T. Collins and R. S. Weiss. Vanishing point calculation as a statistical inference on the unit sphere. In Third International Conference on Computer Vision (ICCV’90), pages 400–403, Osaka, Japan, December 1990. IEEE Computer Society Press.

    Google Scholar 

  10. T. Darrell and A. Pentland. Robust estimation of a multi-layered motion representation. In IEEE Workshop on Visual Motion, pages 173–178, Princeton, New Jersey, October 1991. IEEE Computer Society Press.

    Google Scholar 

  11. P. E. Debevec, C. J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. Computer Graphics (SIGGRAPH’96), pages 11–20, August 1996.

    Google Scholar 

  12. O. Faugeras. Three-dimensional computer vision: A geometric viewpoint. MIT Press, Cambridge, Massachusetts, 1993.

    Google Scholar 

  13. O. D. Faugeras. What can be seen in three dimensions with an uncalibrated stereo rig? In Second European Conference on Computer Vision (ECCV’92), pages 563–578, Santa Margherita Liguere, Italy, May 1992. Springer-Verlag.

    Google Scholar 

  14. O. D. Faugeras, Laveau S., Robert L., Csurka G., and Zeller C. 3-D reconstruction of urban scenes from sequences of images. Computer Vision and Image Understanding, 69(3):292–309, March 1998.

    Google Scholar 

  15. G. Golub and C. F. Van Loan. Matrix Computation, third edition. The John Hopkins University Press, Baltimore and London, 1996.

    Google Scholar 

  16. R. I. Hartley. Self-calibration from multiple views of a rotating camera. In Third European Conference on Computer Vision (ECCV’94), volume 1, pages 471–478, Stockholm, Sweden, May 1994. Springer-Verlag.

    Google Scholar 

  17. M. Irani, P. Anandan, and S. Hsu. Mosiac based representations of video sequences and their applications. In 5th ICCV, pages 605–611, 1995.

    Google Scholar 

  18. A. Jepson and M. J. Black. Mixture models for optical flow computation. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’93), pages 760–761, New York, New York, June 1993.

    Google Scholar 

  19. S. B. Kang and R. Szeliski. 3-D scene data recovery using omnidirectional multibaseline stereo. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’96), pages 364–370, San Francisco, Cal, June 1996.

    Google Scholar 

  20. R. Kumar, P. Anandan, and K. Hanna. Direct recovery of shape from multiple views: A parallax based approach. In Twelfth International Conference on Pattern Recognition (ICPR’94), volume A, pages 685–688, Jerusalem, Israel, October 1994. IEEE Computer Society Press.

    Google Scholar 

  21. Q.-T. Luong and O. Faugeras. Determining the fundamental matrix with planes: Instability and new algorithms. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’93), pages 489–494, New York, New York, June 1993.

    Google Scholar 

  22. L. McMillan and G. Bishop. Plenoptic modeling: An image-based rendering system. Computer Graphics (SIGGRAPH’95), pages 39–46, August 1995.

    Google Scholar 

  23. R. Mohr, L. Veillon, and L. Quan. Relative 3D reconstruction using multiple uncalibrated images. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’93), pages 543–548, New York, N.Y., June 1993.

    Google Scholar 

  24. M. Pollefeys, R. Koch, and L. Van Gool. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. In Sixth International Conference on Computer Vision (ICCV’98), pages 90–95, Bombay, January 1998.

    Google Scholar 

  25. H. S. Sawhney and A. R. Hanson. Identification and 3D description of’ shallow’ environmental structure over a sequence of images. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’91), pages 179–185, Maui, Hawaii, June 1991. IEEE Computer Society Press.

    Google Scholar 

  26. H.S. Sawhney and S. Ayer. Compact representations fo videos through dominant and multiple motion estimation. PAMI, 18(8):814–830, 1996.

    Google Scholar 

  27. S. M. Seitz and C. M. Dyer. Photorealistic scene reconstrcution by space coloring. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’97), pages 1067–1073, San Juan, Puerto Rico, June 1997.

    Google Scholar 

  28. J. Shade, S. Gortler, L.-W. He, and R. Szeliski. Layered depth images. In Computer Graphics (SIGGRAPH’98) Proceedings, Orlando, July 1998. ACM SIGGRAPH.

    Google Scholar 

  29. A. Shashua. Projective structure from uncalibrated images: Structure from motion and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(8):778–790, August 1994.

    Google Scholar 

  30. H.-Y. Shum, M. Han, and R. Szeliski. Interactive construction of 3d models from panoramic mosaics. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’98), pages 427–433, Santa Barbara, June 1998.

    Google Scholar 

  31. H.-Y. Shum and R. Szeliski. Panoramic image mosaicing. Technical Report MSRTR-97-23, Microsoft Research, September 1997.

    Google Scholar 

  32. A. R. Smith and J. F. Blinn. Blue screen matting. In Computer Graphics Proceedings, Annual Conference Series, pages 259–268, Proc. SIGGRAPH’96 (New Orleans), August 1996. ACM SIGGRAPH.

    Google Scholar 

  33. R. Szeliski and P. Golland. Stereo matching with transparency and matting. In Sixth International Conference on Computer Vision (ICCV’98), pages 517–524, Bombay, January 1998.

    Google Scholar 

  34. R. Szeliski and H.-Y. Shum. Creating full view panoramic image mosaics and texture-mapped models. Computer Graphics (SIGGRAPH’97), pages 251–258, August 1997.

    Google Scholar 

  35. R. Szeliski and P. Torr. Geometrically constrained structure from motion: Points on planes. In European Workshop on 3D Structure from Multiple Images of Large-scale Environments (SMILE), Freiburg, Germany, June 1998.

    Google Scholar 

  36. J. Torborg and J. T. Kajiya. Talisman: Commodity realtime 3D graphics for the PC. In Computer Graphics Proceedings, Annual Conference Series, pages 353–363, Proc. SIGGRAPH’96 (New Orleans), August 1996. ACM SIGGRAPH.

    Google Scholar 

  37. P. Torr, A. W. Fitzgibbon, and A. Zisserman. Maintaining multiple motion model hypotheses over many views to recover matching structure. In Sixth International Conference on Computer Vision (ICCV’98), pages 485–491, Bombay, January 1998.

    Google Scholar 

  38. T. Viéville, C. Zeller, and L. Robert. Using collineations to compute motion and structure in an uncalibrated image sequence. International Journal of Computer Vision, 20(3):213–242, 1996.

    Article  Google Scholar 

  39. E. L. Walker and M. Herman. Geometric reasoning for constructing 3D scene descriptions from images. Artificial Intelligence, 37:275–290, 1988.

    Article  Google Scholar 

  40. J. Y. A. Wang and E. H. Adelson. Layered representation for motion analysis. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’93), pages 361–366, New York, New York, June 1993.

    Google Scholar 

  41. Y. Weiss. Smoothness in layers: Motion segmentation using nonparametric mixture estimation. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’97), pages 520–526, San Juan, Puerto Rico, June 1997.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Microsoft Research, USA

    Heung-Yeung Shum, Richard Szeliski & P. Anandan

  2. Columbia University, USA

    Simon Baker

  3. Carnegie Mellon University, USA

    Mei Han

Authors
  1. Heung-Yeung Shum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard Szeliski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simon Baker
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mei Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. P. Anandan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. ESAT-PSI, Katholieke Universiteit Leuven, Kardinaal Mercilaan 9, B-3001, Leuven, Belgium

    Reinhard Koch  & Luc Van Gool  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Shum, HY., Szeliski, R., Baker, S., Han, M., Anandan, P. (1998). Interactive 3D Modeling from Multiple Images Using Scene Regularities. In: Koch, R., Van Gool, L. (eds) 3D Structure from Multiple Images of Large-Scale Environments. SMILE 1998. Lecture Notes in Computer Science, vol 1506. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49437-5_16

Download citation

  • DOI: https://doi.org/10.1007/3-540-49437-5_16

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-65310-3

  • Online ISBN: 978-3-540-49437-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation