Abstract
We describe passive, multiple-view 3D imaging systems that recover 3D information from scenes that are illuminated only with ambient lighting. Much of the material is concerned with using the geometry of stereo 3D imaging to formulate estimation problems. Firstly, we present an overview of the common techniques used to recover 3D information from camera images. Secondly, we discuss camera modeling and camera calibration as an essential introduction to the geometry of the imaging process and the estimation of geometric parameters. Thirdly, we focus on 3D recovery from multiple views, which can be obtained using multiple cameras at the same time (stereo), or a single moving camera at different times (structure from motion). Epipolar geometry and finding image correspondences associated with the same 3D scene point are two key aspects for such systems, since epipolar geometry establishes the relationship between two camera views, while depth information can be inferred from the correspondences. The details of both stereo and structure from motion, the two essential forms of multiple-view 3D reconstruction technique, are presented. Towards the end of the chapter, we present several real-world applications.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
You may wish to compare l T x=0 to two well-known parameterizations of a line in the (x,y) plane, namely: ax+by+c=0 and y=mx+c and, in each case, write down homogeneous coordinates for the point x and the line l.
- 3.
We use a tilde to differentiate n-tuple inhomogeneous coordinates from (n+1)-tuple homogeneous coordinates.
- 4.
We need to use a variety of image coordinate normalizations in this chapter. For simplicity, we will use the same subscript n, but it will be clear about how the normalization is achieved.
- 5.
Skew models a lack of orthogonality between the two image sensor sampling directions. For most imaging situations it is zero.
- 6.
The same homogeneous image coordinates up to scale or the same inhomogeneous image coordinates.
- 7.
Due to the scale equivalence of homogeneous coordinates.
- 8.
No three points collinear.
- 9.
‘Approximately’, because of noise in the imaged corner positions supplied to the calibration process.
- 10.
Extrinsic parameters are always not known in a structure from motion problem, they are part of what we are trying to solve for. Intrinsic parameters may or may not be known, depending on the application.
- 11.
The length of the baseline is the magnitude of the extrinsic translation vector, t.
- 12.
There are several other approaches, such as the seven-point algorithm.
- 13.
An inlier is a putative correspondence that lies within some threshold of its expected position predicted by F. In other words image points must lie within a threshold from their epipolar lines generated by F.
- 14.
Bundle adjustment methods appeared several decades ago in the photogrammetry literature and are now used widely in the computer vision community.
- 15.
- 16.
References
Barfoot, T., Se, S., Jasiobedzki, P.: Vision-based localization and terrain modelling for planetary rovers. In: Howard, A., Tunstel, E. (eds.) Intelligence for Space Robotics, pp. 71–92. TSI Press, Albuquerque (2006)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. Comput. Vis. Image Underst. 110(3), 346–359 (2008)
Belhumeur, P.N.: A Bayesian approach to binocular stereopsis. Int. J. Comput. Vis. 19(3), 237–260 (1996)
Bergen, J.R., Anandan, P., Hanna, K.J., Hinogorani, R.: Hierarchical model-based motion estimation. In: European Conference on Computer Vision (ECCV), Italy, pp. 237–252 (1992)
Birchfield, S., Tomasi, C.: Depth discontinuities by pixel-to-pixel stereo. Int. J. Comput. Vis. 35(3), 269–293 (1999)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)
Brown, D.C.: Decentering distortion of lenses. Photogramm. Eng. 32(3), 444–462 (1966)
Brown, D.C.: Close-range camera calibration. Photogramm. Eng. 37(8), 855–866 (1971)
Camera Calibration Toolbox for MATLAB: http://www.vision.caltech.edu/bouguetj/calib_doc/. Accessed 20th October 2011
Chen, Z., Pears, N.E., Liang, B.: Monocular obstacle detection using Reciprocal-Polar rectification. Image Vis. Comput. 24(12), 1301–1312 (2006)
Cox, I.J., Hingorani, S.L., Rao, S.B., Maggs, B.M.: A maximum likelihood stereo algorithm. Comput. Vis. Image Underst. 63(3), 542–567 (1996)
Coxeter, H.S.M.: Projective Geometry, 2nd edn. Springer, Berlin (2003)
Criminisi, A., Reid, I., Zisserman, A.: Single view metrology. Int. J. Comput. Vis. 40(2), 123–148 (2000)
Davison, A., Reid, I., Molton, N., Stasse, O.: MonoSLAM: real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)
Faugeras, O., Luong, Q.T.: The Geometry of Multiple Images. MIT Press, Cambridge (2001)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Garding, J.: Shape from texture for smooth curved surfaces in perspective projection. J. Math. Imaging Vis. 2, 329–352 (1992)
Harris, C., Stephens, M.J.: A combined corner and edge detector. In: Alvey Vision Conference, pp. 147–152 (1988)
Harley, R.I.: In defense of the 8-point algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 19(6), 580–593 (1997)
Hartley, R.I.: Theory and practice of projective rectification. Int. J. Comput. Vis. 35(2), 115–127 (1999)
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)
Horn, B.K.P., Brooks, M.J. (eds.): Shape from Shading. MIT Press, Cambridge (1989)
Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)
Huang, R., P, S.W.A.: A shape-from-shading framework for satisfying data-closeness and structure-preserving smoothness constraints. In: Proceedings of the British Machine Vision Conference (2009)
Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: International Conference on Computer Vision (ICCV), Vancouver, pp. 508–515 (2001)
Konolige, K.: Small vision system: hardware and implementation. In: Proc. Int. Symp. on Robotics Research, Hayama, Japan, pp. 111–116 (1997)
Longuet-Higgins, H.C.: A computer algorithm for re-constructing a scene from two projections. Nature 293, 133–135 (1981)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Lucas, B.D., Kanade, T.: An interactive image registration technique with an application in stereo vision. In: International Joint Conference on Artificial Intelligence (IJCAI), Vancouver, pp. 674–679 (1981)
Ma, Y., Soatto, S., Kosecka, J., Sastry, S.S.: An Invitation to 3-D Vision. Springer, New York (2003)
Maimone, M., Biesiadecki, J., Tunstel, E., Cheng, Y., Leger, C.: Surface navigation and mobility intelligence on the Mars Exploration Rovers. In: Howard, A., Tunstel, E. (eds.) Intelligence for Space Robotics, pp. 45–69. TSI Press, Albuquerque (2006)
Maimone, M., Cheng, Y., Matthies, L.: Two years of visual odometry on the mars exploration rovers. J. Field Robot. 24(3), 169–186 (2007)
Mallon, J., Whelan, P.F.: Projective rectification from the fundamental matrix. In: Image and Vision Computing, pp. 643–650 (2005)
The Middlebury stereo vision page: http://vision.middlebury.edu/stereo/. Accessed 16th November 2011
Moons, T., Van Gool, L., Vergauwen, M.: 3D reconstruction from multiple images. Found. Trends Comput. Graph. Vis. 4(4), 287–404 (2010)
Mordohai, P., et al.: Real-time video-based reconstruction of urban environments. In: International Workshop on 3D Virtual Reconstruction and Visualization of Complex Architectures (3D-ARCH), Zurich, Switzerland (2007)
Nayar, S.K., Nakagawa, Y.: Shape from focus. IEEE Trans. Pattern Anal. Mach. Intell. 16(8), 824–831 (1994)
Nister, D.: Automatic passive recovery of 3D from images and video. In: International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), Thessaloniki, Greece, pp. 438–445 (2004)
Nister, D.: An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 756–770 (2004)
Open source computer vision library: http://opencv.willowgarage.com/wiki/. Accessed 20th October 2011
Pentland, A.P.: A new sense for depth of field. IEEE Trans. Pattern Anal. Mach. Intell. 9(4), 523–531 (1987)
Pollefeys, M., Koch, R., Van Gool, L.: A simple and efficient rectification method for general motion. In: International Conference on Compute Vision (ICCV), Kerkyra, Greece, pp. 496–501 (1999)
Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004)
Quam, L.H.: Hierarchical warp stereo. In: Image Understanding Workshop, New Orleans, pp. 149–155 (1984)
Roy, S., Cox, I.J.: A maximum-flow formulation of the N-camera stereo correspondence problem. In: International Conference on Computer Vision (ICCV), Bombay, pp. 492–499 (1998)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1/2/3), 7–42 (2002)
Se, S., Jasiobedzki, P.: Photo-realistic 3D model reconstruction. In: IEEE International Conference on Robotics and Automation, Orlando, Florida, pp. 3076–3082 (2006)
Se, S., Jasiobedzki, P.: Stereo-vision based 3D modeling and localization for unmanned vehicles. Int. J. Intell. Control Syst. 13(1), 47–58 (2008)
Se, S., Lowe, D., Little, J.: Mobile robot localization and mapping with uncertainty using scale-invariant visual landmarks. Int. J. Robot. Res. 21(8), 735–758 (2002)
Se, S., Firoozfam, P., Goldstein, N., Dutkiewicz, M., Pace, P.: Automated UAV-based video exploitation for mapping and surveillance. In: International Society for Photogrammetry and Remote Sensing (ISPRS) Commission I Symposium, Calgary (2010)
Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New York, pp. 519–526 (2006)
Smith, S.M., Brady, J.M.: SUSAN—a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997)
Sun, J., Zheng, N., Shum, H.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003)
Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. In: International Conference on Computer Vision (ICCV), Nice, France, pp. 900–907 (2003)
Torr, P.H.S., Murray, D.: The development and comparison of robust methods for estimating the fundamental matrix. Int. J. Comput. Vis. 24(3), 271–300 (1997)
Torresani, L., Hertzmann, A., Bregler, C.: Non-rigid structure-from-motion: estimating shape and motion with hierarchical priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(5), 878–892 (2008)
Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzigibbon, A.W.: Bundle adjustment—a modern synthesis. In: International Workshop on Vision Algorithms, Kerkyra, Greece, pp. 298–372 (1999)
Tsai, R.Y.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 3(4), 323–344 (1987)
Videre Design: http://www.videredesign.com/. Accessed 16th November 2011
White, R., Forsyth, D.A.: Combining cues: shape from shading and texture. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New York, pp. 1809–1816 (2006)
Woodham, R.J.: Analysing images of curved surfaces. Artif. Intell. 17, 117–140 (1981)
Yamauchi, B.: Autonomous urban reconnaissance using man-portable UGVs. In: Unmanned Systems Technology. Orlando, Florida, SPIE, vol. 6230 (2006)
Yang, R., Pollefeys, M.: A versatile stereo implementation on commodity graphics hardware. Real-Time Imaging 11(1), 7–18 (2005)
Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)
Zhang, R., Tsai, P.S., Cryer, J.E., Shah, M.: Shaping from shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 690–706 (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag London
About this chapter
Cite this chapter
Se, S., Pears, N. (2012). Passive 3D Imaging. In: Pears, N., Liu, Y., Bunting, P. (eds) 3D Imaging, Analysis and Applications. Springer, London. https://doi.org/10.1007/978-1-4471-4063-4_2
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4063-4_2
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4062-7
Online ISBN: 978-1-4471-4063-4
eBook Packages: Computer ScienceComputer Science (R0)