Abstract
Prevailing efforts to study the standard formulation of motion and structure recovery have recently been focused on issues of sensitivity and robustness of existing techniques. While many cogent observations have been made and verified experimentally, many statements do not hold in general settings and make a comparison of existing techniques difficult. With an ultimate goal of clarifying these issues, we study the main aspects of motion and structure recovery: the choice of objective function, optimization techniques and sensitivity and robustness issues in the presence of noise.
We clearly reveal the relationship among different objective functions, such as “(normalized) epipolar constraints,” “reprojection error” or “triangulation,” all of which can be unified in a new “optimal triangulation” procedure. Regardless of various choices of the objective function, the optimization problems all inherit the same unknown parameter space, the so-called “essential manifold.” Based on recent developments of optimization techniques on Riemannian manifolds, in particular on Stiefel or Grassmann manifolds, we propose a Riemannian Newton algorithm to solve the motion and structure recovery problem, making use of the natural differential geometric structure of the essential manifold.
We provide a clear account of sensitivity and robustness of the proposed linear and nonlinear optimization techniques and study the analytical and practical equivalence of different objective functions. The geometric characterization of critical points and the simulation results clarify the difference between the effect of bas-relief ambiguity, rotation and translation confounding and other types of local minima. This leads to consistent interpretations of simulation results over a large range of signal-to-noise ratio and variety of configurations.
Similar content being viewed by others
References
Adiv, G. 1989. Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5):477-489.
Boothby, W.M. 1986. An Introduction to Differential Manifolds and Riemannian Geometry. 2nd edn. Academic Press: San Diego.
Danilidis, K. 1997. Visual Navigation. Lawrence Erlbaum Associates.
Danilidis, K. and Nagel, H.-H. 1990. Analytical results on error sensitivity of motion estimation from two views. Image and Vision Computing, 8:297-303.
Edelman, A., Arias, T., and Smith, S.T. 1998. The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Analysis Applications, 20(2):303-353.
Faugeras, O. 1993. Three-dimensional Computer Vision: A Geometric Viewpoint. The MIT Press: Cambridge, MA, USA.
Hartley, R. and Sturm, P. 1997. Triangulation. Computer Vision and Image Understanding, 68(2):146-157.
Horn, B. 1990. Relative orientation. International Journal of Computer Vision, 4:59-78.
Jepson, A.D. and Heeger, D.J. 1993. Spatial Vision in Humans and Robots. Cambridge University Press: Cambridge. pp. 39-62.
Kanatani, K. 1993. Geometric Computation for Machine Vision. Oxford Science Publications: Oxford.
Kobayashi, S. and Nomizu, T. 1996. Foundations of Differential Geometry: Volume I. John Wiley & Sons, Inc.: New York.
Longuet-Higgins, H.C. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133-135.
Luong, Q.-T. and Faugeras, O. 1996. The fundamental matrix: Theory, algorithms, and stability analysis. International Journal of Computer Vision, 17(1):43-75.
Ma, Y., Košeckà, J., and Sastry, S. 1998. A mathematical theory of camera self-calibration. Electronic Research Laboratory Memorandum, UC Berkeley, UCB/ERL M98/64.
Ma, Y., Košeckà, J., and Sastry, S. 2000. Linear differential algorithm for motion recovery: A geometric approach. IJCV, 36(1):71-89.
Maybank, S. 1993. Theory of Reconstruction from Image Motion. Springer-Verlag: Berlin.
Milnor, J. 1969. Morse Theory. Annals of Mathematics Studies no. 51. Princeton University Press: Princeton.
Murray, R.M., Li, Z., and Sastry, S.S. 1994. A Mathematical Introduction to Robotic Manipulation. CRC press Inc.: Boca Raton, FL.
Oliensis, J. 1999. A new structure from motion ambiguity. In IEEE Proceedings from CVPR, pp. 185-191.
Sastry, S.S. 1999. Nonlinear Systems: Analysis, Stability and Control. Springer-Verlag: Berlin.
Smith, S.T. 1993. Geometric optimization methods for adaptive filtering. Ph.D. Thesis, Harvard University, Cambridge, Massachusetts.
Soatto, S. and Brockett, R. 1998. Optimal and suboptimal structure from motion. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.
Soatto, S. and Perona, P. 1996. Motion estimation via dynamic vision. IEEE Transactions on Automatic Control, 41(3):393-413.
Spetsakis, M. 1994. Models of statistical visual motion estimation. CVIPG: Image Understanding, 60(3):300-312.
Spivak, M. 1979. A Comprehensive Introduction to Differential Geometry: 2nd ed. Publish or Perish, Inc. Berkeley.
Taylor, C.J. and Kriegman, D.J. 1995. Structure and motion from line segments in multiple images. IEEE Transactions on PAMI, 17(11):1021-1032.
Thomas, I. and Simoncelli, E. 1995. Linear structure from motion. Ms-cis-94-61, Grasp Laboratory, University of Pennsylvania.
Tian, T.Y., Tomasi, C., and Heeger, D. 1996. Comparison of approaches to egomotion computation. In CVPR.
Weng, J., Huang, T.S., and Ahuja, N. 1989. Motion and structure from two perspective views: Algorithms, error analysis, and error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(5):451-475.
Weng, J., Huang, T.S., and Ahuja, N. 1993a. Motion and Structure from Image Sequences. Springer Verlag: Berlin.
Weng, J., Huang, T.S., and Ahuja, N. 1993b. Optimal motion and structure estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):864-884.
Zhang, T. and Tomasi, C. 1999. Fast, robust and consistent camera motion estimation. In Proceedings of CVPR.
Zhang, Z. 1998. Understanding the relationship between the optimization criteria in two-view motion analysis. In Proceedings of International Conference on Computer Vision, Bombay, India, pp. 772-777.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ma, Y., Košecká, J. & Sastry, S. Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation. International Journal of Computer Vision 44, 219–249 (2001). https://doi.org/10.1023/A:1012276232049
Issue Date:
DOI: https://doi.org/10.1023/A:1012276232049