Abstract
The structure-from-motion problem has been extensively studied in the field of computer vision. Yet, the bulk of the existing work assumes that the scene contains only a single moving object. The more realistic case where an unknown number of objects move in the scene has received little attention, especially for its theoretical treatment. In this paper we present a new method for separating and recovering the motion and shape of multiple independently moving objects in a sequence of images. The method does not require prior knowledge of the number of objects, nor is dependent on any grouping of features into an object at the image level. For this purpose, we introduce a mathematical construct of object shapes, called the shape interaction matrix, which is invariant to both the object motions and the selection of coordinate systems. This invariant structure is computable solely from the observed trajectories of image features without grouping them into individual objects. Once the matrix is computed, it allows for segmenting features into objects by the process of transforming it into a canonical form, as well as recovering the shape and motion of each object. The theory works under a broad set of projection models (scaled orthography, paraperspective and affine) but they must be linear, so it excludes projective “cameras”.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adelson, E. and Bergen, J. 1985. Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America, 2(2):284-299.
Bergen, J., Burt, P., Hingorani, R., and Peleg, S. 1990. Computing two motions from three frames. In Proceedings of the IEEE International Conference on Computer Vision.
Bienvenu, G. and Kopp, L. 1979. Principe de la noniometre passive adaptive. In Proc. 7éme Colloque GRETSI, Nice, France, pp. 106/1-106/10.
Boult, T. and Brown, L. 1991. Factorization-based segmentation of motions. In Proceedings of the IEEE Workshop on Visual Motion.
Cormen, T.H., Leiserson, C.E., and Rivest, R.L. 1986. Introduction to Algorithms. The MIT Press.
Costeira, J. and Kanade, T. 1997. A multi-body factorization method for independently moving objects: Full report.Technical Report CMU-RI-TR-97-30, Robotics Institute, Carnegie Mellon University. Also available at http://www.isr.ist.utl.pt/~jpc.
Demmel, J. 1987. The smallest perturbation of a submatrix which lowers the rank and constrained total least squares problems. SIAM Journal of Numverical Analysis, 24(1).
Faugeras, O. 1994. Three Dimensional Computer Vision. MIT Press: Cambridge, MA.
Gear, C.W. 1994. Feature grouping in moving objects. In Proceedings of the Workshop on Motion of Non-Rigid and Articulated Objects, Austin, Texas.
Golub, G., Hoffman, A., and Stewart, G. 1987. A generalization of the eckart-young-mirsky approximation theorem. Linear Algebra Applications.
Irani, M., Benny, R., and Peleg, S. 1994. Computing occluding and transparent motions. International Journal of Computer Vision, 12(1):5-16.
Jasinschi, R.S., Rosenfeld, A., and Sumi, K. 1992. Perceptual motion transparency: the role of geometrical information. Journal of the Optical Society of America, 9(11):1-15.
Koenderink, J. and van Doorn, A. 1991. Affine structure from motion. Journal of the Optical Society of America, 8(2):377- 385.
Poelman, C. and Kanade, T. 1993. A paraperspective factorization method for shape and motion recovery. Technical Report CS-93-219, School of Computer Science, Carnegie Mellon University.
Poelman, C. 1995. The paraperspective and projective factorization method for recovering shape and motion. Technical Report Also SCS Report CMU-CS-95-173, School of Computer Science, Carnegie Mellon University.
Schmidt, R. 1980. A signal subspace approach to multiple emitter location and spectral estimation. PhD Thesis, Stanford University, CA.
Sinclair, D. 1993. Motion segmentation and local structure. In Proceedings of the 4th International Conference on Computer Vision.
Stewart, G.W. 1992. Determining rank in the presence of error. In Proceedings of the NATO Workshop on Large Scale Linear Algebra, Leuven, Belgium. Also University of Maryland Tech. Report.
Tomasi, C. and Kanade, T. 1992. Shape from motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2):137-154. Originally published as CMU Technical Report CMU-CS-90-166, September 1990.
Ullman, S. 1983. Maximizing rigidity: The incremental recovery of 3D structure from rigid and rubbery motion. Technical Report A.I. Memo No. 721, MIT.
Van Trees. H. 1968. Detection, Estimation, and Modulation Theory, vol. 1. Wiley: New York.
Wilson, R. 1994. Modeling and calibration of automated zoom lenses. PhD Thesis, ECE, Carnegie Mellon University.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Costeira, J.P., Kanade, T. A Multibody Factorization Method for Independently Moving Objects. International Journal of Computer Vision 29, 159–179 (1998). https://doi.org/10.1023/A:1008000628999
Issue Date:
DOI: https://doi.org/10.1023/A:1008000628999