Abstract
In this paper it is shown how multilinear forms can be used in the perception-action cycle. Firstly, these forms can be used to reconstruct an unknown (or partially known) scene from image sequences only. Secondly, from this reconstruction the movement of the camera can be calculated with respect to the scene, which solves the so called hand-eye calibration problem. Then action can be carried out when this relative orientation is known. The results are that it is sufficient to either use bilinear forms between every successive pair of images plus bilinear forms between every second image or trilinear forms between successive triplets of images. We also present a robust and accurate method to obtain reconstruction and hand-eye calibration from a sequence of images taken by uncalibrated cameras, based on multilinear forms. This algorithm requires no initialisation and gives a generic solution in a sense that is clearly specified. Finally, the algorithms are illustrated using real image sequences, showing that reconstruction can be made and that the hand-eye calibration is obtained.
This work has been supported by the Swedish Research Council for Engineering Sciences (TFR), project 95-64-222
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
Berthilsson, R., Heyden, A., Sparr, G., Recursive Structure and Motion from Image Sequences using Shape and Depth Spaces, to be presented at CVPR'97, 1996.
Faugeras, O., D., What can be seen in three dimensions with an uncalibrated stereo rig?, ECCV'92, Lecture notes in Computer Science, Vol 588. Ed. G. Sandini, Springer-Verlag, 1992, pp. 563–578.
Faugeras, O., D., Mourrain, B., On the geometry and algebra on the point and line correspondences between N images, Proc. ICCV'95, IEEE Computer Society Press, 1995, pp. 951–956.
Faugeras, O., D., Mourrain, B., About the correspondences of points between N images, Proc. IEEE Workshop on Representation of Visual Scenes, 1995.
Hartley, R., I., Projective Reconstruction and Invariants from Multiple Images, IEEE Trans. Pattern Anal. Machine Intell., vol. 16, no. 10, pp. 1036–1041, 1994.
Hartley, A linear method for reconstruction from lines and points, Proc. ICCV'95, IEEE Computer Society Press, 1995, pp. 882–887.
Heyden, A., Reconstruction and Prediction from Three Images of Uncalibrated Cameras, Proc. 9th Scandinavian Conference on Image Analysis, Ed. Gunilla Borgefors, Uppsala, Sweden, 1995, pp. 57–66.
Heyden, A., Reconstruction from Image Sequences by means of Relative Depths, Proc. ICCV'95, IEEE Computer Society Press, 1995, pp. 1058–1063. An extended version to appear in IJCV, International Journal of Computer Vision, 1996.
Heyden, A., Åström, K., A Canonical Framework for Sequences of Images, Proc. IEEE Workshop on Representation of Visual Scenes, 1995.
Heyden, A., Åström, K., Algebraic Varieties in Multiple View Geometry, ECCV'96, Lecture notes in Computer Science, Vol 1065, Ed. B. Buxton, R. Chippola, Springer-Verlag 1996, pp. 671–682.
Heyden, A., Projective Structure and Motion from Image Sequences using Subspace Methods, Submitted to SCIA'97, 1996.
Luong, Q.-T, Vieville, T, Canonic Representations for the Geometries of Multiple Projective Views, ECCV'94, Lecture notes in Computer Science, Vol 800. Ed. Jan-Olof Eklund, Springer-Verlag, 1994, pp. 589–599.
Maybank, S., Theory of Reconstruction from Image Motion, Springer-Verlag, Berlin, Heidelberg, New York, 1993.
Mohr, R., Arbogast, E., It can be done without camera calibration, Pattern Recognition Letters, vol. 12, no. 1, 1991, pp. 39–43.
Poelman, C., J., Kanade, T., A Paraperspective Factorization Method for Shape and Motion Recovery, ECCV'94, Lecture notes in Computer Science, Vol 801. Ed. Jan-Olof Eklund, Springer-Verlag, 1994, pp. 97–108.
Shashua, A., Trilinearity in Visual Recognition by Alignment, ECCV'94, Lecture notes in Computer Science, Vol 800. Ed. Jan-Olof Eklund, Springer-Verlag, 1994, pp. 479–484.
Shashua, A., Werman, M., Trilinearity of Three Perspective Views and its Associated Tensor, Proc. ICCV'95, IEEE Computer Society Press, 1995, pp. 920–925.
Sparr, G., An algebraic-analytic method for affine shapes of point configurations, proceedings 7th Scandinavian Conference on Image Analysis, 1991, pp. 274–281.
Sparr, G., A Common Framework for Kinetic Depth, Reconstruction and Motion for Deformable Objects, ECCV'94, Lecture notes in Computer Science, Vol 801. Ed. J-O. Eklund, Springer-Verlag 1994, pp. 471–482.
Sparr, G., Simultaneous Reconstruction of Scene Structure and Camera Locations from Uncalibrated Image Sequences, proceedings 13th International Conference on Pattern Recognition, 1996, pp. 328–333.
Tomasi, C., Kanade, T., Shape and Motion from Image Streams under Orthography: A Factorization Method, IJCV, 9(2):137–154, 1992.
Triggs, B., Matching Constraints and the Joint Image, Proc. ICCV'95, IEEE Computer Society Press, 1995, pp. 338–343.
Sturm, P., Triggs, B., A Factorization Based Algorithm for Multi-Image Projective Structure and Motion, ECCV'96, Lecture notes in Computer Science, Vol 1065. Ed. B. Buxton and R. Cipolla, Springer-Verlag 1996, pp. 709–720.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Heyden, A., Sparr, G., Åström, K. (1997). Perception and action using multilinear forms. In: Sommer, G., Koenderink, J.J. (eds) Algebraic Frames for the Perception-Action Cycle. AFPAC 1997. Lecture Notes in Computer Science, vol 1315. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0017860
Download citation
DOI: https://doi.org/10.1007/BFb0017860
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63517-8
Online ISBN: 978-3-540-69589-9
eBook Packages: Springer Book Archive