Abstract
This paper shows how two image sequences that have no spatial overlap between their fields of view can be aligned both in time and in space. Such alignment is possible when the two cameras are attached closely together and are moved jointly in space. The common motion induces “similar” changes over time within the two sequences. This correlated temporal behavior, is used to recover the spatial and temporal transformations between the two sequences. The requirement of “consistent appearance” in standard image alignment techniques is therefore replaced by “consistent temporal behavior”, which is often easier to satisfy.
This approach to alignment can be used not only for aligning non-overlapping sequences, but also for handling other cases that are inherently difficult for standard image alignment techniques. We demonstrate applications of this approach to three real-world problems: (i) alignment of non-overlapping sequences for generating wide-screen movies, (ii) alignment of images (sequences) obtained at significantly different zooms, for surveillance applications, and, (iii) multi-sensor image alignment for multi-sensor fusion.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Avidan, S. and Shashua, A. 1998. Threading fundamental matrices. In European Conference on Computer Vision.
Beardsley, P.A., Torr, P.H.S., and Zisserman, A. 1996. 3D model aquisition from extended image sequences. In Proc. 4th European Conference on Computer Vision, LNCS 1065, Cambridge, pp. 683–695.
Bjorck, A. 1996. Numerical Methodes for Least Squares Problems. SIAM: Philadelphia.
Burt, P.R. and Kolczynski, R.J. 1993. Enhanced image capture through fusion. In International Conference on Computer Vision, pp. 173–182.
Caspi, Y. and Irani, M. 2000. A step towards sequence-to-sequence alignment. In IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, June 2000, pp. 682–689.
Caspi, Y. and Irani, M. 2001. Alignment of non-overlaping sequences. In International Conference on Computer Vision, vol. II, Vancouver, Canada, pp. 76–83.
Demirdijian, D., Zisserman, A., and Horaud, R. 2000. Stereo auto-calibration from one plane. In European Conference on Computer Vision, pp. 625–639.
Dufournaud, Y., Schmid, C., and Horaud, R. 2000. Matching images with different resolutions. In IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, June 2000, pp. 612–619.
Gantmakher, F.R. 1959. The Theory of Matrices. Chelsea Pub.: New York.
Golub, Gene and Van Loan, Charles. 1989. Matrix Computations. The Johns Hopkins University Press: Baltimore and London.
Hartley, R.I. 1997. In defence of the 8–point algorithm. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(6):580–593.
Hartley, R. and Zisserman, A. 2000. Multiple ViewGeometry in Computer Vision. Cambridge University Press: Cambridge.
Horaud, R. and Csurka, G. 1998. Reconstruction using motions of a stereo rig. In International Conference on Computer Vision, pp. 96–103.
Horaud, R. and Dornaika, F. 1995. Hand-eye calibration. International Journal of Robotics Research, 14(3):195–210.
Irani, M. and Anandan, P. 1999. About direct methods. In Vision Algorithms Workshop, Corfu, pp. 267–277.
Irani, M., Anandan, P., and Weinshall, D. 1998. From reference frames to reference planes: Multi-view parallax geometry and applications. In European Conference on Computer Vision, Freiburg, June 1998, pp. 829–845.
Irani, M., Rousso, B., and Peleg, S. 1994. Computing occluding and transparent motions. International Journal of Computer Vision, 12(1):5–16.
Kumar, R., Anandan, P., and Hanna, K. 1994. Direct recovery of shape from multiple views: Parallax based approach. In International Conference on Pattern Recognition, pp. 685–688.
\({{\tilde P}}\)earson, C.E. (Ed.), 1983. Handbook of Applied Mathematics, 2nd edn. Van Nostrand Reinhold Company: New York.
Sawhney, H. 1994. 3D geometry from planar parallax. In IEEE Conference on Computer Vision and Pattern Recognition, June 1994, pp. 929–934.
Shashua, A. and Navab, N. 1994. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, June 1994, pp. 483–489.
Slama, C.C. 1980. Manual of Photogrammetry. American Society of Photogrammetry and Remote Sensing.
Stein, G.P. 1998. Tracking from multiple view points: Self-calibration of space and time. In DARPA IU Workshop, Montery CA, pp. 1037–1042.
Torr, P.H.S. and Zisserman, A. 1999. Feature based methods for structure and motion estimation. In Vision Algorithms Workshop, Corfu, pp. 279–329.
Tsai, R.Y. and Lenz, R.K. 1989. A new technique for full autonomous and efficient 3D robotics hand/eye calibration. IEEE Journal of Robotics and Automation, 5(3):345–358.
Viola, P. and Wells III, W. 1995. Alignment by maximization of mutual information. In International Conference on Computer Vision, pp. 16–23.
Zisserman, A., Beardsley, P.A., and Reid, I.D. 1995. Metric calibration of a stereo rig. In Workshop on Representations of Visual Scenes, pp. 93–100.
Rights and permissions
About this article
Cite this article
Caspi, Y., Irani, M. Aligning Non-Overlapping Sequences. International Journal of Computer Vision 48, 39–51 (2002). https://doi.org/10.1023/A:1014803327923
Issue Date:
DOI: https://doi.org/10.1023/A:1014803327923