Skip to main content
Log in

Spline-Based Image Registration

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The problem of image registration subsumes a number of problems and techniques in multiframe image analysis, including the computation of optic flow (general pixel-based motion), stereo correspondence, structure from motion, and feature tracking. We present a new registration algorithm based on spline representations of the displacement field which can be specialized to solve all of the above mentioned problems. In particular, we show how to compute local flow, global (parametric) flow, rigid flow resulting from camera egomotion, and multiframe versions of the above problems. Using a spline-based description of the flow removes the need for overlapping correlation windows, and produces an explicit measure of the correlation between adjacent flow estimates. We demonstrate our algorithm on multiframe image registration and the recovery of 3D projective scene geometry. We also provide results on a number of standard motion sequences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Adelson, E. H. and Bergen, J. R. 1985. Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America, A2(2):284–299.

    Google Scholar 

  • Amit, Y. 1993. Anon-linear variational problem for image matching. unpublished manuscript (from Newton Institute).

  • Anandan, P. 1989. A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision, 2(3):283–310.

    Google Scholar 

  • Bajcsy, R. and Broit, C. 1982. Matching of deformed images. In Sixth International Conference on Pattern Recognition (ICPRs'82), IEEE Computer Society Press: Munich, Germany, pp. 351–353.

    Google Scholar 

  • Bajcsy, R. and Kovacic, S. 1989. Multiresolution elastic matching. Computer Vision, Graphics, and Image Processing, 46(1):1–21.

    Google Scholar 

  • Barnard, S. T. and Fischler, M. A. 1982. Computational stereo. Computing Surveys, 14(4):553–572.

    Google Scholar 

  • Barron, J. L., Fleet, D. J., and Beauchemin, S. S. 1994. Performance of optical flow techniques. International Journal of Computer Vision, 12(1):43–77.

    Google Scholar 

  • Beier, T. and Neely, S. 1992. Feature-based image metamorphosis. Computer Graphics (SIGGRAPHs'92), 26(2):35–42.

    Google Scholar 

  • Bergen, J. R., Anandan, P., Hanna, K. J., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In Second European Conference on Computer Vision (ECCVs'92), Santa Margherita Liguere, Springer-Verlag: Italy, pp. 237–252.

    Google Scholar 

  • Beymer, D., Shashua, A., and Poggio, T. 1993. Example based image analysis and synthesis. A. I. Memo 1431, Massachusetts Institute of Technology.

  • Blake, A., Curwen, R., and Zisserman, A. 1993. A framework for spatio-temporal control in the tracking of visual contour. International Journal of Computer Vision, 11(2):127–145.

    Google Scholar 

  • Bolles, R. C., Baker, H. H., and Marimont, D. H. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1:7–55.

    Google Scholar 

  • Brown, L. G. 1992. A survey of image registration techniques. Computing Surveys, 24(4):325–376.

    Google Scholar 

  • Burr, D. J. 1981. A dynamic model for image registration. Computer Graphics and Image Processing, 15(2):102–112.

    Google Scholar 

  • Burt, P. J. and Adelson, E. H. 1983. The Laplacian pyramid as a compact image code. IEEE Transactions on Communications, COM-31(4):532–540.

    Google Scholar 

  • Carlbom, I., Terzopoulos, D., and Harris, K. M. 1991. Reconstructing and visualizing models of neuronal dendrites. In Scientific Visualization of Physical Phenomena, N. M. Patrikalakis (Ed.), Springer-Verlag: New York, pp. 623–638.

    Google Scholar 

  • Dhond, U. R. and Aggarwal, J. K. 1989. Structure from stereo—A review. IEEE Transactions on Systems, Man, and Cybernetics, 19(6):1489–1510.

    Google Scholar 

  • Dreschler, L. and Nagel, H.-H. 1982. Volumetric model and 3D trajectory of a moving car derived from monocular TV frame sequences of a stree scene. Computer Graphics and Image Processing, 20:199–228.

    Google Scholar 

  • Enkelmann, W. 1988. Investigations of multigrid algorithms for estimation of optical flow fields in image sequences. Computer Vision, Graphics, and Image Processing, pp. 150–177.

  • Farin, G. E. 1992. Curves and Surfaces for Computer Aided Geometric Design. Academic Press: Boston, Massachusetts, 3rd edition.

    Google Scholar 

  • Faugeras, O. D. 1992. What can be seen in three dimensions with an uncalibrated stereo rig? In Second European Conference on Computer Vision (ECCVs'92), Santa Margherita Liguere, Springer-Verlag: Italy, pp. 563–578.

    Google Scholar 

  • Fleet, D. and Jepson, A. 1990. Computation of component image velocity from local phase information. International Journal of Computer Vision, 5:77–104.

    Google Scholar 

  • Fuh, C.-S. and Maragos, P. 1991. Motion displacement estimation using an affine model for image matching. Optical Engineering, 30(7):881–887.

    Google Scholar 

  • Geiger, D., Ladendorf, B., and Yuille, A. 1992. Occlusions and binocular stereo. In Second European Conference on Computer Vision (ECCVs'92), Santa Margherita Liguere, Springer-Verlag, Italy, pp. 425–433.

    Google Scholar 

  • Gennert, M. A. 1988. Brightness-based stereo matching. In Second International Conference on Computer Vision (ICCVs'88), IEEE Computer Society Press: Tampa, Florida, pp. 139–143.

    Google Scholar 

  • Goshtasby, A. 1986. Piecewise linear mapping functions for image registration. Pattern Recognition, 19(6):459–466.

    Google Scholar 

  • Goshtasby, A. 1988. Image registration by local approximation methods. Image and Vision Computing, 6(4):255–261.

    Google Scholar 

  • Hanna, K. J. 1991. Direct multi-resolution estimation of ego-motion and structure from motion. In IEEE Workshop on Visual Motion, IEEE Computer Society Press: Princeton, New Jersey, pp. 156–162.

    Google Scholar 

  • Hartley, R. and Gupta, R. 1993. Computing matched-epipolar projections. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPRs'93), IEEE Computer Society Press: New York, pp. 549–555.

    Google Scholar 

  • Hartley, R., Gupta, R., and Chang, T. 1992. Stereo from uncalibrated cameras. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPRs'92), IEEE Computer Society Press: Champaign, Illinois, pp. 761–764,.

    Google Scholar 

  • Heeger, D. J. 1987. Optical flow from spatiotemporal filters. In First International Conference on Computer Vision (ICCVs'87), IEEE Computer Society Press: London, England, pp. 181–190.

    Google Scholar 

  • Hildreth, E. C. 1986. Computing the velocity field along contours. In Motion: Representation and Perception, N. I. Badler and J. K. Tsotsos (Eds.), North-Holland, New York, pp. 121–127.

  • Horn, B. K. P. and Schunck, B. G. 1981. Determining optical flow. Artificial Intelligence, 17:185–203.

    Google Scholar 

  • Horn, B. K. P. and Weldon, E. J., Jr. 1988. Direct methods for recovering motion. International Journal of Computer Vision, 2(1):51–76.

    Google Scholar 

  • Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models. International Journal of Computer Vision, 1(4):321–331.

    Google Scholar 

  • Koenderink, J. J. and van Doorn, A. J. 1991. Affine structure from motion. Journal of the Optical Society of America A, 8:377–385,538.

    Google Scholar 

  • Le Gall, D. 1991. MPEG: A video compression standard for multimedia applications. Communications of the ACM, 34(4):44–58.

    Google Scholar 

  • Lucas, B. D. 1984. Generalized Image Matching by the Method of Differences. Ph. D. Thesis, Carnegie Mellon University.

  • Lucas, B. D. and Kanade, T. 1981. An iterative image registration technique with an application in stereo vision. In Seventh International Joint Conference on Artificial Intelligence (IJCAI-81), Vancouver, pp. 674–679.

  • Manmatha, R. and Oliensis, J. 1992. Measuring the affine transform —I: Scale and rotation. Technical Report 92-74, University of Massachussets, Amherst, Massachussets.

    Google Scholar 

  • Matthies, L. H., Szeliski, R., and Kanade, T. 1989. Kalman filter-based algorithms for estimating depth from image sequences. International Journal of Computer Vision, 3:209–236.

    Google Scholar 

  • Menet, S., Saint-Marc, P., and Medioni, G. 1990. B-snakes: implementation and applications to stereo. In Image Understanding Workshop, Morgan Kaufmann Publishers: Pittsburgh, Pennsylvania, pp. 720–726.

    Google Scholar 

  • Mohr, R., Veillon, L., and Quan, L. 1993. Relative 3D reconstruction using multiple uncalibrated images. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPRs'93), New York, pp. 543–548.

  • Nagel, H.-H. 1987. On the estimation of optical flow: Relations between different approaches and some new results. Artificial Intelligence, 33:299–324.

    Google Scholar 

  • Okutomi, M. and Kanade, T. 1992. A locally adaptive window for signal matching. International Journal of Computer Vision, 7(2):143–162.

    Google Scholar 

  • Okutomi, M. and Kanade, T. 1993. A multiple baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(4):353–363.

    Google Scholar 

  • Poggio, T., Torre, V., and Koch, C. 1985. Computational vision and regularization theory. Nature, 317(6035):314–319.

    Google Scholar 

  • Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T. 1992. Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press: Cambridge, England, 2nd edition.

    Google Scholar 

  • Quam, L. H. 1984. Hierarchical warp stereo. In Image Understanding Workshop, Science Applications International Corporation: New Orleans, Louisiana, pp. 149–155.

    Google Scholar 

  • Rehg, J. and Witkin, A. 1991. Visual tracking with deformation models. In IEEE International Conference on Robotics and Automation, IEEE Computer Society Press: Sacramento, California, pp. 844–850.

    Google Scholar 

  • Sethi, I. K. and Jain, R. 1987. Finding trajectories of feature points in a monocular image sequence. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-9(1):56–73.

    Google Scholar 

  • Shi, J. and Tomasi, C. 1994. Good features to track. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPRs'94), IEEE Computer Society: Seattle, Washington, pp. 593–600.

    Google Scholar 

  • Simoncelli, E. P., Adelson, E. H., and Heeger, D. J. 1991. Probability distributions of optic flow. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPRs'91), IEEE Computer Society Press: Maui, Hawaii, pp. 310–315.

    Google Scholar 

  • Singh, A. 1990. An estimation-theoretic framework for image-flow computation. In Third International Conference on Computer Vision (ICCVs'90), IEEE Computer Society Press: Osaka, Japan, pp. 168–177.

    Google Scholar 

  • Szeliski, R. 1989. Bayesian Modeling of Uncertainty in Low-Level Vision. Kluwer Academic Publishers: Boston, Massachusetts.

    Google Scholar 

  • Szeliski, R. 1990. Fast surface interpolation using hierarchical basis functions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(6):513–528.

    Google Scholar 

  • Szeliski, R. 1996. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 16(2):22–30.

    Google Scholar 

  • Szeliski, R. and Kang, S. B. 1994. Recovering 3D shape and motion from image streams using nonlinear least squares. Journal of Visual Communication and Image Representation, 5(1):10–28.

    Google Scholar 

  • Szeliski, R. and Kang, S. B. 1995. Direct methods for visual scene reconstruction. In IEEE Workshop on Representations of Visual Scenes, Cambridge, Massachusetts, pp. 26–33.

  • Szeliski, R. and Shum, H.-Y. 1996. Motion estimation with quadtree splines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(12):1199–1210.

    Google Scholar 

  • Szeliski, R., Kang, S. B., and Shum, H.-Y. 1995. A parallel feature tracker for extended image sequences. In IEEE International Symposium on Computer Vision, Coral Gables, Florida, pp. 241–246.

    Google Scholar 

  • Terzopoulos, D. 1986. Regularization of inverse visual problems involving discontinuities. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8(4):413–424.

    Google Scholar 

  • Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2):137–154.

    Google Scholar 

  • Witkin, A., Terzopoulos, D., and Kass, M. 1987. Signal matching through scale space. International Journal of Computer Vision, 1:133–144.

    Google Scholar 

  • Wolberg, G. 1990. Digital Image Warping. IEEE Computer Society Press: Los Alamitos, California.

    Google Scholar 

  • Xu, G., Tsuji, S., and Asada, M. 1987. Amotion stereo method based on coarse-to-fine control strategy. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-9(2):332–336.

    Google Scholar 

  • Zheng, Q. and Chellappa, R. 1992. Automatic feature point extraction and tracking in image sequences for arbitrary camera motion. Technical Report CAR-TR-628, Computer Vision Laboratory, Center for Automation Research, University of Maryland.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Szeliski, R., Coughlan, J. Spline-Based Image Registration. International Journal of Computer Vision 22, 199–218 (1997). https://doi.org/10.1023/A:1007996332012

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1007996332012

Navigation