Skip to main content
SpringerLink
Log in

Image mosaic with relaxed motion

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

We propose a novel method to stitch images with relatively large roll or pitch called relaxed motion, which defies most existing mosaic algorithms. Our approach adopts a multi-resolution strategy, which combines the merits of both feature-based and intensity-based methods. The main contribution is a robust motion estimation procedure which integrates an adaptive multi-scale block matching algorithm called TV-BMA, a low contrast filter and a RANSAC motion rectification to jointly refine motion and feature matches. Based on TV − L 1 model, the proposed TV-BMA works on the coarsest layer to find a robust initial displacement field as the initial motion for source images. This motion estimation method can generate robust correspondences for further processing. In the subsequent camera calibration step, we also present two stable methods to estimate the camera matrix. To estimate the focal length, we combine the golden section search and the simplex method based on the angle invariance of feature vectors; to estimate the rotation matrix, we introduce a subspace trust region method, which matches features based on the rotation invariance. Extensive experiments show that our approach leads to improved accuracy and robustness for stitching images with relaxed motion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Szeliski R.: Image alignment and stitching: a tutorial. Found. Trends Comput. Graph. Comput. Vis. 2(1), 1–104 (2006)

    Article  Google Scholar 

  2. Szeliski R.: Video mosaics for virtual environments. IEEE Comput. Graph. Appl. 16(2), 22–30 (1996)

    Article  Google Scholar 

  3. Peleg, S., Herman, J.: Panoramic mosaics by manifold projection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’97), pp. 338. IEEE Computer Society, Washington (1997)

  4. Bartoli, A., Zisserman, A.: Direct estimation of non-rigid registration. In: Proceedings of British Machine Vision Conference 2004 (BMVC 2004), pp. 899–908. British Machine Vision Association, Kingston University, London (2004)

  5. Huang Y.-W., Chen C.-Y., Tsai C.-H., Shen C.-F., Chen L.-G.: Survey on block matching motion estimation algorithms and architectures with new results. J. VLSI Signal Process. 42, 297–320 (2006)

    Article  MATH  Google Scholar 

  6. Zitova B., Flusser J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)

    Article  Google Scholar 

  7. Cho, S.-H., Chung, Y.-K., Lee, J.Y.: Automatic image mosaic system using image feature detection and taylor series. In: Proceedings of the Seventh International Conference on Digital Image Computing: Techniques and Applications (DICTA 2003), pp. 549–560. CSIRO Publishing, Macquarie University, Sydney (2003, Dec)

  8. Brown, M., Lowe, D.G.: Recognising panoramas. In: Proceedings of the Ninth IEEE International Conference on Computer Vision (ICCV ’03), pp. 1218–1225. IEEE Computer Society, Washington (2003)

  9. Fang X., Zhang M., Pan Z., Wang P.: A new method of manifold mosaic for large displacement images. J. Comput. Sci. Technol. 21(2), 218–223 (2006)

    Article  Google Scholar 

  10. Brown M., Lowe D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)

    Article  Google Scholar 

  11. Mikolajczyk K., Schmid C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1615–1630 (2005)

    Article  Google Scholar 

  12. Lowe D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  13. Szeliski, R., Shum, H.-Y.: Creating full view panoramic image mosaics and environment maps. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’97), pp. 251–258. ACM Press/Addison-Wesley Publishing Co, New York (1997)

  14. Zhou, L.: Image matching using resolution pyramids with geometric constraints. Patent number US 6785427 (2004, Aug)

  15. Chen, T., Huang, T.S.: Optimizing image registration by mutually exclusive scale components. In: Proceedings of IEEE 11th International Conference on Computer Vision (ICCV 2007), pp. 1–8. IEEE, Rio de Janeiro (2007, Oct)

  16. Chan T.F., Esedoglu S.: Aspects of total variation regularized l 1 function approximation. SIAM J. Appl. Math. 75(5), 1817–1837 (2004)

    MathSciNet  Google Scholar 

  17. Harris, C., Stephens, M.: A combined corner and edge detection. In: Proceedings of The Fourth Alvey Vision Conference, pp. 147–151 (1988)

  18. Fischler M.A., Bolles R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  19. Hartley R.I., Zisserman A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  20. Brown, M., Hartley, R.I., Nister, D.: Minimal solutions for panoramic stitching. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’07), vol. 0, pp. 1–8. IEEE Computer Society, Los Alamitos (2007)

  21. Jin, H.: A three-point minimal solution for panoramic stitching with lens distortion. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), vol. 0, pp. 1–8. IEEE Computer Society, Anchorage (2008)

  22. Rudin L.I., Osher S., Fatemi E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1–4), 259–268 (1992)

    Article  MATH  Google Scholar 

  23. Wang Y., Yang J., Yin W., Zhang Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  24. Yang, J., Yin, W., Zhang, Y., Wang, Y.: A Fast Algorithm for Edge-Preserving Variational Multichannel Image Restoration. Tech. Rep. TR08-09. Department of Computational and Applied Mathematics, Rice University (2008, July)

  25. Yang J., Zhang Y., Yin W.: An efficient tvl1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM J. Sci. Comput. 31(4), 2842–2865 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  26. Geman D., Yang C.: Nonlinear image recovery with half-quadratic regularization. IEEE Trans. Image Process. 5(7), 932–946 (1995)

    Article  Google Scholar 

  27. Roche, A., Malandain, G., Pennec, X., Ayache, N.: The correlation ratio as a new similarity measure for multimodal image registration. In: MICCAI ’98: Proceedings of the First International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 1115–1124. Springer-Verlag, London (1998)

  28. Noble, A.: Descriptions of Image Surfaces. PhD thesis. Department of Engineering Science, Oxford University (1989)

  29. Kovesi, P.D.: MATLAB and Octave Functions for Computer Vision and Image Processing. School of Computer Science & Software Engineering, The University of Western Australia. Available from: http://www.csse.uwa.edu.au/~pk/research/matlabfns/

  30. Hartley R., Zisserman A.: Multiple View Geometry in Computer Vision. Cambridge University Press, New York (2003)

    Google Scholar 

  31. Moré J.J., Sorensen D.: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4(3), 553–572 (1983)

    Article  MATH  Google Scholar 

  32. Coleman T.F., Li Y.: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6, 418–445 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  33. Byrd R., Schnabel R., Shultz G.: Approximate solution of the trust region problem by minimization over two-dimensional subspaces. Math. Progr. 40, 247–263 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  34. Umeyama S.: Least-squares estimation of transformation parameters between two point patterns. IEEE Trans. Pattern Anal. Mach. Intell. 13, 376–380 (1991)

    Article  Google Scholar 

  35. Strait, J., Smoky City Design, L.: The panorama factory (2009, May)

  36. d’Angelo, P., Behrmann, K.-U., Wilkins, D., Halley, E., Ukai, I., Postle, B., Jin, J., Mesec, Z., Jenny, A., Yaniv, Z., Januszewski, M., Patterson, G., Sharpless, T., Levy, Y.: Hugin—panorama photo stitcher (2010, Mar)

  37. Fang, X., Luo, B., Zhao, H., Tang, J., Zhai, S.: A new multi-resolution image stitching with local and global alignment. IET Comput. Vision (2010, to appear)

Download references

Author information

Authors and Affiliations

  1. Key Lab. of Intelligent Computing and Signal Processing of MOE, School of Computer Science and Technology, Anhui University, Anhui, China

    Xianyong Fang & Bin Luo

  2. Computer Vision Lab, University of Central Florida, Orlando, USA

    Jiejie Zhu

Authors
  1. Xianyong Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiejie Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianyong Fang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fang, X., Zhu, J. & Luo, B. Image mosaic with relaxed motion. SIViP 6, 647–667 (2012). https://doi.org/10.1007/s11760-010-0194-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-010-0194-4

Keywords

Search

Navigation