Abstract
Stitching motions in multiple videos into a single video scene is a challenging task in current video fusion and mosaicing research and film production. In this paper, we present a novel method of video motion stitching based on the similarities of trajectory and position of foreground objects. First, multiple video sequences are registered in a common reference frame, whereby we estimate the static and dynamic backgrounds, with the former responsible for distinguishing the foreground from the background and the static region from the dynamic region, and the latter functioning in mosaicing the warped input video sequences into a panoramic video. Accordingly, the motion similarity is calculated by reference to trajectory and position similarity, whereby the corresponding motion parts are extracted from multiple video sequences. Finally, using the corresponding motion parts, the foregrounds of different videos and dynamic backgrounds are fused into a single video scene through Poisson editing, with the motions involved being stitched together. Our major contributions are a framework of multiple video mosaicing based on motion similarity and a method of calculating motion similarity from the trajectory similarity and the position similarity. Experiments on everyday videos show that the agreement of trajectory and position similarities with the real motion similarity plays a decisive role in determining whether two motions can be stitched. We acquire satisfactory results for motion stitching and video mosaicing.
Similar content being viewed by others
References
Hermans C, Vanaken C, Mertens T, et al. Augmented panoramic video. Computer Graph Forum, 2008, 27: 281–290
Rav-Acha A, Pritch Y, Lischinski D, et al. Dynamosaicing: mosaicing of dynamic scenes. IEEE Trans Pattern Anal Mach Intell, 2007, 29: 1789–1801
Chen J, Paris S, Wang J, et al. The Video Mesh: A Data Structure for Image-based Video Editing. MIT Computer Science and Artificial Intelligence Laboratory Technical Report Series, MIT-CSAIL-TR-2009-062. 2009
Zhao Q. Data acquisition and simulation of natural phenomena. Sci China Inf Sci, 2011, 54: 683–716
Pan B, Zhong F, Wang S, et al. Salient structural elements based texture synthesis. Sci China Inf Sci, 2011, 54: 1199–1206
Colombari A, Fusiello A, Murino V. Video objects segmentation by robust background modeling. In: Proceedings of the 14th International Conference on Image Analysis and Processing. Washington: IEEE Computer Society, 2007. 155–164
Wang J, Bhat P, Colburn R A, et al. Interactive video cutout. ACM Trans Graph, 2005, 24: 585–594
Xiao C, Nie Y, Tang F. Efficient edit propagation using hierarchical data structure. IEEE Trans Vis Comput Graph, 2011, 17: 1135–1147
Lee W, Wontack W, Boyer E. Silhouette segmentation in multiple views. IEEE Trans Pattern Anal Mach Intell, 2011, 33: 1429–1441
Shi J, Tomasi C. Good features to track. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. New Nork: IEEE Computer Society Press, 1994. 593–600
Harris C, Stephens M. A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference. Manchester: University of Manchester, 1988. 147–151
Smith S M, Brady J M. SUSAN-a new approach to low level image processing. Int J Comput Vis, 1997, 23: 45–78
Brown M, Lowe D. Automatic panoramic image stitching using invariant features. Int J Comput Vis, 2007, 74: 59–73
Damerval C, Meignen S. Study of a robust feature: the pointwise lipschitz regularity. Int J Comput Vis, 2010, 88: 363–381
Xu W, Mulligan J. Performance evaluation of color correction approaches for automatic multi-view image and video stitching. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. New York: IEEE Computer Society Press, 2010. 263–270
Agarwala A, Zheng K C, Pal C, et al. Panoramic video textures. ACM Trans Graph, 2005, 24: 821–827
Wedge D, Kovesi P, Huynh D. Trajectory based video sequence synchronization. In: Proceedings of the Digital Image Computing on Techniques and Applications. Washington: IEEE Computer Society, 2005
P dua F, Carceroni R, Santos G, et al. Linear Sequence-to-Sequence alignment. IEEE Trans Pattern Anal Mach Intell, 2010, 32: 304–320
Britoa D N, P duaa F L C, Pereirab G A S, et al. Temporal synchronization of non-overlapping videos using known object motion. Pattern Recogn Lett, 2011, 32: 38–46
Perez P, Gangnet M, Blake A. Poisson image editing. ACM Trans Graph, 2003, 22: 313–318
Fischler M A, Bolles R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Comm ACM, 1981, 24: 381–395
Marzotto R, Fusiello A, Murino V. High resolution video mosaicing with global alignment. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. New York: IEEE Computer Society Press, 2004. 692–698
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chen, X., Li, Q., Li, X. et al. Video motion stitching using trajectory and position similarities. Sci. China Inf. Sci. 55, 600–614 (2012). https://doi.org/10.1007/s11432-011-4534-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11432-011-4534-y