skip to main content
10.1145/2510650.2510654acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Warping trajectories for video synchronization

Published: 21 October 2013 Publication History

Abstract

Temporal synchronization of multiple video recordings of the same dynamic event is a critical task in many computer vision applications e.g. novel view synthesis and 3D reconstruction. Typically this information is implied, since recordings are made using the same timebase, or time-stamp information is embedded in the video streams. Recordings using consumer grade equipment do not contain this information; hence, there is a need to temporally synchronize signals using the visual information itself. Previous work in this area has either assumed good quality data with relatively simple dynamic content or the availability of precise camera geometry. In this paper, we propose a technique which exploits feature trajectories across views in a novel way, and specifically targets the kind of complex content found in consumer generated sports recordings, without assuming precise knowledge of fundamental matrices or homographies. Our method automatically selects the moving feature points in the two unsynchronized videos whose 2D trajectories can be best related, thereby helping to infer the synchronization index. We evaluate performance using a number of real recordings and show that synchronization can be achieved to within 1 sec, which is better than previous approaches.

References

[1]
Y. Caspi and M. Irani. Alignment of non-overlapping sequences. In ICCV, volume 2, pages 76--83. IEEE, 2001.
[2]
Y. Caspi and M. Irani. Spatio-temporal alignment of sequences. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(11):1409--1424, 2002.
[3]
Y. Caspi, D. Simakov, and M. Irani. Feature-based sequence-to-sequence matching. International Journal of Computer Vision, 68(1):53--64, 2006.
[4]
P. Chen and D. Suter. Simultaneously estimating the fundamental matrix and homographies. Robotics, IEEE Transactions on, 25(6):1425--1431, 2009.
[5]
N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 886--893. IEEE, 2005.
[6]
E. Dexter and I. Laptev. Multi-view synchronization of human actions and dynamic scenes. In Proc. British Machine Vision Conference, 2009.
[7]
M. Germann, T. Popa, R. Ziegler, R. Keiser, and M. Gross. Space-time body pose estimation in uncontrolled environments. In 3DIMPVT, pages 244--251. IEEE, 2011.
[8]
N. Hasler, B. Rosenhahn, T. Thormahlen, M. Wand, J. Gall, and H.-P. Seidel. Markerless motion capture with unsynchronized moving cameras. In CVPR, pages 224--231. IEEE, 2009.
[9]
C. Lei and Y.-H. Yang. Tri-focal tensor-based multiple video synchronization with subframe optimization. Image Processing, IEEE Transactions on, 15(9):2473--2480, 2006.
[10]
R. Li, R. Chellappa, and S. K. Zhou. Learning multi-modal densities on discriminative temporal interaction manifold for group activity recognition. In CVPR, pages 2450--2457. IEEE, 2009.
[11]
D. G. Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91--110, 2004.
[12]
J. MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, page 14. California, USA, 1967.
[13]
K. G. Murty and S. N. Kabadi. Some np-complete problems in quadratic and nonlinear programming. Mathematical programming, 39(2):117--129, 1987.
[14]
D. Pundik and Y. Moses. Video synchronization using temporal signals from epipolar lines. In ECCV, pages 15--28. Springer, 2010.
[15]
I. Reid and A. Zisserman. Goal-directed video metrology. In ECCV, pages 647--658. Springer, 1996.
[16]
I. E. Richardson. H. 264 and MPEG-4 video compression: video coding for next-generation multimedia. Wiley. com, 2004.
[17]
P. J. Rousseeuw. Least median of squares regression. Journal of the American statistical association, 79(388):871--880, 1984.
[18]
J. Serrat, F. Diego, F. Lumbreras, and J. M. Álvarez. Synchronization of video sequences from free-moving cameras. In Pattern Recognition and Image Analysis, pages 620--627. Springer, 2007.
[19]
J. Shi and C. Tomasi. Good features to track. In CVPR, pages 593--600. IEEE, 1994.
[20]
M. Singh, A. Basu, and M. Mandal. Event dynamics based temporal registration. Multimedia, IEEE Transactions on, 9(5):1004--1015, 2007.
[21]
L. Spencer and M. Shah. Temporal synchronization from camera motion. In ACCV, pages 515--520, 2004.
[22]
P. Tresadern and I. Reid. Synchronizing image sequences of non-rigid objects. In Proc. British Machine Vision Conference, volume 2, pages 629--638, 2003.
[23]
T. Tuytelaars and L. Van Gool. Synchronizing video sequences. In CVPR, volume 1, pages I--762. IEEE, 2004.
[24]
A. Whitehead, R. Laganiere, and P. Bose. Temporal synchronization of video sequences in theory and in practice. In Application of Computer Vision, 2005. WACV/MOTIONS'05 Volume 1. Seventh IEEE Workshops on, volume 2, pages 132--137. IEEE, 2005.
[25]
L. Wolf and A. Zomet. Correspondence-free synchronization and reconstruction in a non-rigid scene. In Proc. Workshop on Vision and Modelling of Dynamic Scenes, Copenhagen, 2002.
[26]
L. Zelnik-Manor and M. Irani. Event-based analysis of video. In CVPR, volume 2, pages II--123. IEEE, 2001.

Cited By

View all
  • (2018)Camera Synchronization for Panoramic VideosMediaSync10.1007/978-3-319-65840-7_20(565-592)Online publication date: 27-Mar-2018
  • (2014)Multimodal Alignment of VideosProceedings of the 22nd ACM international conference on Multimedia10.1145/2647868.2654862(667-670)Online publication date: 3-Nov-2014

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ARTEMIS '13: Proceedings of the 4th ACM/IEEE international workshop on Analysis and retrieval of tracked events and motion in imagery stream
October 2013
94 pages
ISBN:9781450323932
DOI:10.1145/2510650
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. applications of computer vision methods
  2. pattern recognition

Qualifiers

  • Research-article

Conference

MM '13
Sponsor:
MM '13: ACM Multimedia Conference
October 21, 2013
Barcelona, Spain

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 23 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2018)Camera Synchronization for Panoramic VideosMediaSync10.1007/978-3-319-65840-7_20(565-592)Online publication date: 27-Mar-2018
  • (2014)Multimodal Alignment of VideosProceedings of the 22nd ACM international conference on Multimedia10.1145/2647868.2654862(667-670)Online publication date: 3-Nov-2014

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media