Skip to main content
SpringerLink
Log in

Frame-level temporal calibration of video sequences from unsynchronized cameras

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper describes a method for temporally calibrating video sequences from unsynchronized cameras by image processing operations, and presents two search algorithms to match and align trajectories across different camera views. Existing multi-camera systems assume that input video sequences are synchronized either by genlock or by time stamp information and a centralized server. Yet, hardware-based synchronization increases installation cost. Hence, using image information is necessary to align frames from the cameras whose clocks are not synchronized. The system built for temporal calibration is composed of three modules: object tracking module, calibration data extraction module, and the search module. A robust and efficient search algorithm is introduced that recovers the frame offset by matching the trajectories in different views, and finding the most reliable match. Thanks to information obtained from multiple trajectories, this algorithm is robust to possible errors in background subtraction and location extraction. Moreover, the algorithm can handle very large frame offsets. A RANdom SAmple Consensus (RANSAC) based version of this search algorithm is also introduced. Results obtained with different video sequences are presented, which show the robustness of the algorithms in recovering various range of frame offsets for video sequences with varying levels of object activity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Cai, Q., Aggarwal, J.K.: Automatic tracking of human motion in indoor scenes across multiple synchronized video streams. In: Proc. of Int’l Conf. on Computer Vision (1998)

  2. Chang, T.-H., Gong, S.: Tracking multiple people with a multi-camera system. In: IEEE Workshop on Multi-Object Tracking pp. 19–26 (2001)

  3. Collins, R.T., Lipton, A.J., Kanade, T., Fujiyoshi, H., Duggins, D., Tsin, Y., Tolliver, D., Enomoto, N., Hasegawa, O., Burt, P., Wixson, L.: A system for video surveillance and monitoring: VSAM final report. Technical report CMU-RI-TR-00-12, Robotics Institute, Carnegie Mellon University (2000)

  4. Collins R.T., Lipton A.J., Fujiyoshi H. and Kanade T. (2001). Algorithms for cooperative multisensor surveillance. Proc. IEEE 89: 1456–1477

    Article  Google Scholar 

  5. Ellis, T.: Multi-camera video surveillance. In: Int’l Carnahan Conf. on Security Technology, pp. 228–233 (2002)

  6. Khan S. and Shah M. (2003). Consistent labeling of tracked objects in multiple cameras with overlapping fields of view. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 25: 1355–1360

    Article  Google Scholar 

  7. Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., Shafer, S.: Multi-camera multi-person tracking for EasyLiving. In: Proc. of 3rd IEEE Int’l Workshop on Visual Surveillance (2000)

  8. Nguyen K., Yeung G., Ghiasi S., Sarrafzadeh M. (2002) A general framework for tracking objects in a multi-camera environment. In: Int’l Workshop on Digital and Computational Video pp. 200–204

  9. Pavlidis I., Morellas V., Tsiamyrtzis P. and Harp S. (2001). Urban surveillance systems: from the laboratory to the commercial world. Proc. IEEE 89: 1478–1497

    Article  Google Scholar 

  10. Velipasalar, S., Schlessman, J., Chen, C.-Y., Wolf, W., Singh, J.P.: SCCS: a scalable camera cluster system for multiple object tracking communicating via message passing interface. In: IEEE Int’l Conf. on Multimedia & Expo (2006)

  11. Velipasalar, S., Brown, L.M., Hampapur, A.: Specifying, interpreting and detecting high-level, spatio-temporal composite events in single and multi-camera systems. In: Int’l Workshop on Semantic Learning Applications in Multimedia (SLAM) in Conjunction with IEEE CVPR (2006)

  12. Yuan, X., Sun, Z., Varol, Y., Bebis, G.: A distributed visual surveillance system. In: Proc. of IEEE Conf. on Advanced Video and Signal Based Surveillance pp. 199–204 (2003)

  13. Kuthirummal S., Jawahar C.V. and Narayanan P.J. (2002). Video frame alignment in multiple views. Proc. IEEE Intl Conf. Image Process. 3: 357–360

    Google Scholar 

  14. Lee L., Romano R. and Stein G. (2000). Monitoring activities from multiple video streams: establishing a common coordinate frame. IEEE Trans. Pattern Anal Mach Intell (PAMI) 22: 758–767

    Article  Google Scholar 

  15. Caspi, Y., Irani, M.: A step towards sequence-to-sequence alignment. In: Proc. IEEE Intl Conf. Computer Vis. Pattern Recogn. 682–689 (2000) (to appear)

  16. Caspi, Y., Simakov, D., Irani, M.: Feature-based sequence- to-sequence matching. In: VAMODS Workshop (2002)

  17. Tuytelaars T. and Van Gool L. (2004). Synchronizing video sequences. Proc. IEEE Conf. Comput. Vis. Pattern Recogn. 1: 762–768

    Google Scholar 

  18. Fischler M.A. and Bolles R. (1981). RANSAC random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24: 381–395

    Article  MathSciNet  Google Scholar 

  19. Stauffer C. and Grimson W.E.L. (1999). Adaptive background mixture models for real-time tracking. Proc. IEEE Conf. Comput. Vis. Pattern Recogn. 2: 246–252

    Google Scholar 

  20. Comaniciu D., Ramesh V. and Meer P. (2000). Real-time tracking of non-rigid objects using mean shift. Proc. IEEE Intl Conf. Comput. Vis. Pattern Recogn. 2: 142–149

    Google Scholar 

  21. Hartley R. and Zisserman A. (2001). Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge

    Google Scholar 

  22. Trucco E. and Verri A. (1998). Introductory Techniques for 3-D Computer Vision. Prentice Hall, New Jersey

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, University of Nebraska-Lincoln, 209N WSEC, Lincoln, NE, 68588, USA

    Senem Velipasalar

  2. Georgia Institute of Technology, Van Leer Electrical Engineering Building, 777 Atlantic Drive NW, Atlanta, GA, 30332-0250, USA

    Wayne H. Wolf

Authors
  1. Senem Velipasalar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wayne H. Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Senem Velipasalar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Velipasalar, S., Wolf, W.H. Frame-level temporal calibration of video sequences from unsynchronized cameras. Machine Vision and Applications 19, 395–409 (2008). https://doi.org/10.1007/s00138-008-0122-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-008-0122-6

Keywords

Search

Navigation