Skip to main content
SpringerLink
Log in

Graph-based multilevel temporal video segmentation

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

This paper presents a graph-based multilevel temporal video segmentation method. In each level of the segmentation, a weighted undirected graph structure is implemented. The graph is partitioned into clusters which represent the segments of a video. Three low-level features are used in the calculation of temporal segments’ similarities: visual content, motion content and shot duration. Our strength factor approach contributes to the results by improving the efficiency of the proposed method. Experiments show that the proposed video scene detection method gives promising results in order to organize videos without human intervention.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adams, B., Dorai, C., Venkatesh, S.: Novel approach to determining tempo and dramatic story sections in motion pictures. In: Proceedings of IEEE International Conference on Image Processing, vol. 2, pp. 283–286, 10–13 Sept, 2000

  2. Ding, C.H.Q., He, X., Zha, H., Gu, M., Simon, H.D.: A min–max cut algorithm for graph partitioning and data clustering. In: Proceedings of IEEE International Conference on Data Mining, pp. 107–114, 29 Nov–2 Dec 2001

  3. Gargi U., Kasturi R., Strayer S.H.: Performance characterization of video shot change detection methods. IEEE Trans. Circuits Syst. Video Technol. 10, 1–13 (2000)

    Article  Google Scholar 

  4. Gong Y.: Summarizing audiovisual contents of a video program. EURASIP J. Appl. Signal Process. 2003, 160–169 (2003)

    Article  Google Scholar 

  5. Gu, Z., Mei, T., Hua, X.-S., Wu, X., Li, S.: EMS: Energy minimization based video scene segmentation. In: Proceedings of IEEE International Conference on Multimedia and Expo, pp. 520–523, 2–5 July, 2007

  6. Hanjalic A., Lagendijk R.L., Biemond J.: Automated high-level movie segmentation for advanced video-retrieval systems. IEEE Trans. Circuits Syst. Video Technol. 9, 580–588 (1999)

    Article  Google Scholar 

  7. Hanjalic A.: Shot boundary detection: Unraveled and resolved?. IEEE Trans. Circuits Syst. Video Technol. 12, 90–105 (2002)

    Article  Google Scholar 

  8. Hanjalic A.: Towards theoretical performance limits of video parsing. IEEE Trans. Circuits Syst. Video Technol. 17, 261–272 (2007)

    Article  Google Scholar 

  9. Kender, J.R., Yeo, B.-L.: Video scene segmentation via continuous video coherence. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 367–373, 23–25 June, 1998

  10. Koprinska I., Carrato S.: Temporal video segmentation: A survey. Signal Process. Image Commun. 16, 477–500 (2001)

    Article  Google Scholar 

  11. Lu, S., King, I., Lyu, M.R.: Novel video summarization framework for document preparation and archival applications. In: Proceedings of IEEE Aerospace Conference, pp. 1–10, 5–12 Mar 2005

  12. Ngo C.W., Ma Y.F., Zhang H.J.: Video summarization and scene detection by graph modeling. IEEE Trans. Circuits Syst. Video Technol. 15, 296–305 (2005)

    Article  Google Scholar 

  13. Odobez J.-M., Gatica-Perez D., Guillemot M.: Spectral structuring of home videos. In: Bakker, E.M., Lew, M.S., Huang, T.S., Sebe, N., Zhou, X.S. (eds) CIVR 2003. LNCS, vol. 2728, pp. 310–320. Springer, Heidelberg (2003)

    Google Scholar 

  14. Pavan M., Pelillo M.: Dominant sets and pairwise clustering. IEEE Trans. Pattern Anal. Mach. Intell. 29, 167–172 (2007)

    Article  Google Scholar 

  15. Peng Y., Ngo C.W.: Clip-based similarity measure for query-dependent clip retrieval and video summarization. IEEE Trans. Circuits Syst. Video Technol. 16, 612–627 (2006)

    Article  Google Scholar 

  16. Perona P., Freeman W.: A factorization approach to grouping. In: Burkhardt, H., Neumann, B. (eds) ECCV”98. LNCS, vol. 1406, pp. 655–670. Springer, Heidelberg (1998)

    Google Scholar 

  17. Petersohn, C.: Fraunhofer HHI at TRECVID 2004: Shot boundary detection system. TREC Video Retrieval Evaluation Online Proceedings. TRECVID 2004. http://www-nlpir.nist.gov/projects/tvpubs/tvpapers04/fraunhofer.pdf. Accessed 27 Apr 2006

  18. Radhakrishnan, R., Divakaran, A., Xiong, Z., Otsuka, I.: A content-adaptive analysis and representation framework for audio event discovery from unscripted multimedia. EURASIP J. Appl. Signal Process. (2006). doi:10.1155/ASP/2006/89013

  19. Rasheed Z., Shah M.: Detection and representation of scenes in videos. IEEE Trans. Multimedia 7, 1097–1105 (2005)

    Article  Google Scholar 

  20. Rui Y., Huang T.S., Mehrotra S.: Constructing table of content for videos. Multimedia Syst. 7, 359–368 (1999)

    Article  Google Scholar 

  21. Sakarya U., Telatar Z.: Graph-based multilevel temporal segmentation of scripted content videos. In: Escolano, F., Vento, M. (eds) GbRPR 2007. LNCS, vol. 4538, pp. 168–179. Springer, Heidelberg (2007)

    Google Scholar 

  22. Sakarya, U., Telatar, Z.: Graph partition based scene boundary detection. In: Petrou, M., Saramäki, T., Erçil, A., Lončarić, S. (eds.) Proceedings of the 5th International Symposium on Image and Signal Processing and Analysis (ISPA 2007), Istanbul, Turkey, pp. 544–549, 27–29 Sept 2007

  23. Shi J., Malik J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 888–905 (2000)

    Article  Google Scholar 

  24. Smeaton, A.F., Over, P., Kraaij, W.: Evaluation campaigns and TRECVid. In: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval (Santa Barbara, CA, USA, 26–27 Oct, 2006). MIR’06, pp. 321–330. ACM Press, New York (2006)

  25. Song B.C., Ra J.B.: Automatic shot change detection algorithm using multi-stage clustering for MPEG-compressed videos. J. Vis. Commun. Image Represent. 12, 364–385 (2001)

    Article  Google Scholar 

  26. Sundaram H., Chang S.F.: Computable scenes and structures in films. IEEE Trans. Multimedia 4, 482–491 (2002)

    Article  Google Scholar 

  27. Tavanapong W., Zhou J.: Shot clustering techniques for story browsing. IEEE Trans. Multimedia 6, 517–527 (2004)

    Article  Google Scholar 

  28. The Internet Movie Database. http://www.imdb.com/. Accessed 21 Jan 2008

  29. Uchihashi, S., Foote, J.: Summarizing video using a shot importance measure and a frame-packing algorithm. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 1999, vol. 6, pp. 3041–3044 (1999)

  30. Vasconcelos N., Lippman A.: Statistical models of video structure for content analysis and characterization. IEEE Trans. Image Process. 9, 3–19 (2000)

    Article  Google Scholar 

  31. Wang S., Siskind J.M.: Image segmentation with ratio cut. IEEE Trans. Pattern Anal. Mach. Intell. 25, 675–690 (2003)

    Article  Google Scholar 

  32. Yaşaroğlu Y., Alatan A.A.: Summarizing video: Content, features, and HMM topologies. In: García, N., Martínez, J.M., Salgado, L. (eds) VLBV 2003. LNCS, vol. 2849, pp. 101–110. Springer, Heidelberg (2003)

    Google Scholar 

  33. Yeung M., Yeo B.L., Liu B.: Segmentation of video by clustering and graph analysis. Comput. Vis. Image Underst. 71, 94–109 (1998)

    Article  Google Scholar 

  34. Yuan J., Wang H., Xiao L., Zheng W., Li J., Lin F., Zhang B.: A formal study of shot boundary detection. IEEE Trans. Circuits Syst. Video Technol. 17, 168–186 (2007)

    Article  Google Scholar 

  35. Zhai Y., Shah M.: Video scene segmentation using Markov chain Monte Carlo. IEEE Trans. Multimedia 8, 686–697 (2006)

    Article  Google Scholar 

  36. Zhao, Y., Wang, T., Wang, P., Hu, W., Du, Y., Zhang, Y., Xu, G.: Scene segmentation and categorization using Ncuts. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7, 17–22 June, 2007

Download references

Author information

Authors and Affiliations

  1. TÜBİTAK UZAY (The Scientific and Technological Research Council of Turkey, Space Technologies Research Institute), ODTÜ Yerleşkesi, 06531, Ankara, Turkey

    Ufuk Sakarya

  2. Faculty of Engineering, Department of Electronics Engineering, Ankara University, Beşevler, 06100, Ankara, Turkey

    Ufuk Sakarya & Ziya Telatar

Authors
  1. Ufuk Sakarya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziya Telatar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ufuk Sakarya.

Additional information

Communicated by Changsheng Xu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sakarya, U., Telatar, Z. Graph-based multilevel temporal video segmentation. Multimedia Systems 14, 277–290 (2008). https://doi.org/10.1007/s00530-008-0145-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-008-0145-x

Keywords

Search

Navigation