Skip to main content

Advertisement

Springer Nature Link
Log in

VideoGraph: a non-linear video representation for efficient exploration

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper we introduce VideoGraph, a novel non-linear representation for scene structure of a video. Unlike classical linear sequential organization, VideoGraph concentrates the video content across the time line by structuring scenes and materializes with two-dimensional graph, which enables non-linear exploration on the scenes and their transitions. To construct VideoGraph, we adopt a sub-shot induced method to evaluate the spatio-temporal similarity between shot segments of video. Then, scene structure is derived by grouping similar shots and identifying the valid transitions between scenes. The final stage is to represent the scene structure using a graph with respect to scene transition topology. Our VideoGraph can provide a condensed representation in the scene level and facilitate a non-linear manner to browse videos. Experimental results are presented to demonstrate the effectiveness and efficiency by using VideoGraph to explore and access the video content.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Hu, S.M., Chen, T., Xu, K., Cheng, M.M., Martin, R.R.: Internet visual media processing: a survey with graphics and vision applications. Vis. Comput. 29(5), 393–405 (2013)

    Article  Google Scholar 

  2. Lu, S.P., Zhang, S.H., Wei, J., Hu, H.M., Martin, R.R.: Time-line editing of objects in video. IEEE Trans. Vis. Comput. Graph. 19(7), 1218–1227 (2013)

    Article  Google Scholar 

  3. Li, F., Gupta, A., Sanocki, E., He, L., Rui, Y.: Browsing digital video. In: Proc. SIGCHI, pp. 169–176 (2000)

    Google Scholar 

  4. Daniel, G., Chen, M.: Video visualization. In: Proc. Visualization, pp. 409–416 (2003)

    Google Scholar 

  5. Chen, M., Botchen, R., Hashim, R., Weiskopf, D., Ertl, T., Thornton, I.: Visual signatures in video visualization. IEEE Trans. Vis. Comput. Graph. 12(5), 1093–1100 (2006)

    Article  Google Scholar 

  6. Kim, K., Essa, I., Abowd, G.D.: Interactive mosaic generation for video navigation. In: ACM Multimedia, pp. 655–658 (2006)

    Google Scholar 

  7. Mei, T., Yang, B., Yang, S.Q., Hua, X.S.: Video collage: presenting a video sequence using a single image. Vis. Comput. 25(1), 39–51 (2008)

    Article  Google Scholar 

  8. Barnes, C., Goldman, D.B., Shechtman, E., Finkelstein, A.: Video tapestries with continuous temporal zoom. ACM Trans. Graph. 89, 1 (2010)

    Article  Google Scholar 

  9. Correa, C.D., Ma, K.L.: Dynamic video narratives. ACM Trans. Graph. 88, 1 (2010)

    Article  Google Scholar 

  10. Cong, L., Tong, R., Dong, J.: Selective image abstraction. Vis. Comput. 27(3), 187–198 (2001)

    Article  Google Scholar 

  11. Zhang, L., Huang, H.: Hierarchical narrative collage for digital photo album. Comput. Graph. Forum 31, 2173–2181 (2012)

    Article  Google Scholar 

  12. Smith, M.A., Kanade, T.: Video skimming and characterization through the combination of image and language understanding techniques. In: Proc. CVPR, pp. 775–781 (1997)

    Google Scholar 

  13. Sundaram, H., Xie, L., Chang, S.F.: A utility framework for the automatic generation of audio-visual skims. In: ACM Multimedia, pp. 189–198 (2002)

    Google Scholar 

  14. Teodosio, L., Bender, W.: Salient stills. ACM Trans. Multimed. Comput. Commun. Appl. 1(1), 16–36 (2005)

    Article  Google Scholar 

  15. Caspi, Y., Axelrod, A., Matsushita, Y., gamliel, A.: Dynamic stills and clip trailers. Vis. Comput. 22(9), 642–652 (2006)

    Article  Google Scholar 

  16. Rav-Acha, A., Pritch, Y., Peleg, S.: Making a long video short: dynamic video synopsis. In: Proc. CVPR, pp. 435–441 (2006)

    Google Scholar 

  17. Dragicevic, P., Ramos, G., Bibliowitcz, J., Nowrouzezahrai, D., Balakrishnan, R., Singh, K.: Dragon: a direct manipulation interface for frame-accurate in-scene video navigation. In: Proc. SIGCHI, pp. 247–250 (2008)

    Google Scholar 

  18. Schoeffmann, K., Boeszoermenyi, L.: Video browsing using interactive navigation summaries. In: International Workshop on Content-Based Multimedia Indexing 7, pp. 243–248 (2002)

    Google Scholar 

  19. Chen, T., Lu, A.D., Hu, S.M.: Visual storylines: semantic visualization of movie sequence. Comput. Graph. 36(4), 241–249 (2012)

    Article  Google Scholar 

  20. Erol, B., Lee, D.S., Hul, J.: Multimodal summarization of meeting recording. In: Proc. ICME, pp. 25–28 (2003)

    Google Scholar 

  21. Kim, J.G., Chang, H.S., Kang, K., Kim, M., Kim, J., Kim, H.M.: Summarization of news video and its description for content-based access. Int. J. Imaging Syst. Technol. 13(5), 267–274 (2003)

    Article  Google Scholar 

  22. Truong, B., Venkatesh, S.: Video abstraction: a systematic review and classification. ACM Trans. Multimedia Comput. Commun. Appl. 3(1) (2007)

  23. Lu, S., King, I.K., Lyu, M.R.: Video summarization by video structure analysis and graph optimization. In: Proc. ICME, pp. 1959–1962 (2004)

    Google Scholar 

  24. Sidiropoulos, P., Mezaris, V., Kompatsiaris, I., Meinedo, H., Trancoso, I.: Multi-modal scene segmentation using scene transition graphs. In: ACM Multimedia, pp. 665–668 (2009)

    Google Scholar 

  25. Feng, B.L., Cao, J., Bao, X.J., Bao, L., Zhang, Y.D., Lin, S.X., Yun, X.C.: Graph-based multi-space semantic correlation propagation for video retrieval. Vis. Comput. 27(1), 21–34 (2011)

    Article  Google Scholar 

  26. Yu, L., Lu, A.D., Ribarsky, W., Chen, W.: Automatic animation for time-varying data visualization. Comput. Graph. Forum 29(7), 2271–2280 (2010)

    Article  Google Scholar 

  27. Steiner, T., Verborgh, R., Vallés, J.G., Hausenblas, M., Troncy, R., de Walle, R.V.: Enabling on-the-fly video shot detection on youtube. In: Proc. WWW (2012)

    Google Scholar 

  28. Tang, L.X., Mei, T., Hua, X.S.: Near-lossless video summarization. In: ACM Multimedia, pp. 351–360 (2009)

    Google Scholar 

  29. Jia, Y.T., Hu, S.M., Martin, R.R.: Video completion using tracking and fragment merging. Vis. Comput. 21(8–10), 601–610 (2005)

    Article  Google Scholar 

  30. Lowe, D.G.: Object recognition from local scale invariant features. In: Proc. ICCV, pp. 1150–1157 (1999)

    Google Scholar 

  31. Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315, 972–976 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  32. Corrigan, T., White, P.: The Film Experience: An Introduction. (2004)

    Google Scholar 

  33. Harary, F.: In: Graph Theory (1994)

    Google Scholar 

  34. Tollis, I., Di Battista, G., Eades, P., Tamassia, R.: Graph Drawing: Algorithms for the Visualization of Graphs. (1998)

    Google Scholar 

  35. Ellson, J., Gansner, E., Koutsofios, E., North, S., Woodhull, G.: Graphviz—open source graph drawing tools. In: Proc. Graph Drawing, pp. 483–484 (2001)

    Google Scholar 

  36. Rother, C., Bordeaux, L., Hamadi, Y., Blake, A.: Autocollage. ACM Trans. Graph. 25(3), 847–852 (2006)

    Article  Google Scholar 

Download references

Acknowledgements

We thank the anonymous reviewers for their valuable comments. This work was partly supported by the grants from the National Natural Science Foundation of China (No. 61133008, 61103159) and Excellent Young Scholars Research Fund of Beijing Institute of Technology (No. 2012YR0709).

Author information

Authors and Affiliations

  1. Beijing Key Lab of Intelligent Information Technology, School of Computer Science, Beijing Institute of Technology, Beijing, China

    Lei Zhang, Qian-Kun Xu, Lei-Zheng Nie & Hua Huang

Authors
  1. Lei Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Qian-Kun Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Lei-Zheng Nie
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Hua Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Hua Huang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, L., Xu, QK., Nie, LZ. et al. VideoGraph: a non-linear video representation for efficient exploration. Vis Comput 30, 1123–1132 (2014). https://doi.org/10.1007/s00371-013-0882-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-013-0882-5

Keywords