Skip to main content
SpringerLink
Log in

Perceptual synoptic view-based video retrieval using metadata

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Content-based video retrieval and video synopsis are generally considered as two different areas. In this paper, we present an efficient approach for video retrieval based on the perceptual synopsis database of the videos. Video synopsis encapsulates an overview of a shot in a single frame. This is the first time video synopsis is used for video indexing providing the user an intuitive link for accessing actions in the video. We propose an enhanced synopsis called meta synopsis for the video database index, which will contain all essential information for retrieval. Various information such as background of a scene, motion trajectory of the foreground objects, color, texture, and mutual information in the synopsis database will empower us to retrieve relevant video content from huge video databases. Experiments were conducted on the OVP, BBC Motion Gallery, TRECVID data set, and other videos. Instead of using key frames as the query frames, the method accepts any arbitrary query frames. The experimental results illustrate that our proposed method can accurately identify a pertinent video from huge video databases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Hu, W., Xie, D., Fu, Z., Zeng, W., Maybank, S.: Semantic-based surveillance video retrieval. IEEE Trans. Image Process. 16(4), 1168–1181 (2007)

    Article  MathSciNet  Google Scholar 

  2. Stringa, E., Regazzoni, C.: Content-based retrieval and real time detection from video sequences acquired by surveillance systems. In: Proceedings on ICIP’98, pp. 138–142. IEEE (1998)

  3. Hu, W., Xie, N., Li, L., Zeng, X., Maybank, S.: A survey on visual content-based video indexing and retrieval. IEEE Trans. Syst. Man Cybern. C Appl. Rev. 41(6), 797–819 (2011)

    Article  Google Scholar 

  4. Chun, Y.D., Kim, N.C., Jang, I.: Content-based image retrieval using multiresolution color and texture features. IEEE Trans. Multimedia 10(6), 1073–1084 (2008)

    Article  Google Scholar 

  5. Lin, T., Ngo, C., Zhang, H., Shi, Q.: Integrating color and spatial features for content-based video retrieval. In: Proceedings on ICIP’01, vol. 3, pp. 592–595. IEEE (2001)

  6. An, J., Lee, S., Cho, N.: Content-based image retrieval using color features of salient regions. In: Proceedings on ICIP’14, pp. 3042–3046. IEEE (2014)

  7. Lie, W., Hsiao, W.: Content-based video retrieval based on object motion trajectory. In: Proceedings on MMSP’02, pp. 237–240. IEEE (2002)

  8. Hsieh, J., Yu, S., Chen, Y.: Motion-based video retrieval by trajectory matching. IEEE Trans. Circuits Syst. Video Technol. 16(3), 396–409 (2006)

    Article  Google Scholar 

  9. Wang, S., Yang, J., Yi, D., Wang, Z.: Video retrieval synopsis for moving objects. In: Proceedings on ICMEW’13, pp. 1–2. IEEE (2013)

  10. Dyana, A., Das, S.: MST-CSS (Multi-Spectro-Temporal Curvature Scale Space), a novel spatio-temporal representation for content-based video retrieval. IEEE Trans. Circuits Syst. Video Technol. 20(8), 1080–1094 (2010)

    Article  Google Scholar 

  11. Chattopadhyay, C., Das, S.: Use of trajectory and spatiotemporal features for retrieval of videos with a prominent moving foreground object. Signal Image Video Process. 10(2), 319–326 (2016)

    Article  Google Scholar 

  12. Tapaswi, M., Bauml, M., Stiefelhagen, R.: Aligning plot synopses to videos for story-based retrieval. Int. J. Multimedia Inf. Retr. 4(1), 3–16 (2015)

    Article  Google Scholar 

  13. Zhang, H., Wang, J., Altunbasak, Y.: Content-based video retrieval and compression: a unified solution. In: Proceedings on ICIP’97, vol. 1, pp. 13–16. IEEE (1997)

  14. Papushoy, A., Bors, A.: Visual attention for content based image retrieval. In: Proceedings on ICIP’15, pp. 971–975. IEEE (2015)

  15. Zhu, X., Elmagarmid, A., Xue, X., Wu, L., Catlin, A.: InsightVideo: toward hierarchical video content organization for efficient browsing, summarization and retrieval. IEEE Trans. Multimedia 7(4), 648–666 (2005)

    Article  Google Scholar 

  16. Lin, T., Yang, M., Tsai, C., Wang, Y.: Query-adaptive multiple instance learning for video instance retrieval. IEEE Trans. Image Process. 24(4), 1330–1340 (2015)

    Article  MathSciNet  Google Scholar 

  17. Sze, K., Lam, K., Qiu, G.: A new key frame representation for video segment retrieval. IEEE Trans. Circuits Syst. Video Technol. 15(9), 1148–1155 (2005)

    Article  Google Scholar 

  18. Qi, X., Chang, R.: A fuzzy statistical correlation-based approach to content-based image retrieval. In: Proceedings on ICME’08, pp. 1265–1268. IEEE (2008)

  19. Niu, J.,Wang, Z., Feng, D.: Two-step similarity matching for Content-Based Video Retrieval in P2P networks. In: Proceedings on ICME’10, pp. 1690–1694. IEEE (2010)

  20. Beecks, C., Uysal, M., Seidl, T.: A comparative study of similarity measures for content-based multimedia retrieval. In: Proceedings on ICME’10, pp. 1552–1557. IEEE (2010)

  21. Yang, H., Meinel, C.: Content based lecture video retrieval using speech and video text information. IEEE Trans. Learn. Technol. 7(2), 142–154 (2014)

    Article  Google Scholar 

  22. Gen, S., Bastan, M., Gdkbay, U., Atalay, V., Ulusoy, z: Handvr: a hand-gesture-based interface to a video retrieval system. Signal Image Video Process. 7, 1717–1726 (2015)

    Google Scholar 

  23. Vallet, D., Hopfgartner, F., Halvey, M., Jose, J.M.: Community based feedback techniques to improve video search. Signal Image Video Process. 2(4), 289–306 (2008)

    Article  Google Scholar 

  24. Basharat, A., Zhai, Y., Shah, M.: Content based video matching using spatiotemporal volumes. Comput. Vis. Image Underst. 110(3), 360–377 (2008)

    Article  Google Scholar 

  25. Thomas, S.S., Gupta, S., Venkatesh, K.S.: Perceptual video summarization-a new framework for video summarization (accepted). IEEE Trans. Circuits Syst. Video Technol. (2016). doi:10.1109/TCSVT.2016.2556558

  26. Chang, H.C., Yang, C.K.: Fast content-aware video length reduction. Signal Image Video Process. 8(7), 1383–1397 (2014)

    Article  Google Scholar 

  27. Xu, Z.: Consistent image alignment for video mosaicing. Signal Image Video Process. 7(1), 129–135 (2013)

    Article  Google Scholar 

  28. Assfalg, J., Del Bimbo, A., Hirakawa, M.: A mosaic-based query language for video databases. In: Proceedings on VL’00, pp. 31–38. IEEE (2000)

  29. Jain, A., Vailaya, A., Xiong, W.: Query by video clip. In: Proceedings on ICPR’98, vol. 1, pp. 909–911. IEEE (1998)

  30. Tsai, D.M., Chiu, W.Y., Lee, M.H.: Optical flow-motion history image (OF-MHI) for action recognition. Signal Image Video Process. 9(8), 1897–1906 (2015)

    Article  Google Scholar 

  31. Pritch, Y., Acha, A.R., Peleg, S.: Nonchronological video synopsis and indexing. IEEE Trans. Pattern Anal. Mach. Intell. 30(11), 1971–1984 (2008)

    Article  Google Scholar 

  32. Thomas, S.S., Gupta, S., Venkatesh, K.S.: Perceptual synoptic view of pixel, object and semantic based attributes of video. J. Vis. Commun. Image Represent. 38, 367–377 (2016)

    Article  Google Scholar 

  33. Howarth, P., Rger, S.: Evaluation of texture features for content-based image retrieval. In: Image and Video Retrieval, Lecture Notes in Computer Science, vol. 3115, pp. 326–334. (2004)

Download references

Acknowledgments

The authors would like to thank the anonymous reviewer and the editor for the constructive and thoughtful comments and useful suggestions that helped them in improving the quality, presentation, and organization of this paper.

Author information

Authors and Affiliations

  1. Electrical Engineering Department, Indian Institute of Technology Kanpur, Kanpur, UP, India

    Sinnu Susan Thomas, Sumana Gupta & K. S. Venkatesh

Authors
  1. Sinnu Susan Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sumana Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. S. Venkatesh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sinnu Susan Thomas.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thomas, S.S., Gupta, S. & Venkatesh, K.S. Perceptual synoptic view-based video retrieval using metadata. SIViP 11, 549–555 (2017). https://doi.org/10.1007/s11760-016-0993-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-016-0993-3

Keywords

Search

Navigation