Skip to main content
SpringerLink
Log in

Compressed Domain Video Abstraction Based on I-Frame of HEVC Coded Videos

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Video abstraction allows indexing, searching, browsing and evaluating a video only by accessing its useful contents. Several studies have been done in this field, but most of them are in pixel domain and require decoding process. It makes these methods more time and process consuming than compressed domain video abstraction. In this paper, we present a new video abstraction method in H.265/HEVC compressed domain, HVAIF. The method is based on the normalized histogram of extracted I-frame prediction modes from an H.265/HEVC coded video. The frames’ similarity is calculated by intersecting their I-frame prediction modes’ histogram. The similarity measure detects and removes redundant key-frames to increase the quality of final video abstraction. Moreover, we employ fuzzy c-means clustering to categorize similar frames and extract key-frames as representatives of the entire video frames. The interpretation of the results shows that using the proposed method achieves on average 86% accuracy and 19% error rate in compressed domain video abstraction which is higher than the other tested methods in the pixel domain. Also, it has an acceptable robustness to coding parameters, and on average it generates video key-frames that are closer to human summaries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. J. Almeida, N.J. Leite, R.D.S. Torres, Online video summarization on compressed domain. J. Vis. Commun. Image Represent. 24(6), 729–738 (2013). https://doi.org/10.1016/j.jvcir.2012.01.009

    Article  Google Scholar 

  2. S.E.D. Avila, A.P.B. Lopes, A. Da Luz, A.D.A. Araújo, VSUMM: a mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recognit. Lett. 32(1), 56–68 (2011). https://doi.org/10.1016/j.patrec.2010.08.004

    Article  Google Scholar 

  3. S. Cha, Comprehensive survey on distance/similarity measures between probability density functions. Int. J. Math. Models Methods Appl. Sci. 1(4), 300–307 (2007). https://doi.org/10.1007/s00167-009-0884-z

    Google Scholar 

  4. T. Chheng, Video Summarization Using Clustering (Department of Computer Science University of California, Irvine, 2007)

    Google Scholar 

  5. S. De Bruyne, D. Van Deursen, J. De Cock, W. De Neve, P. Lambert, R. Van de Walle, A compressed-domain approach for shot boundary detection on H.264/AVC bit streams. Signal Process. Image Commun. 23(7), 473–489 (2008). https://doi.org/10.1016/j.image.2008.04.012

    Article  Google Scholar 

  6. D. DeMenthon, V. Kobla, D. Doermann, Video summarization by curve simplification. In: Proceedings of the ACM International Conference on Multimedia, New York, USA (1998), pp. 211–218. https://doi.org/10.1145/290747.290773

  7. R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2nd edn. (Wiley, New York, 2001)

    MATH  Google Scholar 

  8. N. Ejaz, T.B. Tariq, S.W. Baik, Adaptive key frame extraction for video summarization using an aggregation mechanism. J. Vis. Commun. Image Represent. 23(7), 1031–1040 (2012). https://doi.org/10.1016/j.jvcir.2012.06.013

    Article  Google Scholar 

  9. M. Furini, F. Geraci, M. Montangero, M. Pellegrini, STIMO: STIll and MOving video storyboard for the web scenario. Multimed. Tools Appl. 46(1), 47–69 (2010). https://doi.org/10.1007/s11042-009-0307-7

    Article  Google Scholar 

  10. H.265/HEVC Reference Software, https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/. Last seen on Feb 2017

  11. A. Hanjalic, H. Zhang, An Integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE Trans. Circuits Syst. 9(8), 1280–1289 (1999). https://doi.org/10.1109/76.809162

    Google Scholar 

  12. J. He, F. Yang, Y. Zhou, High-speed implementation of rate-distortion optimised quantisation for H.265/HEVC. IET Image Process. 9(8), 652–661 (2015). https://doi.org/10.1049/iet-ipr.2014.0849

    Article  Google Scholar 

  13. L. Herranz, J.M. Martínez, An efficient summarization algorithm based on clustering and bitstream extraction. In: Proceedings of International Conference on Multimedia and Expo (2009), pp. 654–657. https://doi.org/10.1109/icme.2009.5202581

  14. W. Hu, S. Member, N. Xie, L. Li, X. Zeng, S. Maybank, A survey on visual content-based video indexing and retrieval. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 41(6), 797–819 (2011). https://doi.org/10.1109/TSMCC.2011.2109710

    Article  Google Scholar 

  15. J. Kavitha, P.A.J. Rani, Static and multiresolution feature extraction for video summarization. Procedia Comput. Sci. 47(C), 292–300 (2015). https://doi.org/10.1016/j.procs.2015.03.209

    Article  Google Scholar 

  16. J. Li, T. Yao, Q. Ling, T. Mei, Detecting shot boundary with sparse coding for video summarization. Neurocomputing 266, 66–78 (2017). https://doi.org/10.1016/j.neucom.2017.04.065

    Article  Google Scholar 

  17. Y. Li, T. Zhang, D. Tretter, An overview of video abstraction techniques an overview of video abstraction techniques. Imaging, 1–23 (2001). Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.6173&rep=rep1&type=pdf

  18. T. Liu, X. Zhang, J. Feng, K.T. Lo, Shot reconstruction degree: a novel criterion for key frame selection. Pattern Recognit. Lett. 25(12), 1451–1457 (2004). https://doi.org/10.1016/j.patrec.2004.05.020

    Article  Google Scholar 

  19. A.G. Money, H. Agius, Video summarization: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121–143 (2008). https://doi.org/10.1016/j.jvcir.2007.04.002

    Article  Google Scholar 

  20. P. Mundur, Y. Rao, Y. Yesha, Key-frame based video summarization using Delaunay clustering. Int. J. Digit. Libr. 6(2), 219–232 (2006). https://doi.org/10.1007/s00799-005-0129-9

    Article  Google Scholar 

  21. J.H. Oh, Q. Wen, S. Hwang, J. Lee, Video abstraction. In: Video Data Management and Information Retrieval (2005), pp. 321–346

  22. F. Rahmani, F. Zargari, Compressed domain visual information retrieval based on I-frames in HEVC. Multimed. Tools Appl. (2016). https://doi.org/10.1007/s11042-016-3391-5

    Google Scholar 

  23. G.-H. Song, Q.G. Ji, Z.-M. Lu, Z.D. Fang, Z.H. Xie, A novel video abstraction method based on fast clustering of the regions of interest in keyframes. Int. J. Electron. Commun. (AEÜ) 68, 237–243 (2014). https://doi.org/10.1016/j.aeue.2014.03.004

    Article  Google Scholar 

  24. X. Song, G. Fan, Joint key-frame extraction and object segmentation for content-based video analysis. IEEE Trans. Circuits Syst. Video Technol. 16(7), 904–914 (2006). https://doi.org/10.1109/TCSVT.2006.877419

    Article  Google Scholar 

  25. M. Srinivas, M.M.M. Pai, R.M. Pai, An improved algorithm for video summarization—a rank based approach. Procedia Comput. Sci. 89, 812–819 (2016). https://doi.org/10.1016/j.procs.2016.06.065

    Article  Google Scholar 

  26. G.J. Sullivan, J. Ohm, W. Han, T. Wiegand, Overview of the high efficiency video coding. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012). https://doi.org/10.1109/TCSVT.2012.2221191

    Article  Google Scholar 

  27. Z. Sun, K. Jia, H. Chen, Video keyframe extraction based on spatial-temporal color distribution. In: Proceedings—4th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP (2008), pp. 196–199. https://doi.org/10.1109/iih-msp.2008.245

  28. B.T. Truong, S. Venkatesh, Video abstraction: a systematic review and classification. ACM Trans. Multimed. Comput. Commun. Appl. 3(1), 1–37 (2007). https://doi.org/10.1145/1198302.1198305

    Article  Google Scholar 

  29. J. Wu, S. Zhong, J. Jiang, Y. Yang, A novel clustering method for static video summarization. Multimed. Tools Appl. 76(7), 9625–9641 (2017). https://doi.org/10.1007/s11042-016-3569-x

    Article  Google Scholar 

  30. L. Xiang-wei, Z. Li-dong, Z. Kai, Hierarchical video summarization extraction algorithm in compressed domain. Phys. Procedia 24, 2360–2366 (2012). https://doi.org/10.1016/j.phpro.2012.02.350

    Article  Google Scholar 

  31. W. Yao, Z. Li, S. Rahardja, Dynamic threshold-based keyframe detection and its application in rate control. IET Image Process. 6(7), 986 (2012). https://doi.org/10.1049/iet-ipr.2011.0189

    Article  Google Scholar 

  32. X.D. Zhang, T.Y. Liu, K.T. Lo, J. Feng, Dynamic selection and effective compression of keyframes for video abstraction. Pattern Recognit. Lett. 24(9–10), 1523–1532 (2003). https://doi.org/10.1016/S0167-8655(02)00391-4

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Ali Reza Yamghani

  2. Department of Information Technology, Research Institute for ICT, Tehran, Iran

    Farzad Zargari

Authors
  1. Ali Reza Yamghani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farzad Zargari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farzad Zargari.

Additional information

Research Institute for ICT is formerly known as Iran Telecom Research Center (ITRC).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yamghani, A.R., Zargari, F. Compressed Domain Video Abstraction Based on I-Frame of HEVC Coded Videos. Circuits Syst Signal Process 38, 1695–1716 (2019). https://doi.org/10.1007/s00034-018-0932-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-018-0932-3

Keywords

Search

Navigation