Skip to main content
SpringerLink
Log in

An auto-encoder-based summarization algorithm for unstructured videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video summarization is an effective way to quick view videos and relieve the pressure of videos storage. However the traditional algorithms are hardly adapted to unstructured videos, due to the unobvious for scenes changing and ignoring the structure of the videos. Therefore, an Auto-encoder-based summarization algorithm is proposed in this paper for unstructured videos. Each video structure is detected by an Auto-encoder and both of the interestingness and representativeness of each video segment are predicted by the reconstruction errors of the segment. Meanwhile, most interesting and representative summarization is generated with the limited summary length. The experimental results show that the proposed algorithm obtained a better performance by comparing with the state-of-the-art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Avila SEFD, Lopes APB, Luz AD et al (2011) VSUMM: a mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recogn Lett 32(1):56–68

    Article  Google Scholar 

  2. Basak J, Luthra V, Chaudhury S (2008) Video summarization with supervised learning. Pattern Recognition, 2008. ICPR 2008. 19th International Conference on, IEEE

  3. Chu W-S, Song Y, Jaimes A (2015) Video co-summarization: Video summarization by visual co-occurrence. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  4. Dang CT, Radha H (2014) Heterogeneity image patch index and its application to consumer video summarization. Image Processing, IEEE Transactions on 23(6):2704–2718

    Article  MathSciNet  MATH  Google Scholar 

  5. Gong B et al. (2014) Diverse sequential subset selection for supervised video summarization. Advances in Neural Information Processing Systems

  6. Gygli M et al. (2014) Creating summaries from user videos. Computer Vision–ECCV 2014. Springer International Publishing, pp 505–520

  7. Gygli M, Grabner H, Van Gool L (2015) Video summarization by learning submodular mixtures of objectives. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  8. Japkowicz N, Myers C, Gluck M (1995) A novelty detection approach to classification. IJCAI

  9. Kang H-W, Hua X-S (2005) To learn representativeness of video frames. Proceedings of the 13th annual ACM international conference on Multimedia, ACM

  10. Lee YJ, Ghosh J, Grauman K (2012) Discovering important people and objects for egocentric video summarization. CVPR, 2. no. 6

  11. Li K, Wang J, Wang H et al (2015) Structuring lecture videos by automatic projection screen localization and analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on 37(6):1233–1246

    Article  MathSciNet  Google Scholar 

  12. Lin C-Y (2004) Rouge: A package for automatic evaluation of summaries. Text summarization branches out: Proceedings of the ACL-04 workshop 8

  13. Lu Z, Grauman K (2013) Story-driven summarization for egocentric video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  14. Luan Q et al. (2014) Video Summarization based on Nonnegative Linear Reconstruction. Multimedia and Expo (ICME), 2014 I.E. International Conference on, IEEE

  15. Mahmoud KM, Ghanem NM, Ismail MA (2013) Unsupervised video summarization via dynamic modeling-based hierarchical clustering. Machine Learning and Applications (ICMLA), 2013 12th international conference on, 2. IEEE

  16. Manevitz LM, Yousef M (2002) One-class SVMs for document classification. J Mach Learn Res 2:139–154

    MATH  Google Scholar 

  17. Masci J et al. (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. Artificial Neural Networks and Machine Learning–ICANN 2011, Springer, Berlin Heidelberg, pp 52–59

  18. Mei S, Guan G, Wang Z et al (2015) Video summarization via minimum sparse reconstruction. Pattern Recogn 48(2):522–533

    Article  Google Scholar 

  19. Money AG, Agius H (2008) Video summarisation: a conceptual framework and survey of the state of the art. J Vis Commun Image Represent 19(2):121–143

    Article  Google Scholar 

  20. Potapov D et al. (2014) Category-specific video summarization. European conference on computer vision. Springer International Publishing

  21. Rumelhart DE, Hinton GE, Williams RJ (1985) Learning internal representations by error propagation. California Univ San Diego la Jolla Inst for Cognitive Science

  22. Scovanner P, Ali S, Shah M (2007) A 3-dimensional sift descriptor and its application to action recognition. Proceedings of the 15th international conference on Multimedia, ACM

  23. Sun M, Farhadi A, Seitz S (2014) Ranking domain-specific highlights by analyzing edited videos. European conference on computer vision, Springer International Publishing

  24. Truong BT, Venkatesh S (2007) Video abstraction: A systematic review and classification. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 3(1):3

    Article  Google Scholar 

  25. Tsai CM, Kang LW, Lin CW et al (2013) Scene-based movie summarization via role-community networks. IEEE Trans Circuits Syst Video Technol 23(11):1927–1940

    Article  Google Scholar 

  26. Valdés V, Martínez JM (2012) On-line video abstract generation of multimedia news. Multimedia Tools and Applications 59(3):795–832

    Article  Google Scholar 

  27. Wang Z, Yu J, He Y et al (2014) Affection arousal based highlight extraction for soccer video. Multimedia Tools and Applications 73(1):519–546

    Article  Google Scholar 

  28. Weninger F et al. (2014) Deep recurrent de-noising auto-encoder and blind de-reverberation for reverberated speech recognition. Acoustics, Speech and Signal Processing (ICASSP), 2014 I.E. International Conference on, IEEE

  29. Xu J et al. (2015) Gaze-enabled egocentric video summarization via constrained submodular maximization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  30. Yang H et al. (2015) Unsupervised extraction of video highlights via robust recurrent auto-encoders. Proceedings of the IEEE International Conference on Computer Vision

  31. Yeung S, Fathi A, Li F-F (2014) Videoset: Video summary evaluation through text. arXiv preprint arXiv:1406.5824

  32. Zhao B, Xing E (2014) Quasi real-time summarization for consumer videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (No.61370121), the National Hi-Tech Research and Development Program (863 Program) of China (No.2014AA015102).

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Digital Media, School of Computer Science and Engineering, Beihang University, Beijing, 100083, China

    Meng-Xiong Han, Hai-Miao Hu, Chi Zhang, Rong-Peng Tian & Jin Zheng

  2. Beijing Institute of Graphics, Beijing, 100029, China

    Yang Liu

Authors
  1. Meng-Xiong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai-Miao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rong-Peng Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hai-Miao Hu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, MX., Hu, HM., Liu, Y. et al. An auto-encoder-based summarization algorithm for unstructured videos. Multimed Tools Appl 76, 25039–25056 (2017). https://doi.org/10.1007/s11042-017-4485-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4485-4

Keywords

Search

Navigation