Skip to main content
SpringerLink
Log in

VISCOM: A robust video summarization approach using color co-occurrence matrices

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video summarization techniques have allowed the content analysis of large volumes of digital video sequences of different categories, such as movies, documentaries, lectures, sports, surveillance, and news. This paper proposes and evaluates a novel video summarization approach called VISCOM, which is based on color co-occurrence matrices to describe the video frames and generate a synopsis with the most representative frames. Experiments conducted on two different data sets of various genres demonstrate the effectiveness of the proposed method in terms of quality. The resulting video summaries are compared against several others using a specific quantitative evaluation metric, producing competitive outcomes among the evaluated methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ajmal M, Ashraf MH, Shakir M, Abbas Y, Shah FA (2012) Video summarization: techniques and classification. In: International Conference on Computer Vision and Graphics. Springer, Warsaw, Poland, pp 1–13

  2. Almeida J, Leite NJ, Torres RS (2012) VISON: VIdeo summarization for ONline applications. Pattern Recogn Lett 33(4):397–409

    Article  Google Scholar 

  3. Amel AM, Abdessalem BA, Abdellatif M (2010) Video shot boundary detection using motion activity descriptor. J Telecommun 2(1):54–59

    Google Scholar 

  4. Angadi S, Naik V (2014) Entropy Based Fuzzy C Means Clustering and Key Frame Extraction for Sports Video Summarization. In: IEEE Fifth International Conference on Signal and Image Processing, Bangalore, India, pp 271–279

  5. Apostolidis E, Mezaris V (2014) Fast Shot Segmentation Combining Global and Local Visual Descriptors. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp 6583–6587

  6. Arvis V, Debain C, Berducat M, Benassi A (2011) Generalization of the coocurrence matrix for colour images: application to colour texture classification. Image Anal Stereol 23(1):63–72

    Article  MATH  Google Scholar 

  7. Baber J, Afzulpurkar N, Dailey M, Bakhtyar M (2011) Shot boundary detection from videos using entropy and local descriptor

  8. Benni V, Dinesh R, Punitha P, Rao V (2015) Keyframe extraction and shot boundary detection using eigen values. Int J Inform Electron Eng 5(1):40–45

    Article  Google Scholar 

  9. Birinci M, Kiranyaz S (2014) A perceptual scheme for fully automatic video shot boundary detection. Signal Process: Image Commun 29(3):410–423

    Google Scholar 

  10. Borth D, Ulges A, Schulze C, Breuel T (2008) Keyframe extraction for video tagging and summarization. In: G. fur Informatik (ed.) Informatiktage 2008. GI, pp 45–48

  11. Boulkenafet Z, Komulainen J, Hadid A (2015) Face anti-spoofing based on color texture analysis. In: IEEE International Conference on Image Processing. IEEE, Québec City, Canada, pp 2636–2640

  12. Cirne MVM, Pedrini H (2013) Video summarization method based on spectral clustering. In: 18th Iberoamerican Congress on Pattern Recognition, vol 8259, Havana, Cuba, pp 479–486

  13. Cirne MVM, Pedrini H (2014) Summarization of videos by image quality assessment. In: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, Lecture Notes in Computer Science, vol 8827. Springer International Publishing, pp 901–908

  14. De Avila SEF, Lopes APB, Da Luz Jr. A., De Albuquerque Araújo A. (2011) VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method. Pattern Recogn Lett 32(1):56–68

    Article  Google Scholar 

  15. De Bruyne S, De Cock J, Poppe C, Hollemeersch CF, Lambert P, Van de Walle R (2011) Compressed-domain shot boundary detection for H.264/AVC using intra partitioning maps. In: Advances in Multimedia Modeling, Lecture Notes in Computer Science, vol 6523. Springer, Berlin Heidelberg, pp 29–39

  16. Ejaz N, Tariq TB, Baik SW (2012) Adaptive key frame extraction for video summarization using an aggregation mechanism. J Vis Commun Image Represent 23 (7):1031–1040

    Article  Google Scholar 

  17. Fang H, Jiang J, Feng Y (2006) A Fuzzy Logic Approach for Detection of Video Shot Boundaries. Pattern Recogn 39(11):2092–2100

    Article  MATH  Google Scholar 

  18. Furini M, Geraci F, Montangero M, Pellegrini M (2010) STIMO: STIll and MOving video storyboard for the Web scenario. In: Multimedia Tools and Applications, vol 46. Kluwer Academic Publishers, MA,USA, pp 47–69

  19. Gharbi H, Massaoudi M, Bahroun S, Zagrouba E (2016) Key frames extraction based on local features for efficient video summarization. In: Blanc-Talon J., Distante C., Philips W., Popescu D., Scheunders P. (eds) 17th International Conference Advanced Concepts for Intelligent Vision Systems. Springer International Publishing, pp 275–285

  20. Gonzalez TF (1985) Clustering to minimize the maximum intercluster distance. Theor Comput Sci 38:293–306

    Article  MathSciNet  MATH  Google Scholar 

  21. Guan G, Wang Z, Lu S, Deng JD, Feng DD (2013) Keypoint-Based Keyframe Selection. IEEE Trans Circ Syst Video Technol 23(4):729–734. doi:10.1109/TCSVT.2012.2214871

  22. Haralick R, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst, Man Cybern 3(6):610–621

    Article  Google Scholar 

  23. Hild F, Roux S (2012) Comparison of local and global approaches to digital image correlation. Exper Mech 52(9):1503–1519

    Article  Google Scholar 

  24. Islam MB, Kundu K, Ahmed A (2014) Texture feature based image retrieval algorithms. Int J Eng Tech Res 2:170–173

    Google Scholar 

  25. Jacob IJ, Srinivasagan K, Jayapriya K (2014) Local oppugnant color texture pattern for image retrieval system. Pattern Recogn Lett 42:72–78

    Article  Google Scholar 

  26. Janwe N, Bhoyar K (2013) Video shot boundary detection based on JND color histogram. In: IEEE Second International Conference on Image Information Processing, pp 476–480

  27. Jiang H, Zhang G, Wang H, Bao H (2015) Spatio-temporal video segmentation of static scenes and its applications. IEEE Trans Multimed 17(1):3–15

    Article  Google Scholar 

  28. Jiang X, Sun T, Liu J, Chao J, Zhang W (2013) An adaptive video shot segmentation scheme based on dual-detection model. Neurocomputing 116:102–111

    Article  Google Scholar 

  29. Katti H, Yadati K, Kankanhalli M, Tat-Seng C (2011) Affective Video Summarization and Story Board Generation using Pupillary Dilation and Eye Gaze. In: IEEE International Symposium on Multimedia. Dana Point, CA, USA, pp 319–326

  30. Lavanya AL, Sreepada R (2012) A generic frame work for image data clustering via weighted clustering ensemble. Int J Comput Sci Inf Technol 3:5429–5433

    Google Scholar 

  31. Lee K, Kolsch M (2015) Shot boundary detection with graph theory using keypoint features and color histograms

  32. Lee YJ, Ghosh J, Grauman K (2012) Discovering Important People and Objects for Egocentric Video Summarization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1346–1353

  33. Lin T, Zhang H (2000) Automatic video scene extraction by shot grouping. Int Conf Pattern Recogn 4:39–42

    Article  Google Scholar 

  34. Lu Z, Grauman K (2013) Story-Driven Summarization for Egocentric Video. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, DC, USA, pp 2714–2721

  35. Lu ZM, Shi Y (2013) Fast video shot boundary detection based on SVD and pattern matching. IEEE Trans Image Process 22(12):5136–5145

    Article  MathSciNet  Google Scholar 

  36. Luan Q, Song M, Liau CY, Bu J, Liu Z, Sun MT (2014) Video summarization based on nonnegative linear reconstruction. In: IEEE International Conference on Multimedia and Expo, Chengdu, China, pp 1–6

  37. Mahmoud KM, Ismail MA, Ghanem NM (2013) VSCAN: An Enhanced Video Summarization Using Density-Based Spatial Clustering. In: Lecture Notes in Computer Science, vol 8156. Springer, pp 733–742

  38. Mei S, Guan G, Wang Z, Wan S, He M, Feng DD (2015) Video summarization via minimum sparse reconstruction. Pattern Recogn 48(2):522–533

    Article  Google Scholar 

  39. Mohanta PP, Chowdhury S, Roy A, Saha SK, Chanda B (2013) Static summarization of video scenes based on minimal spanning tree. In: Pattern Recognition and Machine Intelligence. Springer, pp 437–444

  40. Mohanta PP, Saha SK, Chanda B (2012) A model-based shot boundary detection technique using frame transition parameters. IEEE Trans Multimed 14(1):223–233

    Article  Google Scholar 

  41. Müller A., Lux M, Böszörmenyi L. (2012) The video summary GWAP: Summarization of videos based on a social game. In: 12th International Conference on Knowledge Management and Knowledge Technologies. ACM, Graz, Austria, pp 1–15

  42. Mundur P, Rao Y, Yesha Y (2006) Keyframe-based video summarization using delaunay clustering. Int J Dig Libr 6:219–232

    Article  Google Scholar 

  43. Ngo CW, Ma YF, Zhang H (2005) Video summarization and scene detection by graph modeling. IEEE Trans Circ Syst Video Technol 15:296–305

    Article  Google Scholar 

  44. OpenCV: Open Source Computer Vision (2016). http://www.opencv.org

  45. Pal G, Rudrapaul D, Acharjee S, Ray R, Chakraborty S, Dey N (2015) Video shot boundary detection: a review. In: Satapathy S.C., Govardhan A., Raju K.S., Mandal J.K. (eds) Emerging ICT for Bridging the Future - Proceedings of the 49th Annual Convention of the Computer Society of India CSI Volume 2, Advances in Intelligent Systems and Computing, vol 338. Springer International Publishing, pp 119–127

  46. Pan B, Wang Z (2013) Recent progress in digital image correlation. In: Application of Imaging Techniques to Mechanics of Materials and Structures, Volume 4, Conference Proceedings of the Society for Experimental Mechanics Series. Springer, New York, pp 317–326

  47. Patel NV, Sethi IK (1997) Video shot detection and characterization for video databases. Pattern Recogn 30(4):583–592

    Article  Google Scholar 

  48. Patel U, Shah P, Panchal P (2013) Shot detection using pixel wise difference with adaptive threshold and color histogram method in compressed and uncompressed video. Int J Comput Appl 64(4):38–44

    Google Scholar 

  49. Pfeiffer S, Lienhart R, Kuhne G, Effelsberg W (1998) The MoCA project – movie content analysis research at the university of mannheim. In: Informatik ’98 : Informatik Zwischen Bild und Sprache, 1. Springer, pp 329–338

  50. Rodriguez MD (2010) CRAM: Compact representation of actions in movies. In: IEEE Computer Vision and Pattern Recognition, pp 3328–3335

  51. Santos A, Pedrini H (2016) Adaptive video shot detection improved by fusion of dissimilarity measures. In: IEEE International Conference on Systems, Man, and Cybernetics, Budapest, Hungary

  52. Santos A, Pedrini H (2016) Adaptive video transition detection based on multiscale structural dissimilarity. In: 12th International Symposium on Visual Computing, vol. Lecture Notes in Computer Science - 10072. Springer-Verlag, NV, USA, pp 181–190

  53. Singh R, Aggarwal N (2015) Novel research in the field of shot boundary detection – a survey. In: Advances in Intelligent Informatics, Advances in Intelligent Systems and Computing, vol 320. Springer International Publishing, pp 457–469

  54. Tian DP (2013) A review on image feature extraction and representation techniques. Int J Multimed Ubiquitous Eng 8(4):385–396

    Google Scholar 

  55. The Open Video Project (2016). http://www.open-video.org

  56. TREC Video Retrieval Evaluation: TRECVID (2016). http://trecvid.nist.gov

  57. Unser M (1986) Sum and difference histograms for texture classification. IEEE Trans Pattern Anal Mach Intell 8(1):118–125

    Article  Google Scholar 

  58. VidSeg (2016). http://www.site.uottawa.ca/laganier/videoseg/

  59. VSUMM (Video SUMMarization) (2016). https://sites.google.com/site/vsummsite

  60. Wang M, Hong R, Li G, Zha ZJ, Yan S, Chua TS (2012) Event driven web video summarization by tag localization and key-shot identification. IEEE Trans Multimed 14(4):975–985

    Article  Google Scholar 

  61. Whitehead A, Bose P, Laganiere R (2004) Feature based cut detection with automatic threshold selection. In: Third International Conference on Image and Video Retrieval. Springer, Dublin, Ireland, pp 410–418

  62. Xu J, Mukherjee L, Li Y, Warner J, Rehg JM, Singh V (2015) Gaze-enabled egocentric video summarization via constrained submodular maximization. In: IEEE Conf Comput Vis Pattern Recogn, pp 2235–2244

  63. Yi H, Pengzhou Z, Yanfeng W (2012) Adaptive threshold based video shot boundary detection framework. In: International Conference on Image Analysis and Signal Processing, pp 1–5

  64. Yuan Z, Lu T, Wu D, Huang Y, Yu H (2011) Video summarization with semantic concept preservation. In: 10th International Conference on Mobile and Ubiquitous Multimedia, NY, USA, pp 109–112

  65. Zhang H, Kankanhalli A, Smoliar SW (1993) Automatic partitioning of full-motion video. Multimed Syst 1(1):10–28

    Article  Google Scholar 

  66. Zhu X, Loy CC, Gong S (2013) Video synopsis by heterogeneous multi-source correlation. In: IEEE International Conference on Computer Vision. IEEE Computer Society, DC, USA, pp 81–88

Download references

Acknowledgments

The authors are thankful to São Paulo Research Foundation (FAPESP grant #2015/12228-1) and Brazilian National Council for Scientific and Technological Development (CNPq grant #305169/2015-7) for their financial support.

Author information

Authors and Affiliations

  1. Institute of Computing, University of Campinas, Campinas, SP, 13083-852, Brazil

    Marcos Vinicius Mussel Cirne & Helio Pedrini

Authors
  1. Marcos Vinicius Mussel Cirne
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Helio Pedrini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Helio Pedrini.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mussel Cirne, M.V., Pedrini, H. VISCOM: A robust video summarization approach using color co-occurrence matrices. Multimed Tools Appl 77, 857–875 (2018). https://doi.org/10.1007/s11042-016-4300-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4300-7

Keywords

Search

Navigation