Skip to main content
SpringerLink
Log in

Online video scene clustering by competitive incremental NMF

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Efficient clustering and categorizing of video are becoming more and more vital in various applications including video summarization, content-based representation and so on. The large volume of video data is the biggest challenge that this task presents, for most the clustering techniques suffer from high dimensional data in terms of both accuracy and efficiency. In addition to this, most video applications require online processing; therefore, clustering should also be done online for such tasks. This paper presents an online video scene clustering/segmentation method that is based on incremental nonnegative matrix factorization (INMF), which has been shown to be a powerful content representation tool for high dimensional data. The proposed algorithm (Comp-INMF) enables online representation of video content and increases efficiency significantly by integrating a competitive learning scheme into INMF. It brings a systematic solution to the issue of rank selection in nonnegative matrix factorization, which is equivalent to specifying the number of clusters. The clustering performance is evaluated by tests on TRECVID video sequences, and a performance comparison to baseline methods including Adaptive Resonance Theory (ART) is provided in order to demonstrate the efficiency and efficacy of the proposed video clustering scheme. Clustering performance reported in terms of recall, precision and F1 measures shows that the labeling accuracy of the algorithm is notable, especially at edit effect regions that constitute a challenging point in video analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Truong, B.T., Venkatesh, S.: Video abstraction: a systematic review and classification. ACM Trans. Multimedia Comput. Commun. Appl. 3(1), 3:1–3:37 (2007)

    Google Scholar 

  2. Xu, L., Luo, B.: Appearance-based video clustering in 2D locality preserving projection subspace. In: Proceeding of CIVR, pp. 356–362 (2007)

  3. Bucak S.S., Gunsel B.: Incremental subspace learning via non-negative matrix factorization. Pattern Recognit. 42, 788–797 (2009)

    Article  MATH  Google Scholar 

  4. Chang, Y., Lee, D.J., Hong, Y., Archibald, J.: Unsupervised video shot detection using clustering ensemble with a color global scale-invariant feature transform descriptor. EURASIP J. Image Video Process. 2008, 9:1–9:10 (2008). doi:10.1155/2008/860743

    Google Scholar 

  5. Damnjanovic, U., Izquierdo, E.: Shot boundary detection using spectral clustering. In: Proceeding of European Signal Processing Conference (2007)

  6. Lo C.-C., Wang S.-J.: A histogram-based moment-preserving clustering algorithm for video segmentation. Pattern Recognit. Lett. 24, 2209–2218 (2003)

    Article  Google Scholar 

  7. Zhou, J., Zhang, X.-P.: Video shot boundary detection using independent component analysis. In: Proceeding of ICASSP, pp. 541–544 (2005)

  8. Gunsel B., Tekalp A.M.: Temporal video segmentation using unsupervised clustering and semantic object tracking. J Electron. Imaging 64(7), 592–604 (1998)

    Article  Google Scholar 

  9. Shahnaz F., Berry M.W., Pauca V.P., Plemmons R.J.: Document clustering using nonnegative matrix factorization. Inf. Process. Manag. 42(2), 373–386 (2006)

    Article  MATH  Google Scholar 

  10. Li, T., Ding, C.: The relationships among various nonnegative matrix factorization methods for clustering. In: Proceeding of ICDM, pp. 362–371 (2006)

  11. Bucak, S.S., Gunsel, B.: Incremental clustering via nonnegative matrix factorization. In: Proceeding of ICPR, pp. 1–4 (2008)

  12. Mairal J., Bach F., Ponce J., Sapiro G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11, 10–60 (2010)

    MathSciNet  Google Scholar 

  13. Yuan Y., Li X., Pang Y., Lu X., Tao D.: Binary sparse nonnegative matrix factorization. IEEE Trans. Circuits Syst. Video Technol. 19(5), 772–777 (2009)

    Article  Google Scholar 

  14. Shen J., Tao D., Li X.: Modality mixture projections for semantic video event detection. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1587–1596 (2008)

    Article  Google Scholar 

  15. Cirakman, O., Gunsel, B., Sengor, N., Gursoy, O.: Key-frame based video fingerprinting by NMF. In: Proceeding of ICIP (2010)

  16. TRECVID: http://www-nlpir.nist.gov/projects/trecvid

  17. Carpenter G.A., Grossberg S.: Adaptive resonance theory. In: Arbib, M.A. (eds) The Handbook of Brain Theory and Neural Networks, 2nd edn, MIT Press, Cambridge (2003)

    Google Scholar 

  18. Lee D.D., Seung H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  19. Lin C.-J.: Projected gradient methods for non-negative matrix factorization. Neural Comput. 19, 2756–2779 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  20. Li, T., Ding, C., Jordan, M.I.: Solving consensus and semi-supervised clustering problems using nonnegative matrix factorization. In: Proceeding of ICDM, pp. 577–582 (2007)

  21. Ding C., Li T., Jordan M.I.: Convex and semi-nonnegative matrix factorizations. IEEE Trans. Pattern Anal. Mach. Intell 32(1), 45–55 (2010). doi:10.1109/TPAMI.2008.277

    Article  Google Scholar 

  22. Bucak, S.S., Gunsel, B., Gursoy, O.: Incremental non-negative matrix factorization for dynamic background modeling. In: Proceeding of ICEIS 8th International Workshop on Pattern Recognition in Information Systems (2007)

  23. Bucak, S.S., Gunsel, B., Video content representation by incremental non-negative matrix factorization. In: Proceeding of ICIP, pp. 113–116 (2007)

  24. Jain A.K., Murty N.M., Flynn P.J.: Data clustering: a review. ACM Comput. Surv. 31(3), 264–323 (1999)

    Article  Google Scholar 

  25. Otsu N.: A threshold selection method from gray-level histograms. IEEE Trans Syst. Man. Cyber 9, 62–66 (1979)

    Article  Google Scholar 

  26. Lee D.D., Seung H.S.: Algorithms for non-negative matrix factorization. Proc. Adv. Neural Inf. Process 13, 556–562 (2001)

    Google Scholar 

  27. Weitzenfeld A., Arbib M., Alexander A.: The Neural Simulation Language a System for Brain Modeling. MIT Press, Cambridge (2002)

    Google Scholar 

  28. Patrikainen A., Meila M.: Comparing subspace clusterings. IEEE Trans. Knowl. Data Eng 18(7), 902–916 (2006)

    Article  Google Scholar 

  29. Yeo B., Liu B.: Rapid scene analysis on compressed videos. IEEE Trans. Circuits Syst. Video Technol. 5, 533–544 (1995)

    Article  Google Scholar 

  30. Kutluk, S., Gunsel, B.: ITU MSPR TRECVID 2010 video copy detection system. In: Proceeding of TRECVID 2010, pp. 224–231 (2010)

  31. Friedman M., Kandel A.: Introduction to Pattern Recognition Statistical, Structural, Neural and Fuzzy Logic Approaches. World Scientific, Singapore (1999)

    Book  Google Scholar 

  32. Wang, F., Li, P.: Efficient non-negative matrix factorization with random projections. In: Proceeding of The 10th SIAM International Conference on Data Mining (2010)

  33. Wang, F., Li, T., Zhang, C.: Semi-supervised Clustering via matrix factorization. In: Proceeding of The 8th SIAM Conference on Data Mining, pp. 1–12 (2008)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, 48824, USA

    Serhat Selcuk Bucak

  2. Multimedia Signal Processing and Pattern Recognition Laboratory, Department of Electronics and Communications Engineering, Istanbul Technical University, 34469, Maslak Istanbul, Turkey

    Bilge Gunsel

Authors
  1. Serhat Selcuk Bucak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bilge Gunsel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bilge Gunsel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bucak, S.S., Gunsel, B. Online video scene clustering by competitive incremental NMF. SIViP 7, 723–739 (2013). https://doi.org/10.1007/s11760-011-0264-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-011-0264-2

Keywords

Search

Navigation