Skip to main content
SpringerLink
Log in

Symmetry Structured Analysis Sparse Coding for Key Frame Extraction

  • Conference paper
  • First Online:
Machine Learning for Cyber Security (ML4CS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13655))

Included in the following conference series:

  • 1755 Accesses

Abstract

The efficiency of sparse coding based key frame extraction algorithm is influenced by various sparse regularization and optimization strategies. However, sparse coding with an analytical model for key frame extraction is still a challenging task. In this paper, we present a new analysis sparse coding algorithm for key frame extraction using minimax concave penalty (MCP). Analysis sparse coding has low computation complexity compared to the common synthesis model. Furthermore, analysis sparse coding can automatically lead to symmetry structured for key frame extraction. In this context, the MCP sparse regularization is non-convex that can promote the sparsity of solutions. Unlike conventional non-convex sparse regularization in formulating a non-convex sparse coding cost function, MCP can maintain the convexity that can be used to solve the optimization problem for obtaining the global minimum. The proposed key frame extraction algorithm leads into the following: 1) provides more compressed key frames, 2) decreases the computational complexity and 3) accelerates the process tasks. Our results demonstrate the effectiveness of the proposed symmetry structured with analysis sparse coding algorithm that is validated with both simulations and a number of challenging real-world scenarios, outperforming the state-of-the-art techniques.

This work was supported in part by the National Natural Science Foundation of China (61903090), Guangxi Natural Science Foundation (2022GXNSFBA035644, 2021GXNSFBA220039), Guangxi Science and Technology Major Project (AA22068057), and the Foreign Young Talent Program (QN2021033002L).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.vision.ee.ethz.ch/~gyglim/vsum/.

References

  1. Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006). https://doi.org/10.1109/TSP.2006.881199

    Article  MATH  Google Scholar 

  2. Antani, S., Kasturi, R., Jain, R.: A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video. Pattern Recogn. 35(4), 945–965 (2002)

    Article  MATH  Google Scholar 

  3. Bao, C., Hui, J., Quan, Y., Shen, Z.: Dictionary learning for sparse coding: algorithms and convergence analysis. IEEE Trans. Pattern Anal. Mach. Intell. 38(7), 1356–1369 (2015)

    Article  Google Scholar 

  4. Dang, C., Radha, H.: RPCA-KFE: key frame extraction for video using robust principal component analysis. IEEE Trans. Image Process. 24(11), 3742–3753 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  5. Ejaz, N., Mehmood, I., Wook Baik, S.: Efficient visual attention based framework for extracting key frames from videos. Sig. Process. Image Commun. 28(1), 34–44 (2013). https://doi.org/10.1016/j.image.2012.10.002. https://www.sciencedirect.com/science/article/pii/S0923596512001828

  6. Elhamifar, E., Sapiro, G., Vidal, R.: See all by looking at a few: sparse modeling for finding representative objects. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1600–1607. IEEE (2012)

    Google Scholar 

  7. Gu, X., Lu, L., Qiu, S., Zou, Q., Yang, Z.: Sentiment key frame extraction in user-generated micro-videos via low-rank and sparse representation. Neurocomputing 410, 441–453 (2020)

    Article  Google Scholar 

  8. Gygli, M., Grabner, H., Riemenschneider, H., Van Gool, L.: Creating summaries from user videos. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 505–520. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10584-0_33

    Chapter  Google Scholar 

  9. Nandini, H.M., Chethana, H.K., Rashmib, B.S.: Shot based keyframe extraction using edge-LBP approach. J. King Saud Univ. - Comput. Inf. Sci. 34(7), 4537–4545 (2020)

    Google Scholar 

  10. Hu, W., et al.: Multi-perspective cost-sensitive context-aware multi-instance sparse coding and its application to sensitive video recognition. IEEE Trans. Multimedia 18(1), 76–89 (2015)

    Article  Google Scholar 

  11. Huang, C., Wang, H.: Novel key-frames selection framework for comprehensive video summarization. IEEE Trans. Circ. Syst. Video Technol. 30(2), 577-589 (2019)

    Google Scholar 

  12. Jeong, D., Yoo, H.J., Cho, N.I.: A static video summarization method based on the sparse coding of features and representativeness of frames. EURASIP J. Image Video Process. 2017(1), 1–14 (2016). https://doi.org/10.1186/s13640-016-0122-9

    Article  Google Scholar 

  13. Ji, Z., Ma, Y., Pang, Y., Li, X.: Query-aware sparse coding for multi-video summarization. Information Sci. 478, 152–166 (2017)

    Google Scholar 

  14. Ju, S.X., Black, M.J., Minneman, S., Kimber, D.: Analysis of gesture and action in technical talks for video indexing. In: IEEE Computer Society Conference on Computer Vision & Pattern Recognition (2002)

    Google Scholar 

  15. Kumar, M., Loui, A.C.: Key frame extraction from consumer videos using sparse representation. In: IEEE International Conference on Image Processing (2011)

    Google Scholar 

  16. Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In Adv. NIPS 19, 801–808 (2007)

    Google Scholar 

  17. Li, H., Chen, G.: Segment-based stereo matching using graph cuts. In: Computer Vision and Pattern Recognition (2004)

    Google Scholar 

  18. Li, N., Sun, B., Yu, J.: A weighted sparse coding framework for saliency detection. In: Computer Vision & Pattern Recognition (2015)

    Google Scholar 

  19. Li, Y., Kanemura, A., Asoh, H., Miyanishi, T., Kawanabe, M.: Extracting key frames from first-person videos in the common space of multiple sensors. In: IEEE International Conference on Image Processing, ICIP, pp. 3993–3997 (2017)

    Google Scholar 

  20. Li, Y., Kanemura, A., Asoh, H., Miyanishi, T., Kawanabe, M.: Key frame extraction from first-person video with multi-sensor integration. In: IEEE International Conference on Multimedia and Expo, ICME, pp. 1303–1308 (2017)

    Google Scholar 

  21. Li, Y., Tan, B., Kanemura, A., Ding, S., Chen, W.: Analysis sparse representation for nonnegative signals based on determinant measure by DC programming. Complexity 2018, 1–12 (2018)

    MATH  Google Scholar 

  22. Li, Y., Shi, J., Lin, D.: Low-latency video semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 5997–6005 (2018)

    Google Scholar 

  23. Mademlis, I., Tefas, A., Pitas, I.: Regularized SVD-based video frame saliency for unsupervised activity video summarization. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2691–2695. IEEE (2018)

    Google Scholar 

  24. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE International Conference on Computer Vision (2002)

    Google Scholar 

  25. Meng, Y., Dai, D., Shen, L., Gool, L.V.: Latent dictionary learning for sparse representation based classification. In: IEEE Conference on Computer Vision & Pattern Recognition (2014)

    Google Scholar 

  26. Money, A.G., Agius, H.: Video summarisation: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121–143 (2008)

    Article  Google Scholar 

  27. Nam, S., Davies, M.E., Elad, M., Gribonval, R.: The cosparse analysis model and algorithms. Appl. Comput. Harmonic Anal. 34(1), 30–56 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  28. Nasreen, A., Shobha, G.: Key frame extraction from videos-a survey. Int. J. Comput. Sci. Commun. Netw. 3(3), 194 (2013)

    Google Scholar 

  29. Otani, M., Nakashima, Y., Rahtu, E., Heikkilä, J., Yokoya, N.: Video summarization using deep semantic features. In: Asian Conference on Computer Vision, pp. 361–377 (2016)

    Google Scholar 

  30. Phan, S., et al.: Multimedia event detection using segment-based approach for motion feature. J. Sign. Process. Syst. 74(1), 19–31 (2013). https://doi.org/10.1007/s11265-013-0825-4

  31. Selesnick, I.: Sparse regularization via convex analysis. IEEE Trans. Signal Process. 65(17), 4481–4494 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Computer Science (2014)

    Google Scholar 

  33. Simsek, M., Polat, E.: Performance evaluation of pan-sharpening and dictionary learning methods for sparse representation of hyperspectral super-resolution. SIViP 15(6), 1099–1106 (2021). https://doi.org/10.1007/s11760-020-01836-8

    Article  Google Scholar 

  34. Tan, B., Li, Y., Zhao, H., Li, X., Ding, S.: A novel dictionary learning method for sparse representation with nonconvex regularizations. Neurocomputing 417, 128–141 (2020)

    Google Scholar 

  35. Tan, B., Li, Y., Ding, S., Paik, I., Kanemura, A.: DC programming for solving a sparse modeling problem of video key frame extraction. Digit. Sign. Process. 83, 214–222 (2018)

    Article  Google Scholar 

  36. Wang, S., Chen, X., Dai, W., Selesnick, I.W., Cai, G., Benjamin, C.: Vector minimax concave penalty for sparse representation. Digit. Sign. Process. 83, 165–179 (2018)

    Article  MathSciNet  Google Scholar 

  37. Yaghoobi, M., Nam, S., Gribonval, R., Davies, M.E.: Constrained overcomplete analysis operator learning for cosparse signal modelling. IEEE Trans. Signal Process. 61(9), 2341–2355 (2013)

    Article  Google Scholar 

  38. Zhang, C.H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38(2), 894–942 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  39. Zhao, B., Li, F., Xing, E.P.: Online detection of unusual events in videos via dynamic sparse coding. In: Computer Vision & Pattern Recognition (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Guilin University of Electronic Technology, Guilin, 541004, China

    Yujie Li, Benying Tan, Shuxue Ding & Ahmad Chaddad

  2. Ecole de Technology Superieure, Montreal, Canada

    Christian Desrosiers

Authors
  1. Yujie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benying Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuxue Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christian Desrosiers
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ahmad Chaddad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Benying Tan or Ahmad Chaddad .

Editor information

Editors and Affiliations

  1. School of Computing and Informatics, University of Louisiana at Lafayette, Lafayette, IN, USA

    Yuan Xu

  2. Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, China

    Hongyang Yan

  3. Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, China

    Huang Teng

  4. Guangdong Polytechnic Normal University, Guangzhou, China

    Jun Cai

  5. Institute of Artificial Intelligence and Blockchain, Guangzhou University, Guangzhou, China

    Jin Li

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, Y., Tan, B., Ding, S., Desrosiers, C., Chaddad, A. (2023). Symmetry Structured Analysis Sparse Coding for Key Frame Extraction. In: Xu, Y., Yan, H., Teng, H., Cai, J., Li, J. (eds) Machine Learning for Cyber Security. ML4CS 2022. Lecture Notes in Computer Science, vol 13655. Springer, Cham. https://doi.org/10.1007/978-3-031-20096-0_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20096-0_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20095-3

  • Online ISBN: 978-3-031-20096-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation