Skip to main content

Performance Characterization of 2D CNN Features for Partial Video Copy Detection

  • Conference paper
  • First Online:
Computer Analysis of Images and Patterns (CAIP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14184))

Included in the following conference series:

  • 571 Accesses

Abstract

2D CNN are main components for Partial Video Copy Detection (PVCD). 2D CNN features serve for the retrieval and matching of videos. Robustness is a key property of these features. It is a well-known problem in the computer vision field but little investigated for PVCD. The contributions of this paper are twofold: (i) based on a public video dataset, we provide large-scale experiments with 700 B of comparisons of 4.4 M feature vectors. We report conclusions for PVCD consistent with the state-of-the-art. (ii) the regular protocol for performance characterization is misleading for PVCD as it is bounded to the video level. A method for the characterization of key-frames with 2D CNN features is proposed. It is based on a goodness criterion and a time series modelling. It provides a fine categorization of key-frames and allows a deeper characterization of a PVCD problem with 2D CNN features.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Maximum Activations of Convolutions (MAC) and Regional-MAC (R-MAC).

  2. 2.

    http://mathieu.delalandre.free.fr/projects/stvd/pvcd/.

  3. 3.

    A positive pair \((v_i, v_j)\) is a combination of two partial video copies \(v_i\) and \(v_j\) [7, 10].

  4. 4.

    Detailed at http://mathieu.delalandre.free.fr/publications/CAIP2023.pdf.

  5. 5.

    Experiments on a GPU RTX 2070 (7 GiB for the features/1 GiB for the programs), dataset fully loaded, matching with a fast vector multiplication on all the cores.

  6. 6.

    The Eq. (1) is defined for \(SC(X,Y) \in [0,1]\) with 2D CNN using a RELU function.

  7. 7.

    No possibility for X to be classified as a false negative (X matched with a negative frame or assigned to another video reference).

  8. 8.

    With \(S(X,X^*) = S(X^*,X)\), the comparison number of m features is \(m\left( \frac{m+1}{2}\right) \).

References

  1. Cheng, H., Wang, P., Qi, C.: Cnn features based unsupervised metric learning for near-duplicate video retrieval. In: Open-Access Repository (2021). arXiv:2105.14566

  2. Cools, A., Belarbi, M., Mahmoudi, S.: A comparative study of reduction methods applied on a convolutional neural network. Electronics 11, 1422 (2022)

    Article  Google Scholar 

  3. Gkelios, S., Sophokleous, A., Plakias, S., Boutalis, Y., Chatzichristofis, S.: Deep convolutional features for image retrieval. Expert Syst. Appl. 177, 114940 (2021)

    Article  Google Scholar 

  4. Han, Z., He, X., Tang, M., Lv, Y.: Video similarity and alignment learning on partial video copy detection. In: ACM International Conference on Multimedia (MM), pp. 4165–4173 (2021)

    Google Scholar 

  5. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

    Google Scholar 

  6. He, S., et al.: Transvcl: attention-enhanced video copy localization network with flexible supervision. In: AAAI Conference on Artificial Intelligence (AAAI) (2023)

    Google Scholar 

  7. He, S., et al.: A large-scale comprehensive dataset and copy-overlap aware evaluation protocol for segment-level video copy detection. In: Computer Vision and Pattern Recognition (CVPR), pp. 21086–21095 (2022)

    Google Scholar 

  8. Jiang, C., et al.: Learning segment similarity and alignment in large-scale content based video retrieval. In: ACM International Conference on Multimedia (MM), pp. 1618–1626 (2021)

    Google Scholar 

  9. Jiang, Q., He, Y., Li, G., Lin, J., Li, L., Li, W.: Svd: a large-scale short video dataset for near-duplicate video retrieval. In: International Conference on Computer Vision (ICCV), pp. 5281–5289 (2019)

    Google Scholar 

  10. Jiang, Y., Wang, J.: Partial copy detection in videos: a benchmark and an evaluation of popular methods. IEEE Trans. Big Data 2(1), 32–42 (2016)

    Article  Google Scholar 

  11. Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., Kompatsiaris, I.: Fivr: fine-grained incident video retrieval. IEEE Trans. Multimedia 21(10), 2638–2652 (2019)

    Article  Google Scholar 

  12. Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., Kompatsiaris, Y.: Near-duplicate video retrieval with deep metric learning. In: International Conference on Computer Vision Workshops (ICCV), pp. 347–356 (2017)

    Google Scholar 

  13. Le, V., Delalandre, M., Conte, D.: A large-scale tv dataset for partial video copy detection. In: International Conference on Image Analysis and Processing (ICIAP). Lecture Notes in Computer Science (LNCS), vol. 13233, pp. 388–399. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-06433-3_33

  14. Roy, P., Ghosh, S., Bhattacharya, S., Pal, U.: Effects of degradations on deep neural network architectures. In: Open-Access Repository (2023). arXiv:1807.10108

  15. Tan, W., Guo, H., Liu, R.: A fast partial video copy detection using knn and global feature database. In: Winter Conference on Applications of Computer Vision (WACV), pp. 2191–2199 (2022)

    Google Scholar 

  16. Tolias, G., Sicre, R., Jégou, H.: Particular object retrieval with integral max-pooling of cnn activations. In: International Conference on Learning Representations (ICLR), pp. 1–12 (2016)

    Google Scholar 

  17. Wang, K., Cheng, C., Chen, Y., Song, Y., Lai, S.: Attention-based deep metric learning for near-duplicate video retrieval. In: International Conference on Pattern Recognition (ICPR), pp. 5360–5367 (2021)

    Google Scholar 

  18. Wang, L., Bao, Y., Li, H., Fan, X., Luo, Z.: Compact cnn based video representation for efficient video copy detection. In: International conference on multimedia modeling (MMM), pp. 576–587 (2017)

    Google Scholar 

  19. Zhang, C., Hu, B., Suo, Y., Zou, Z., Ji, Y.: Large-scale video retrieval via deep local convolutional features. Adv. Multimedia 2020, 1687–5680 (2020)

    Article  Google Scholar 

  20. Zhang, X., Gao, J.: Measuring feature importance of convolutional neural networks. IEEE Access 8, 196062–196074 (2020)

    Article  Google Scholar 

  21. Zhang, X., Xie, Y., Luan, X., He, J., Zhang, L., Wu, L.: Video copy detection based on deep cnn features and graph-based sequence matching. Wirel. Pers. Commun. 103(1), 401–416 (2018)

    Article  Google Scholar 

  22. Zhao, G., Zhang, B., Zhang, M., Li, Y., Liu, J., Wen, J.: Star-gnn: spatial-temporal video representation for content-based retrieval. In: International Conference on Multimedia and Expo (ICME), pp. 01–06 (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Van-Hao Le .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Le, VH., Delalandre, M., Cardot, H. (2023). Performance Characterization of 2D CNN Features for Partial Video Copy Detection. In: Tsapatsoulis, N., et al. Computer Analysis of Images and Patterns. CAIP 2023. Lecture Notes in Computer Science, vol 14184. Springer, Cham. https://doi.org/10.1007/978-3-031-44237-7_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44237-7_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44236-0

  • Online ISBN: 978-3-031-44237-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics