Abstract
2D CNN are main components for Partial Video Copy Detection (PVCD). 2D CNN features serve for the retrieval and matching of videos. Robustness is a key property of these features. It is a well-known problem in the computer vision field but little investigated for PVCD. The contributions of this paper are twofold: (i) based on a public video dataset, we provide large-scale experiments with 700 B of comparisons of 4.4 M feature vectors. We report conclusions for PVCD consistent with the state-of-the-art. (ii) the regular protocol for performance characterization is misleading for PVCD as it is bounded to the video level. A method for the characterization of key-frames with 2D CNN features is proposed. It is based on a goodness criterion and a time series modelling. It provides a fine categorization of key-frames and allows a deeper characterization of a PVCD problem with 2D CNN features.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Maximum Activations of Convolutions (MAC) and Regional-MAC (R-MAC).
- 2.
- 3.
- 4.
- 5.
Experiments on a GPU RTX 2070 (7 GiB for the features/1 GiB for the programs), dataset fully loaded, matching with a fast vector multiplication on all the cores.
- 6.
The Eq. (1) is defined for \(SC(X,Y) \in [0,1]\) with 2D CNN using a RELU function.
- 7.
No possibility for X to be classified as a false negative (X matched with a negative frame or assigned to another video reference).
- 8.
With \(S(X,X^*) = S(X^*,X)\), the comparison number of m features is \(m\left( \frac{m+1}{2}\right) \).
References
Cheng, H., Wang, P., Qi, C.: Cnn features based unsupervised metric learning for near-duplicate video retrieval. In: Open-Access Repository (2021). arXiv:2105.14566
Cools, A., Belarbi, M., Mahmoudi, S.: A comparative study of reduction methods applied on a convolutional neural network. Electronics 11, 1422 (2022)
Gkelios, S., Sophokleous, A., Plakias, S., Boutalis, Y., Chatzichristofis, S.: Deep convolutional features for image retrieval. Expert Syst. Appl. 177, 114940 (2021)
Han, Z., He, X., Tang, M., Lv, Y.: Video similarity and alignment learning on partial video copy detection. In: ACM International Conference on Multimedia (MM), pp. 4165–4173 (2021)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
He, S., et al.: Transvcl: attention-enhanced video copy localization network with flexible supervision. In: AAAI Conference on Artificial Intelligence (AAAI) (2023)
He, S., et al.: A large-scale comprehensive dataset and copy-overlap aware evaluation protocol for segment-level video copy detection. In: Computer Vision and Pattern Recognition (CVPR), pp. 21086–21095 (2022)
Jiang, C., et al.: Learning segment similarity and alignment in large-scale content based video retrieval. In: ACM International Conference on Multimedia (MM), pp. 1618–1626 (2021)
Jiang, Q., He, Y., Li, G., Lin, J., Li, L., Li, W.: Svd: a large-scale short video dataset for near-duplicate video retrieval. In: International Conference on Computer Vision (ICCV), pp. 5281–5289 (2019)
Jiang, Y., Wang, J.: Partial copy detection in videos: a benchmark and an evaluation of popular methods. IEEE Trans. Big Data 2(1), 32–42 (2016)
Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., Kompatsiaris, I.: Fivr: fine-grained incident video retrieval. IEEE Trans. Multimedia 21(10), 2638–2652 (2019)
Kordopatis-Zilos, G., Papadopoulos, S., Patras, I., Kompatsiaris, Y.: Near-duplicate video retrieval with deep metric learning. In: International Conference on Computer Vision Workshops (ICCV), pp. 347–356 (2017)
Le, V., Delalandre, M., Conte, D.: A large-scale tv dataset for partial video copy detection. In: International Conference on Image Analysis and Processing (ICIAP). Lecture Notes in Computer Science (LNCS), vol. 13233, pp. 388–399. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-06433-3_33
Roy, P., Ghosh, S., Bhattacharya, S., Pal, U.: Effects of degradations on deep neural network architectures. In: Open-Access Repository (2023). arXiv:1807.10108
Tan, W., Guo, H., Liu, R.: A fast partial video copy detection using knn and global feature database. In: Winter Conference on Applications of Computer Vision (WACV), pp. 2191–2199 (2022)
Tolias, G., Sicre, R., Jégou, H.: Particular object retrieval with integral max-pooling of cnn activations. In: International Conference on Learning Representations (ICLR), pp. 1–12 (2016)
Wang, K., Cheng, C., Chen, Y., Song, Y., Lai, S.: Attention-based deep metric learning for near-duplicate video retrieval. In: International Conference on Pattern Recognition (ICPR), pp. 5360–5367 (2021)
Wang, L., Bao, Y., Li, H., Fan, X., Luo, Z.: Compact cnn based video representation for efficient video copy detection. In: International conference on multimedia modeling (MMM), pp. 576–587 (2017)
Zhang, C., Hu, B., Suo, Y., Zou, Z., Ji, Y.: Large-scale video retrieval via deep local convolutional features. Adv. Multimedia 2020, 1687–5680 (2020)
Zhang, X., Gao, J.: Measuring feature importance of convolutional neural networks. IEEE Access 8, 196062–196074 (2020)
Zhang, X., Xie, Y., Luan, X., He, J., Zhang, L., Wu, L.: Video copy detection based on deep cnn features and graph-based sequence matching. Wirel. Pers. Commun. 103(1), 401–416 (2018)
Zhao, G., Zhang, B., Zhang, M., Li, Y., Liu, J., Wen, J.: Star-gnn: spatial-temporal video representation for content-based retrieval. In: International Conference on Multimedia and Expo (ICME), pp. 01–06 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Le, VH., Delalandre, M., Cardot, H. (2023). Performance Characterization of 2D CNN Features for Partial Video Copy Detection. In: Tsapatsoulis, N., et al. Computer Analysis of Images and Patterns. CAIP 2023. Lecture Notes in Computer Science, vol 14184. Springer, Cham. https://doi.org/10.1007/978-3-031-44237-7_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-44237-7_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44236-0
Online ISBN: 978-3-031-44237-7
eBook Packages: Computer ScienceComputer Science (R0)