Skip to main content
Log in

Efficient subsequence matching over large video databases

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

Video similarity matching has broad applications such as copyright detection, news tracking and commercial monitoring, etc. Among these applications, one typical task is to detect the local similarity between two videos without the knowledge on positions and lengths of each matched subclip pair. However, most studies so far on video detection investigate the global similarity between two short clips using a pre-defined distance function. Although there are a few works on video subsequence detection, all these proposals fail to provide an effective query processing mechanism. In this paper, we first generalize the problem of video similarity matching. Then, a novel solution called consistent keyframe matching (CKM) is proposed to solve the problem of subsequence matching based on video segmentation. CKM is designed with two goals: (1) good scalability in terms of the query sequence length and the size of video database and (2) fast video subsequence matching in terms of processing time. Good scalability is achieved by employing a batch query paradigm, where keyframes sharing the same query space are summarized and ordered. As such, the redundancy of data access is eliminated, leading to much faster video query processing. Fast subsequence matching is achieved by comparing the keyframes of different video sequences. Specifically, a keyframe matching graph is first constructed and then divided into matched candidate subgraphs. We have evaluated our proposed approach over a very large real video database. Extensive experiments demonstrate the effectiveness and efficiency of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Berchtold, S., Böhm, C., Kriegel, H.-P.: The pyramid-technique: towards breaking the curse of dimensionality. In: SIGMOD, pp. 142–153 (1998)

  2. Bohm C., Berchtold S., Keim D.: Searching in high-dimensional spaces: index structures for improving the performance of multimedia databases. ACM Comput. Surv. 33(3), 322–373 (2001)

    Article  Google Scholar 

  3. Chen, L., Özsu, M.T., Oria, V.: Robust and fast similarity search for moving object trajectories. In: SIGMOD, pp. 491–502 (2005)

  4. Chen L., Stentiford F.W.M.: Video sequence matching based on temporal ordinal measurement. Pattern Recogn. Lett. 29(13), 1824–1831 (2008)

    Article  Google Scholar 

  5. Chen, Y., Nascimento, M.A., Ooi, B.C., Tung, A.K.H.: Spade: on shape-based pattern detection in streaming time series. In: ICDE, pp. 786–795 (2007)

  6. Cheung S.S., Zakhor A.: Efficient video similarity measurement with video signature. IEEE Trans. Circuits Syst. Video Technol. 13(1), 59–74 (2003)

    Article  Google Scholar 

  7. Hoi, S.C.H., Wong, L.L.S., Lyu, A.: Chinese university of hong kong at trecvid 2006: shot boundary detection and video search. In: TRECVID (2006)

  8. Chiu, C.-Y., Li, C.-H., Wang, H.-A., Chen, C.-S., Chien, L.-F.: A time warping based approach for video copy detection. In: ICPR, pp. 228–231 (2006)

  9. Ciaccia, P., Patella, M., Zezula, P.: M-tree: an efficient access method for similarity search in metric spaces. In: VLDB, pp. 426–435 (1997)

  10. Douze, M., Gaidon, A., Jegou, H., Marszalek, M., Schmid, C.: Inria-lear’s video copy detection system. In: TRECVID (2008)

  11. Faloutsos, C., Ranganathan, M., Manolopoulos, Y.: Fast subsequence matching in time-series databases. In: SIGMOD, pp. 419–429 (1994)

  12. Hua, X.-S., Chen, X., Zhang, H.: Robust video signature based on ordinal measure. In: ICIP, pp. 685–688 (2004)

  13. Huang Z., Shen H.T., Shao J., Zhou X., Cui B.: Bounded coordinate system indexing for real-time video clip search. TOIS 27(3), 17–11733 (2009)

    Article  Google Scholar 

  14. Jagadish H.V., Ooi B.C., Tan K.-L., Yu C., Zhang R.: idistance: an adaptive b+-tree based indexing method for nearest neighbor search. TODS 30(2), 364–397 (2005)

    Article  Google Scholar 

  15. Jiang, Y.-G., Ngo, C.-W., Chang S.-F.: Semantic context transfer across heterogeneous sources for domain adaptive video search. In: ACM Multimedia, pp. 155–164 (2009)

  16. Joly A., Buisson O., Frélicot C.: Content-based copy retrieval using distortion-based probabilistic similarity search. IEEE Trans. Multimed. 9(2), 293–306 (2007)

    Article  Google Scholar 

  17. Joly, A., Frélicot, C., Buisson, O.: Robust content-based video copy identification in a large reference database. In: CIVR, pp. 414–424 (2003)

  18. Kashino K., Kurozumi T., Murase H.: A quick search method for audio and video signals based on histogram pruning. IEEE Trans. Multimed. 5(3), 348–357 (2003)

    Article  Google Scholar 

  19. Ke, Y., Sukthankar, R., Huston, L.: An efficient parts-based near-duplicate and sub-image retrieval system. In: ACM Multimedia, pp. 869–876 (2004)

  20. Kim C., Vasudev B.: Spatiotemporal sequence matching for efficient video copy detection. IEEE Trans. Circuits Syst. Video Technol 15(1), 127–132 (2005)

    Article  Google Scholar 

  21. Kim S.H., Park R.-H.: An efficient algorithm for video sequence matching using the modified hausdorff distance and the directed divergence. IEEE Trans. Circuits Syst. Video Technol. 12(7), 592–596 (2002)

    Article  Google Scholar 

  22. Koprinska I., Carrato S.: Temporal video segmentation: a survey. Signal Process Image Commun. 16(5), 477–500 (2001)

    Article  Google Scholar 

  23. Koudas, N., Ooi, B.C., Shen, H.T., Tung, A.: Ldc: enabling search by partial distance in a hyper-dimensional space. In: ICDE, pp. 6–17 (2004)

  24. Law-To, J., Buisson, O., Gouet-Brunet, V., Boujemaa, N.: Robust voting algorithm based on labels of behavior for video copy detection. In: ACM Multimedia, pp. 835–844 (2006)

  25. Lee, J., Oh, J.-H., Hwang, S.: Scenario based dynamic video abstractions using graph matching. In: ACM Multimedia, pp. 810–819 (2005)

  26. Lee, J., Oh, J.-H., Hwang, S.: Strg-index: spatio-temporal region graph indexing for large video databases. In: SIGMOD, pp. 718–729 (2005)

  27. Lee, S.-L., Chun, S.-J., Kim, D.-H., Lee, J.-H., Chung, C.-W.: Similarity search for multidimensional data sequences. In: ICDE, pp. 599–608 (2000)

  28. Li, C., Zhai, P., Zheng, S.-Q., Prabhakaran, B.: Segmentation and recognition of multi-attribute motion sequences. In: ACM Multimedia, pp. 836–843 (2004)

  29. Liu, Z., Gibbon, D., Zavesky, E., Shahraray, B., Haffner, P.: At&t research at trecvid 2006. In: TRECVID (2006)

  30. Liu, Z., Zavesky, E., Gibbon, D., Shahraray, B., Haffner, P.: At&t research at trecvid 2007. In: TRECVID (2007)

  31. Moon, Y.-S., Whang, K.-Y., Han, W.-S.: General match: a subsequence matching method in time-series databases based on generalized windows. In: SIGMOD, pp. 382–393 (2002)

  32. Moon, Y.-S., Whang, K.-Y., Loh, W.-K.: Duality-based subsequence matching in time-series databases. In: ICDE, pp. 263–272 (2001)

  33. Poullot, S., Buisson, O., Crucianu, M.: Z-grid-based probabilistic retrieval for scaling up content-based copy detection. In: CIVR, pp. 348–355 (2007)

  34. Poullot, S., Crucianu, M., Buisson, O.: Scalable mining of large video databases using copy detection. In: ACM Multimedia, pp. 61–70 (2008)

  35. Robinson, J.T.: The k-d-b-tree: a search structure for large multidimensional dynamic indexes. In: SIGMOD, pp. 10–18 (1981)

  36. Sánchez J.M., Binefa X., Vitrià J.: Shot partitioning based recognition of tv commercials. Multimed. Tools Appl. 18(3), 233–247 (2002)

    Article  Google Scholar 

  37. Shao, J., Huang, Z., Shen, H.T., Zhou, X., Li, Y.: Dynamic batch nearest neighbor search in video retrieval. In: ICDE, pp. 1395–1399 (2007)

  38. Shen, H.T., Ooi, B.C., Zhou, X.: Towards effective indexing for very large video sequence database. In: SIGMOD, pp. 730–741 (2005)

  39. Weber, R., Schek, H., Blott, S.: A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In: VLDB, pp. 194–205 (1998)

  40. Wu, H., Salzberg, B., Sharp, G.C., Jiang, S.B., Shirato, H., Kaeli, D.R.: Subsequence matching on structured time series data. In: SIGMOD, pp. 682–693 (2005)

  41. Wu, X., Hauptmann, A.G., Ngo, C.-W.: Practical elimination of near-duplicates from web video search. In: ACM Multimedia, pp. 218–227 (2007)

  42. Yeh, M.-C., Cheng, K.-T.: Video copy detection by fast sequence matching. In: CIVR, pp. 1–7 (2009)

  43. Yu, C., Ooi, B.C., Tan, K.-L., Jagadish, H.V.: Indexing the distance: an efficient method to knn processing. In: VLDB, pp. 421–430 (2001)

  44. Yuan, J., Tian, Q., Ranganath, S.: Fast and robust search method for short video clips from large video collection. In: ICPR (3), pp. 866–869 (2004)

  45. Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: ACM Multimedia, pp. 815–824 (2006)

  46. Zhang, J., Mamoulis, N., Papadias, D., Tao, Y.: All-nearest-neighbors queries in spatial databases. In: SSDBM, pp. 297–306 (2004)

  47. Zhou, X., Zhou, X., Bouguettaya, A., Taylor, J.A.: A subspace symbolization approach to content-based video search. In: ICDE, pp. 1191–1194 (2009)

  48. Zhou X., Zhou X., Chen L., Bouguettaya A., Xiao N., Taylor J.A.: An efficient near-duplicate video shot detection method using shot-based interest points. Trans. Multimed. 11(5), 879–891 (2009)

    Article  Google Scholar 

  49. Zhou X., Zhou X., Chen L., Shu Y., Bouguettaya A., Taylor J.A.: Adaptive subspace symbolization for content-based video detection. TKDE 2(10), 1372–1387 (2010)

    Google Scholar 

  50. Zobel J., Hoad T.C.: Detection of video sequences using compact signatures. TOIS 24(1), 1–50 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiangmin Zhou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhou, X., Zhou, X., Chen, L. et al. Efficient subsequence matching over large video databases. The VLDB Journal 21, 489–508 (2012). https://doi.org/10.1007/s00778-011-0255-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-011-0255-5

Keywords

Navigation