Skip to main content
SpringerLink
Log in

Concept-based near-duplicate video clip detection for novelty re-ranking of web video search results

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

State-of-the-art near-duplicate video clip (NDVC) detection for novelty re-ranking uses non-semantic low-level features (color/texture) to detect and eliminate “content-based NDVC” and increases content level novelty in the top results. However, humans may perceive a video as near duplicate from a semantic perspective as well. In this paper, we propose concept-based near-duplicate video clip (CBNDVC) detection technique for novelty re-ranking. We identify “semantic NDVC”, making use of the semantic features (events/concepts) and re-rank the top results to increase the content as well as semantic novelty. Videos are represented as a multivariate time series of confidence values of relevant concepts and thereafter discovery of CBNDVC clusters is achieved by conceptual clustering. Obtained results show higher precision and recall from the user’s perspective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Adjeroh, D.A., Lee, M.C., King, I.: A distance measure for video sequences. Comput. Vis. Image Underst. 75, 25–45 (1999)

    Google Scholar 

  2. Alexis Joly, O.B., Frlicot, C.: Content-based copy retrieval using distortion-based probabilistic similarity search. IEEE Trans. Multimed. 9, 293–306 (2007)

    Google Scholar 

  3. Aly, R., Hiemstra, D.: Concept detectors: how good is good enough? In: ACM International Conference on Multimedia, pp. 233–242 (2009)

  4. Basharat, A., Zhai, Y., Shah, M.: Content based video matching using spatiotemporal volumes. Comput. Vis. Image Underst. 110, 360–377 (2008)

    Google Scholar 

  5. Cherubini, M., de Oliveira, R., Oliver, N.: Understanding near-duplicate videos: a user-centric approach. In: ACM International Conference on Multimedia, pp. 35–44 (2009)

  6. Fanizzi, N., d’Amato, C., Esposito, F.: Conceptual clustering its application to concept drift and novelty detection. In: European Semantic Web Conference, pp. 318–332. Springer, Berlin (2008)

  7. Fisher, D.H.: Knowledge acquisition via incremental conceptual clustering. Mach. Learn. 2, 139–172 (1987)

    Google Scholar 

  8. Han, J.: Data Mining: Concepts and Techniques. Morgan Kaufmann, San Francisco (2005)

  9. Hauptmann, A., Yan, R., Lin, W.H.: How many high-level concepts will fill the semantic gap in news video retrieval? In: ACM International Conference on Image and Video Retrieval, pp. 627–634 (2007)

  10. Huang, Z., Hu, B., Cheng, H., Shen, H.T., Hongyan, L., Zhou, X.: Mining near-duplicate graph for cluster-based reranking of web video search results. In: ACM Transactions on Information Systems (2010)

  11. Keogh, E., Ratanamahatana, C.A.: Exact indexing of dynamic time warping. Knowl. Inf. Syst. 7, 358–386 (2005)

    Google Scholar 

  12. Law-To, J., Buisson, O., Gouet-Brunet, V., Boujemaa, N.: Robust voting algorithm based on labels of behavior for video copy detection. In: ACM International Conference on Multimedia, pp. 835–844 (2006)

  13. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    Google Scholar 

  14. Min, H.S., Choi, J., De Neve, W., Ro, Y.M.: Near-duplicate video detection using temporal patterns of semantic concepts. In: IEEE International Symposium on Multimedia, pp. 65–71 (2009)

  15. Rand, W.M.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66, 846–850 (1971)

    Google Scholar 

  16. Sen Ching, S.C., Zakhor, A.: Fast similarity search and clustering of video sequences on the world-wide-web. IEEE Trans. Multimed. 7, 524–537 (2005)

    Google Scholar 

  17. Shen, H.T., Zhou, X., Huang, Z., Shao, J., Zhou, X.: Uqlips: a real-time near-duplicate video clip detection system. In: International Conference on Very Large Databases, pp. 1374–1377 (2007)

  18. Snoek, C.G.M., Worring, M.: Concept-based video retrieval foundations and trends in information retrieval, vol. 2, pp. 215–322. Now Publishers Inc., Hanover (2009)

  19. Snoek, C.G.M., Worring, M., van Gemert, J.C., Geusebroek, J.M., Smeulders, A.W.M.: The challenge problem for automated detection of 101 semantic concepts in multimedia. In: ACM International Conference on Multimedia, pp. 421–430 (2006)

  20. Stricker, M., Orengo, M.: Similarity of Color Images. vol. 2185, pp. 381–392 (1995)

  21. Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., Hua, X-S.: Bayesian video search reranking. In: ACM International Conference on Multimedia, pp. 131–140. ACM, New York (2008)

  22. Wei, W.: Time Series Analysis Univariate Multivariate Methods. Pearson Addison Wesley, Boston (2006)

  23. Wu, X., Hauptmann, A.G., Ngo, C.W.: Practical elimination of near-duplicates from web video search. In: ACM International Conference on Multimedia, pp. 218–227 (2007)

  24. Wu, X., Ngo, C.W., Hauptmann, A.G.: http://vireo.cs.cityu.edu.hk/webvideo. Carnegie Mellon University (Informedia group) and City University of Hong Kong (VIREO group)

  25. Wu, X., Ngo, C.W., Hauptmann, A.G., Tan, H.K.: Real-time near-duplicate elimination for web video search with content and context. Trans. Multimed. 11, 196–207 (2009)

    Google Scholar 

  26. Wu, X., Zhao, W.L., Ngo, C.W., Hauptmann, A.G.: Near-duplicate web video detection. In: Hua, X.S., Worring, M., Chua, T.S. (eds.) Internet Multimedia Search and Mining, Bentham Science Publishers (2010)

  27. Yanagawa, A., Chang, S.F., Kennedy, L., Hsu, W.: Columbia university’s baseline detectors for 374 lscom semantic visual concepts. Columbia University ADVENT Technical Report, pp. 846–850 (2007)

  28. Yuan, J., Duan, L.Y., Tian, Q., Xu, C.: Fast and robust short video clip search using an index structure. In: ACM SIGMM International Workshop on Multimedia Information Retrieval, pp. 61–68 (2004)

  29. Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., Hua, X.S.: Bayesian video search reranking. In: ACM International Conference on Multimedia, pp. 131–140 (2008)

  30. Tan, H.-K., Ngo C.-W.,Wu, X.: Modeling video hyperlinks with hypergraph for Web video reranking. In: ACM International Conference on Multimedia, pp. 659–662 (2008)

  31. Liu, Y., Mei, T., Hua, X.-S., Tang, J., Wu, X., Li, S.: Learning to video search rerank via pseudo preference feedback. In: International Conference on Multimedia and Expo, pp. 297–300 (2008)

  32. Hsu, W., Kennedy, L., Chang, S.F.: Video search reranking through random walk over document-level context graph. In: ACM International Conference on Multimedia, pp. 971–980 (2007)

  33. McKusick, K.,Thompson, K.: COBWEB/3: a portable implementation. Technical Report No. FIA-90-6-18-2, pp. 139–172 (1990)

  34. Reich, Y., Fenves, S.J.: The formation and use of abstract concepts in design. In: Concept formation knowledge and experience in unsupervised learning, pp. 323–353. Morgan Kaufmann Publishers Inc., San Francisco (1991)

  35. Cheeseman, P., Stutz, J.: Bayesian classification (AutoClass): theory and results. In: Advances in knowledge discovery and data mining, pp. 53–180. American Association for Artificial Intelligence, Menlo Park (1996)

  36. Talavera, L., Bejar, J.J.: Generality-based conceptual clustering with probabilistic concepts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 196–206 (2001)

    Google Scholar 

  37. Chu, W.W., Chiang, K., Hsu, C., Yau, H.: An error-based conceptual clustering method for providing approximate query answers. vol. 39, pp. 216–230. Commun. ACM (1996)

  38. Wei, X., Ngo, C., Jiang, Y.: Selection of concept detectors for video search by ontology-enriched semantic spaces. IEEE Trans. Multimed. 10, 1085–1096 (2008)

    Google Scholar 

  39. Snoek, C.G.M., Huurnink, B., Hollink, L., Rijke, M., Schreiber, G., Worring, M.: Adding semantics to detectors for video retrieval. IEEE Trans. Multimed. 9, 975–986 (2007)

    Google Scholar 

  40. Bhatt, C., Kankanhalli, M.: Multimedia data mining: state of the art and challenges. Multimed. Tools Appl. 51, 35–76 (2011)

    Google Scholar 

  41. Bhatt, C., Kankanhalli, M.: Probabilistic temporal multimedia data mining. In: ACM Transactions on Intelligent Systems and Technology, vol. 2, pp. 17:1–17:19. ACM, New York (2011)

  42. Bather, J.: An Introduction to Dynamic Programming and Sequential Decisions. Wiley-Interscience Series in Systems Optimisation, Wiley, Hoboken (2000)

Download references

Author information

Authors and Affiliations

  1. School of Computing, National University of Singapore, Singapore, 117417, Singapore

    Chidansh A. Bhatt & Mohan S. Kankanhalli

  2. Applied Computer Science, University of Winnipeg, Winnipeg, MB, R3B 2E9, Canada

    Pradeep K. Atrey

Authors
  1. Chidansh A. Bhatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pradeep K. Atrey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohan S. Kankanhalli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chidansh A. Bhatt.

Additional information

Communicated by Thomas Plagemann.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bhatt, C.A., Atrey, P.K. & Kankanhalli, M.S. Concept-based near-duplicate video clip detection for novelty re-ranking of web video search results. Multimedia Systems 18, 337–358 (2012). https://doi.org/10.1007/s00530-011-0253-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-011-0253-x

Keywords

Search

Navigation