skip to main content
10.1145/3007669.3007725acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicimcsConference Proceedingsconference-collections
research-article

Fast Video Deduplication via Locality Sensitive Hashing with Similarity Ranking

Published: 19 August 2016 Publication History

Abstract

The explosive growth of the massive video data brings great challenges to the fast video deduplication. There is encouraging progress of the deduplication techniques in the past few years, especially with the help of the binary hashing methods. However, till now there is rare work that studies the generic hash based framework and the efficient similarity ranking strategy for video deduplication. This paper proposes a flexible and fast video deduplication framework based on hash codes, which supports the hash table indexing using any existing hashing algorithm, and ranks the candidate videos by exploring the similarities among the key frames over multiple tables. Our experiments on the popular large-scale dataset demonstrate that the proposed framework can achieve satisfying performance in the task of video deduplication.

References

[1]
J. L. Bentley. Multidimensional binary search trees used for associative searching. Commun. ACM, 18:509--517, 1975.
[2]
Y. Cai, L. Yang, W. Ping, F. Wang, T. Mei, X.-S. Hua, and S. Li. Million-scale near-duplicate video retrieval system. In ACM MM, 2011.
[3]
M. S. Charikar. Similarity estimation techniques from rounding algorithms. In ACM Symposium on Theory of Computing, pages 380--388, 2002.
[4]
J. Cheng, C. Leng, J. Wu, H. Cui, and H. Lu. Fast and accurate image matching with cascade hashing for 3d reconstruction. In IEEE CVPR, pages 4321--4328, 2014.
[5]
C. Deng, H. Deng, X. Liu, and Y. Yuan. Adaptive multi-bit quantization for hashing. Neurocomputing, 151, Part 1(0):319--326, 2015.
[6]
C. Deng, X. Liu, Y. Mu, and J. Li. Large-scale multi-task image labeling with adaptive relevance discovery and feature hashing. Signal Process., 112(C):137--145, July 2015.
[7]
Y. Gong, S. Kumar, H. A. Rowley, and S. Lazebnik. Learning binary codes for high-dimensional data using bilinear projections. In IEEE CVPR, pages 484--491, 2013.
[8]
Y. Gong and S. Lazebnik. Iterative quantization: A procrustean approach to learning binary codes. In IEEE CVPR, pages 817--824, 2011.
[9]
J. He, J. Feng, X. Liu, T. Cheng, T.-H. Lin, H. Chung, and S.-F. Chang. Mobile product search with bag of hash bits and boundary reranking. In IEEE CVPR, pages 3005--3012, 2012.
[10]
J. He, S. Kumar, and S.-F. Chang. On the difficulty of nearest neighbor search. In International Conference on Machine Learning, pages 1--8, 2012.
[11]
K. He, F. Wen, and J. Sun. K-means hashing: An affinity-preserving quantization method for learning binary compact codes. In IEEE CVPR, pages 2938--2945, 2013.
[12]
J.-P. Heo, Y. Lee, J. He, S.-F. Chang, and S.-E. Yoon. Spherical hashing. In IEEE CVPR, pages 2957--2964, 2012.
[13]
B. Kulis and K. Grauman. Kernelized locality-sensitive hashing for scalable image search. In IEEE ICCV, pages 2130--2137, 2009.
[14]
Z. Li, X. Liu, J. Wu, and H. Su. Adaptive binary quantization for fast nearest neighbor search. In ECAI, pages 64--72, 2016.
[15]
W. Liu, C. Mu, S. Kumar, and S.-F. Chang. Discrete graph hashing. In NIPS, pages 3419--3427, 2014.
[16]
W. Liu, J. Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang. Supervised hashing with kernels. In IEEE CVPR, pages 2074--2081, 2012.
[17]
X. Liu, J. He, C. Deng, and B. Lang. Collaborative hashing. In IEEE CVPR, pages 2147--2154, 2014.
[18]
X. Liu, J. He, and B. Lang. Reciprocal hash tables for nearest neighbor search. In AAAI, pages 626--632, 2013.
[19]
X. Liu, J. He, B. Lang, and S.-F. Chang. Hash bit selection: a unified solution for selection problems in hashing. In IEEE CVPR, pages 1570--1577, 2013.
[20]
X. Liu, L. Huang, C. Deng, J. Lu, and B. Lang. Multi-view complementary hash tables for nearest neighbor search. In IEEE ICCV, pages 1107--1115, 2015.
[21]
X. Liu, Y. Mu, B. Lang, and S.-F. Chang. Compact hashing for mixed image-keyword query over multi-label images. In ACM ICMR, page 18, 2012.
[22]
Y. Mu, X. Chen, X. Liu, T.-S. Chua, and S. Yan. Multimedia semantics-aware query-adaptive hashing with bits reconfigurability. International Journal of Multimedia Information Retrieval, pages 1--12, 2012.
[23]
Y. Mu and S. Yan. Non-metric locality-sensitive hashing. In AAAI, pages 539--544, 2010.
[24]
M. Norouzi and D. J. Fleet. Cartesian k-means. In IEEE CVPR, pages 2938--2945, 2013.
[25]
M. Norouzi, A. Punjani, and D. J. Fleet. Fast search in hamming space with multi-index hashing. In IEEE CVPR, pages 3108--3115, 2012.
[26]
F. Shen, C. Shen, W. Liu, and H. Tao Shen. Supervised discrete hashing. In IEEE CVPR, pages 37--45, 2015.
[27]
M. Slaney, Y. Lifshits, and J. He. Optimal parameters for locality-sensitive hashing. Proceedings of the IEEE, 100(9):2604--2623, 2012.
[28]
J. Song, Y. Yang, Z. Huang, H. T. Shen, and R. Hong. Multiple feature hashing for real-time large scale near-duplicate video retrieval. In ACM MM, 2011.
[29]
X. Wu, A. G. Hauptmann, and C.-W. Ngo. Practical elimination of near-duplicates from web video search. In ACM MM, pages 218--227. ACM, 2007.
[30]
G. Ye, D. Liu, J. Wang, and S. F. Chang. Large-scale video hashing via structure learning. In IEEE ICCV, pages 2272--2279, Dec 2013.

Cited By

View all
  • (2022)Privacy-Preserving Record Linkage Using Local Sensitive Hash and Private Set IntersectionApplied Cryptography and Network Security Workshops10.1007/978-3-031-16815-4_22(398-424)Online publication date: 24-Sep-2022
  • (2018)Centralized Duplicate Removal Video Storage System with Privacy Preservation in IoTSensors10.3390/s1806181418:6(1814)Online publication date: 4-Jun-2018

Index Terms

  1. Fast Video Deduplication via Locality Sensitive Hashing with Similarity Ranking

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        ICIMCS'16: Proceedings of the International Conference on Internet Multimedia Computing and Service
        August 2016
        360 pages
        ISBN:9781450348508
        DOI:10.1145/3007669
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        In-Cooperation

        • Xidian University

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 19 August 2016

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Hash Table Indexing
        2. Locality Sensitive Hashing
        3. Similarity Ranking
        4. Video Deduplication

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Conference

        ICIMCS'16

        Acceptance Rates

        ICIMCS'16 Paper Acceptance Rate 77 of 118 submissions, 65%;
        Overall Acceptance Rate 163 of 456 submissions, 36%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)4
        • Downloads (Last 6 weeks)0
        Reflects downloads up to 15 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2022)Privacy-Preserving Record Linkage Using Local Sensitive Hash and Private Set IntersectionApplied Cryptography and Network Security Workshops10.1007/978-3-031-16815-4_22(398-424)Online publication date: 24-Sep-2022
        • (2018)Centralized Duplicate Removal Video Storage System with Privacy Preservation in IoTSensors10.3390/s1806181418:6(1814)Online publication date: 4-Jun-2018

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media