Loading [a11y]/accessibility-menu.js
Lazy exact deduplication | IEEE Conference Publication | IEEE Xplore

Abstract:

During data deduplication, on-disk fingerprint lookups lead to high disk traffic, resulting in a bottleneck. In this paper, we propose a “lazy” data deduplication method ...Show More

Abstract:

During data deduplication, on-disk fingerprint lookups lead to high disk traffic, resulting in a bottleneck. In this paper, we propose a “lazy” data deduplication method which buffers incoming fingerprints and performs on-disk lookups in batches, aiming to reduce the disk bottleneck. In deduplication in general, prefetching is used to improve the cache hit rate by exploiting locality within the incoming fingerprint stream. For lazy deduplication, we design a buffering strategy that preserves locality in order to similarly facilitate prefetching. Experimental results indicate that the lazy method improves fingerprint identification performance by over 50% compared with an “eager” method with the same data layout.
Date of Conference: 02-06 May 2016
Date Added to IEEE Xplore: 13 April 2017
ISBN Information:
Electronic ISSN: 2160-1968
Conference Location: Santa Clara, CA, USA

Contact IEEE to Subscribe

References

References is not available for this document.