Time-Decaying Bloom Filters for Efficient Middle-Tier Data Management

Abstract

Distributed enterprise applications are typically based on a multiple–tier client–server architecture where large volume of data is transferred between tiers frequently. When the amount and frequency of data to be transferred become large, network bandwidth will become a bottleneck and efficient middle–tier data management is critical. In this paper, we propose a semi–persistence model to capture the evolving nature of data in a middle tier data management system. We also propose to use Bloom Filters (BF) as an efficient data structure to maintain the time-sensitive frequency profile of the underlying data items. We first extend the standard Bloom Filters by replacing the bit-vector with an array of counters. We then optimize it by allocating lowest space necessary for each counter to store its value. The preliminary experiments show that the optimized BF achieves considerable improvement on space usage while providing the same results of frequency profile.