Know when to persist: Deriving value from a stream buffer

Abstract

We consider Persistence, a new online problem concerning optimizing weighted observations in a stream of data when the observer has limited buffer capacity. A stream of weighted items arrive one at a time at the entrance of a buffer with two holding locations. A processor (or observer) can process (observe) an item at the buffer location it chooses, deriving this way the weight of the observed item as profit. The main constraint is that the processor can only move synchronously with the item stream. Persistence is the online problem of scheduling the processor movements through the buffer so that its total derived value is maximized under this constraint. We study the performance of the straight-forward heuristic Threshold, i.e., forcing the processor to “follow” an item through the whole buffer only if its value is above a threshold. We analyze both the optimal offline and Threshold algorithms in the cases where the input stream is either a random permutation, or its items are iid valued. We show that in both cases the competitive ratio achieved by the Threshold algorithm is at least 2/3 when the only statistical knowledge of the items is the median of all possible values. We generalize our results by showing that Threshold, equipped with some minimal statistical advice about the input, achieves competitive ratios in the whole spectrum between 2/3 and 1, following the variation of a newly defined density-like measure of the input. This result is a significant improvement over the case of arbitrary input streams, where we show that no online algorithm can achieve a competitive ratio better than 1/2.