On-the-fly, incremental, consistent reading of entire databases

Abstract

We describe an algorithm to read entire databases with locking concurrency control allowing multiple readersor an exclusive writer. The algorithm runs concurrently with the normal transaction processing (on-the-fly), and locks the entities in the database one by one (incremental). We prove that the algorithm produces consistent pictures of the database. We also show that conflicts between the algorithm and some updates, although necessary, can be resolved.

Reading entire databases consistently, on-the-fly, incremental algorithms can avoid the database downtime trade-off between frequent checkpoints and crash recovery delay, thus improving system availability and reliability. Our algorithm reads the database once and writes only to sequential output, lowering its implementation and execution costs. A simple extension runs in parallel on several processors and produces consistent pictures of entire distributed databases.