System reorganization and load balancing of parallel database rule processing

Abstract

In the coming decade, high-speed network computing using processors that are orders of magnitude faster than the platforms available today, will enable the integration and coalescing of vast amounts of information stored in diverse databases. This will provide unprecedented new opportunities for acquiring new knowledge by applying various inferential processes against such massive databases. Meeting this challenge requires significant advances in our understanding of how to build efficient, high-performance knowledge-base systems targeted to run on a variety of parallel and distributed hardware architectures.

We address these concerns in the context of the PARADISER distributed rule processing system. We present an approach that combines statically computed restrictions on rule programs to partition the workload of rule evaluation among an arbitrary number of processing sites, and dynamic load balancing protocols that update and reorganize the distribution of workload at runtime. Finally, we analyze the dynamic load balancing protocols in terms of efficiency and scalability criteria.

This work has been supported in part by the New York State Science and Technology Foundation through the Center for Advanced Technology under contract NYSSTFCU01207901, and in part by NSF CISE grant CDA-90-24735.