Dynamic Lookup Networks

Abstract

This paper surveys exisiting work in distributed lookup services using dynamic shared hash tables (DHTs), with the goal of pointing out open issues and key challenges. Discovering and locating resources in networked systems are quintessential problems of distributed computing. Their goal is to design a distributed lookup service that allows clients to contact the service from anywhere and find any resource in a reasonable amount of work. The solution paradigm of interest to us employs a distributed hash table(DHT), as employed, e.g., in [17.6], [17.11], 17.12, [17.14], [17.15], [17.18], [17.20] TheDHTparadigm uses a hash map to look up keys quickly. The hash table is distributed among a dynamic set of servers, each one holding a fraction of the map. An overlay network is employed in order to route each query to the server that holds the relevant information. Acentral challenge is to construct an overlay network that routes lookup queries from any starting point to the server closest to the target. Much is known on routing networks [17.4], [17.17], including networks of low degree, small dilation and even congestion, such as Shuffle-Exchange, Butterfly, and Cube-Connected-Cycle.

This work was supported in part by the Israeli Ministry of Science grant #1230-3-01.

Note that Kleinberg’s paper is descriptive, trying to explain how a social network allowing small hop routing may develop. We are in the process of developing a constructive algorithm providing a design to a network based directly on this approach.