Efficient processing of shortest path queries in evolving graph sequences

Abstract

In many applications, information is best represented as graphs. In a dynamic world, information changes and so the graphs representing the information evolve with time. We propose that historical graph-structured data be maintained for analytical processing. We call a historical evolving graph sequence an EGS. We observe that in many applications, graphs of an EGS are large and numerous, and they often exhibit much redundancy among them. We study the problem of efficient shortest path query processing on an EGS and put forward a solution framework called FVF. Two algorithms, namely, FVF-F and FVF-H, are proposed. While the FVF-F algorithm works on a sequence of flat graph clusters, the FVF-H algorithm works on a hierarchy of such clusters. Through extensive experiments on both real and synthetic datasets, we show that our FVF framework is highly efficient in shortest query processing on EGSs. Comparing FVF-F and FVF-H, the latter gives a larger speedup, is more flexible in terms of memory requirements, and is far less sensitive to parameter values.

Introduction

Graphs are a pervasive structure that is used to model the state of the world in many real-life applications. For example, users and their relationships in a social network (such as Facebook and Flickr) can be modeled as a graph, with vertices representing users and edges representing friendships among users. In a dynamic world, such relationships are continuously evolving. For example, users join Facebook and friendships are established. A graph that models the world can only capture the world’s state at a particular instant, or just a “snapshot” of the world. To fully capture the dynamics of the world, we propose that a sizable collection of snapshots should be used. For example, snapshots of the Facebook graph should be taken periodically, forming a sequence of snapshot graphs. We call such a sequence an Evolving Graph Sequence or EGS for short. One can interrogate an EGS with many interesting graph-based queries that characterize the world the snapshot graphs depict. For example, given two vertices u and v, “What is the most popular shortest path that connects u to v among all the snapshots in an EGS?” “How does the centrality of a node evolve in an EGS?” Queries of this sort, which are hardly meaningful when applied to a single graph, provide valuable insights into the dynamic world.

Many traditional studies on graph queries and algorithms, such as shortest-path and reachability, focus on answering queries efficiently on a single graph. Different from those previous works, our goal is to efficiently answer queries on a large sequence of related graphs given in an EGS. We are particularly interested in those applications in which the snapshot graphs are large, numerous, and gradually evolving. The first two properties call for highly efficient algorithms to deal with the large amount of graph data. The third property implies that successive snapshots in a graph sequence are likely similar to each other. This property allows techniques that exploit redundancies among similar snapshots to be developed to achieve high efficiency.

To further illustrate how real-world information can be modeled as an EGS, consider social networks, such as Facebook, Youtube and Flickr. In a social network, people connect to and interact with others to share their interests and experience. Social network analysis (SNA) [1] is a research area whose aims are to capture the various interactions among users and to understand users’ behavior. In most studies, the interactions among users are modeled as graphs. For example, an edge between two (user) vertices could model a simple and somewhat static friendship relationship; or an edge can be used to model a more dynamic interaction, such as whether two users have communicated within a certain period of time, or whether they have written about the same topic. Numerous statistical measures have been defined and studied on social network graphs. These include global (graph-based) measures such as the diameters, radii, and degree distributions of the graphs; and local (vertex-based) measures such as centrality. To understand the dynamics of social networks, daily snapshots of various social networks have been collected [2], [3]. For example, an EGS of a few hundred daily snapshots of Facebook’s New Orleans regional network is publicly available [3]. The graphs are big (e.g., the last snapshot contains about 60,000 vertices and about 900,000 edges), numerous (hundreds and many more if the collection were continued), and gradually evolving (successive snapshots are very similar, sharing more than 99% of their edges). These snapshot graphs facilitate many interesting studies, particularly trend analysis, which discovers pattern of change over time. For example, Fig. 1 plots the shortest-path distances between two Facebook users over a one-year period (365 snapshots). This plot gives interesting insights to how “friendships” are established in the social network — we see that the users were disconnected until snapshot #178. Since then, they got closer to each other until they finally became friends at snapshot #365. This plot reveals a few key moments in their friendship development (at snapshots #186, #304, #365). By analyzing the changes in their shortest paths at those key moments and the snapshot graphs surrounding those moments, we can answer some interesting questions: Did the users get closer because of a completely new (and shorter) path appeared, which was “disjoint” from the previously shortest path (see Fig. 2(a))? Or was it because a “short-circuiting bridge” was established (Fig. 2(b))? Or was it because a new user had arrived that acted as a “common friend” of some users along the previously shortest path (Fig. 2(c))? How “important” was this common friend in the network and how does its importance evolve over time? (E.g., how does its centrality evolve across snapshots?) Furthermore, how does the diameter of the friendship network evolve over time? All these queries may provide interesting insights to social network analysts.

In the preliminary version of this paper [4], we presented a solution framework, FVF, for evaluating queries of this kind. This paper is an extension of [4]. In this version, we focus on using the FVF solution framework to efficiently answer shortest path queries on an EGS. We show how an EGS can be effectively partitioned into graph clusters, and how these clusters can be arranged in a hierarchical structure. We extend the FVF-F algorithm, which works with a sequence of flat graph clusters to the FVF-H algorithm, which answers shortest path queries based on the cluster hierarchy. We analyze the FVF-H algorithm and show the various advantages it has over the FVF-F algorithm. In particular, we show that FVF-H offers larger speedups, is much less sensitive to parameter values, and is highly flexible in terms of memory requirements.

The remainder of this paper is organized as follows. In Section 2, we introduce the FVF framework. Sections 3 and 4 respectively present the details of using the FVF framework to pre-process an EGS and to answer shortest path queries on an EGS. In Section 5, we present issues related to storing EGS. In Section 6, we present the FVF-H algorithm. In Sections 7 and 8, we present a case study and the experimental results, respectively. In Section 9, we discuss related work. In Section 10, we conclude the paper.