Community-based anomaly detection in evolutionary networks

Abstract

Networks of dynamic systems, including social networks, the World Wide Web, climate networks, and biological networks, can be highly clustered. Detecting clusters, or communities, in such dynamic networks is an emerging area of research; however, less work has been done in terms of detecting community-based anomalies. While there has been some previous work on detecting anomalies in graph-based data, none of these anomaly detection approaches have considered an important property of evolutionary networks—their community structure. In this work, we present an approach to uncover community-based anomalies in evolutionary networks characterized by overlapping communities. We develop a parameter-free and scalable algorithm using a proposed representative-based technique to detect all six possible types of community-based anomalies: grown, shrunken, merged, split, born, and vanished communities. We detail the underlying theory required to guarantee the correctness of the algorithm. We measure the performance of the community-based anomaly detection algorithm by comparison to a non–representative-based algorithm on synthetic networks, and our experiments on synthetic datasets show that our algorithm achieves a runtime speedup of 11–46 over the baseline algorithm. We have also applied our algorithm to two real-world evolutionary networks, Food Web and Enron Email. Significant and informative community-based anomaly dynamics have been detected in both cases.

Appendices

Appendix

Proofs for Theorems and Lemmas of Section 4.1

Lemma 1

If community $C^i_t$ has more than one predecessor (or successor), the sizes of its predecessors (or successors) are either all larger than $\left| C^i_t \right|$ or all smaller than $\left| C^i_t \right|$ .

Proof

Suppose otherwise, that $C^1_t$ has a predecessor with smaller size, as well as one with a larger size. Let $C^1_{t-1}$, $C^2_{t-1}$,..., $C^n_{t-1}$ (where n ≥ 2) be all predecessors of $C^i_t$, and suppose that $\left| C^j_{t-1} \right| < \left| C^i_t \right|$ and $\left| C^k_{t-1} \right| > \left| C^i_t \right|$ for some 1 ≤ j, k ≤ n, j ≠ k. From Definition 5 and the sizes of the three communities, we know that $C^j_{t-1} \subset C^i_t$ and $C^i_t \subset C^k_{t-1}$, so $C^j_{t-1} \subset C^k_{t-1}$. However, $C^j_{t-1}$ and $C^k_{t-1}$ are both maximal cliques in the same graph, and $C^j_{t-1} \subset C^k_{t-1}$ contradicts the definition of a maximal clique. Therefore, it is impossible to have the size of one predecessor be larger than the size of the community and the size of another predecessor be smaller than the size of the community.□

Theorem 1

Let G _t and G _t + 1 both be simple, undirected graphs, where communities are defined as maximal cliques. If G _t + 1 is the perturbed graph formed by either adding edges/nodes to or removing edges/nodes from the baseline graph G _t , then there are only six possible types of community-based anomalies between G _t and G _t + 1 : grown communities, shrunken communities, merged communities, split communities, born communities, and vanished communities, as defined in Definition 7.

Proof

Assume that $C_t^1, C_t^2, \ldots, C_t^m$ are all communities in G _t and that $V_t^1, V_t^2,$ $\ldots, V_t^m$ are the node sets of the communities, respectively. Also assume that $C_{t+1}^1, C_{t+1}^2, \ldots, C_t^n$ are all communities in G _t + 1 and that $V_{t+1}^1, V_{t+1}^2, \ldots, V_{t+1}^n$ are the node sets of the communities, respectively. Here, we define $V_t^i = V_{t+1}^j$ to mean that $V_t^i$ only contains all the nodes in $V_{t+1}^j$.

To determine the type of a specific community, we only need to compare the node sets of communities in G _t + 1 with the node sets of communities in G _t . If $V_{t+1}^j = V_t^i$, where 1 ≤ i ≤ m and 1 ≤ j ≤ n, then community $C_{t+1}^j$ contains exactly those nodes in community $C_t^i$, which means that $C_{t+1}^j$ is a conserved community and not an anomaly.

In the following, we consider all possible anomalies by analyzing all possible mappings between predecessors and successors. In particular, when deciding if community $C_{t}^i$ is an anomaly, we do not need to consider the situation where $C_{t}^i$ has a single successor as long as we have covered all cases for the predecessors of $C_{t+1}^j$. If the community $C_{t}^i$ has only one successor $C_{t+1}^j$, then the community $C_{t+1}^j$ should have either one predecessor or more than one predecessor, both of which can be covered by using predecessor conditions. The same reasoning applies for not considering the case where a community has more than one successor of larger size. In other words, we need to consider all cases for predecessors, but only two cases for successors: when a community has no successor and when a community has more than one successor of smaller size.

• 1. For a specific j (where 1 ≤ j ≤ n), there is at least one i (where 1 ≤ i ≤ m) that satisfies $V_{t+1}^j \subset V_t^i$. Then, by Definition 5, community $C_{t+1}^j$ has at least one predecessor, including $C_{t}^i$, with larger size than $C_{t+1}^j$. Let $I =\{i \mid V_{t+1}^j \subset V_t^i\}$. There are two non-exclusive sub-cases here:

  • (a) For ℓ ∈ I, if there is some k (where 1 ≤ k ≤ n) other than j that satisfies $V_{t+1}^k \subset V_t^\ell$, then $C_t^\ell$ has more than one smaller-size successor ($C_{t+1}^j$ and $C_{t+1}^k$). Additionally, by Lemma 1, we know that $C_{t}^\ell$ cannot have a successor with larger size than $C_{t}^j$. Thus, $C_{t}^\ell$ is a split community, and $C_{t+1}^j$ is one of its products.

  • (b) For ℓ ∈ I, if there is no k (where 1 ≤ k ≤ n) other than j that satisfies $V_{t+1}^k \subset V_t^\ell$, then $C_t^\ell$ has only one smaller-size successor $C_{t+1}^j$, and $C_{t+1}^j$ has at least one predecessor, including $C_t^\ell$, with larger size. Also, by Lemma 1, we know that $C_{t+1}^j$ cannot have a predecessor with smaller size than $C_{t+1}^j$. Thus, $C_{t+1}^j$ is a shrunken community.

• 2. For a specific j (where 1 ≤ j ≤ n), there is only one i (where 1 ≤ i ≤ m) that satisfies $V_{t+1}^j \supset V_t^i$. Then, community $C_{t+1}^j$ has one predecessor $C_{t}^i$ with smaller size than $C_{t+1}^j$. Additionally, by Lemma 1, we know that $C_{t+1}^j$ cannot have a predecessor with larger size than $C_{t+1}^j$. Thus, community $C_{t+1}^j$ is a grown community.

• 3. For a specific j (where 1 ≤ j ≤ n), there is more than one i (where 1 ≤ i ≤ m) that satisfies $V_{t+1}^j \supset V_t^i$. Then, community $C_{t+1}^j$ has more than one predecessor with smaller size. Also, by Lemma 1, we know that $C_{t+1}^j$ cannot have a predecessor with larger size than $C_{t+1}^j$. Thus, community $C_{t+1}^j$ is a merged community.

• 4. For a specific j (where 1 ≤ j ≤ n), there is no i (where 1 ≤ i ≤ m) that satisfies $V_{t+1}^j \supset V_t^i$ or $V_{t+1}^j \subset V_t^i$, which means that community $C_{t+1}^j$ has no predecessor. Thus, $C_{t+1}^j$ is a born community.

• 5. For a specific i (where 1 ≤ i ≤ m), there is at least one j (where 1 ≤ j ≤ n) that satisfies $V_{t+1}^j \subset V_t^i$. Let $J = \{j \mid V_{t+1}^{j} \subset V_t^i\}$. Then, for each k ∈ J, there is at least one i (where 1 ≤ i ≤ m) that satisfies $V_{t+1}^{k} \subset V_t^i$, which is case 1. Thus, this case can be converted to case 1.

• 6. For a specific i (where 1 ≤ i ≤ m), there is at least one j (where 1 ≤ j ≤ n) that satisfies $V_{t+1}^j \supset V_t^i$. Let $J = \{j \mid V_{t+1}^{j} \supset V_t^i\}$. Then, for each k ∈ J, there is at least one i (where 1 ≤ i ≤ m) that satisfies $V_{t+1}^{k} \supset V_t^i$, which is case 2 or 3. Thus, this case can be converted to case 2 or 3.

• 7. For a specific i (where 1 ≤ i ≤ m), there is no j (where 1 ≤ j ≤ n) that satisfies $V_{t+1}^j \supset V_t^i$ or $V_{t+1}^j \subset V_t^i$, which means that community $C_{t}^i$ has no successor. Thus, $C_{t}^i$ is a vanished community.

Since all relationships between $V_{t+1}^j$ (where 1 ≤ j ≤ n) and $V_t^i$ (where 1 ≤ i ≤ m) have been covered, there are only six possible different types of community-based anomalies.□

Theorem 2

If community $C_t^i$ is represented by vertex $v_i \in C_t^i$ , and community $C_{t+1}^j$ is represented by vertex $v_j \in C_{t+1}^j$ , where $C_t^i \to C_{t+1}^j$ , then $v_i \in C_{t+1}^j$ or $v_j \in C_t^i$.

Proof

By Definition 5, $C_t^i \to C_{t+1}^j$ implies that $C_t^i \subseteq C_{t+1}^j$ or $C_{t+1}^j \subseteq C_t^i$. If $C_t^i \subseteq C_{t+1}^j$, then $v_i \in C_{t+1}^j$, and if $C_{t+1}^j \subseteq C_t^i$, then $v_j \in C_t^i$.□