Event storm detection and identification in communication systems

Abstract

Event storms are the manifestation of an important class of abnormal behaviors in communication systems. They occur when a large number of nodes throughout the system generate a set of events within a small period of time. It is essential for network management systems to detect every event storm and identify its cause, in order to prevent and repair potential system faults.

This paper presents a set of techniques for the effective detection and identification of event storms in communication systems. First, we introduce a new algorithm to synchronize events to a single node in the system. Second, the system's event log is modeled as a normally distributed random process. This is achieved by using data analysis techniques to explore and then model the statistical behavior of the event log. Third, event storm detection is proposed using a simple test statistic combined with an exponential smoothing technique to overcome the non-stationary behavior of event logs. Fourth, the system is divided into non-overlapping regions to locate the main contributing regions of a storm. We show that this technique provides us with a method for event storm identification. Finally, experimental results from a commercially deployed multimedia communication system that uses these techniques demonstrate their effectiveness.

Introduction

Abnormal behavior in communication systems results in a number of events generated by nodes affected by such behavior. Abnormality is defined as behavior that deviates from expectation. The operator uses these events to detect and identify abnormalities in the system they manage. The problem under investigation is the detection and identification of event storms in communication systems. Event storms are an important class of abnormal behaviors in such systems. In many cases, such storms indicate failures that involve a large number of objects throughout the system. These objects react to such faults by generating a number of events within a short period of time.

The term ‘event storm’ implies that the number of events observed within a time unit is much larger than expected. In the statistical literature, such a concept has been called an ‘outlier’ among other names. An outlier is an observation that appears suspicious in light of some provisional initial assignment of a probability model to explain the data generation process [1].

Many researchers using a variety of techniques have investigated fault management in communication systems. FSM and probabilistic FSMs have been used by Refs. [2], [3], [4], [5]. A Petri Net approach to fault detection has been investigated by Refs. [6], [7]. Bayesian networks have been used by Ref. [8] for fault diagnosis in lightwave networks while the authors in Ref. [9] provided a fuzzy temporal reasoning model for event correlation. A statistical and heuristic approach to modeling of error logs captured from a distributed computer network at Carnegie Mellon University was presented in Ref. [10]. These authors presented a failure prediction heuristic based on the inter-arrival time function of errors. A review of many fault detection and identification models and algorithms is provided in Ref. [11] while the author in Ref. [12] provided an OSI model for the more general problem of network management. The statistical literature has addressed the more general problem of outlier detection [1], [13]. A time series approach is used in Ref. [14] to investigate computer system performance problem detection. A method for proactive anomaly detection in networks is introduced in Ref. [15]. Here, the authors used time series data obtained from selected MIB variables, which are processed to provide proactive alarms that are indicative of impending network problems. The exponential smoothing technique used in our work has been widely used in time series forecasting [16], [17], [18].

There are many challenges to solving the problem of event storm detection and identification in communication systems. First, in many large communication systems, nodes are not synchronized to a global clock. This is the case in the system under study. As a result there are events that cannot be temporally correlated and therefore detection and identification of such storms is not possible. This problem is solved by synchronizing all nodes in the system relative to a single node, i.e., the OMC.

Second, communication systems are dynamic in nature—the size and complexity changes on a daily basis. The result is an ever-increasing volume of events generated during a time unit. In such an environment, a small system's event storm is a large system's normal behavior. This behavior has been modeled as a non-stationary normally distributed random process, which consists of small stationary segments. To provide an effective storm detection technique in such an environment, we applied an exponential smoothing technique. The authors in Refs. [19], [20] investigated the problem of anomaly detection in an Ethernet system by monitoring packet traffic and processor loading data and show that performance anomalies are indicative of failure. The concept of exponential smoothing is used to update their system's daily behavior in a manner similar to ours. However, while analysis of performance data has been shown to be a promising technique for anomaly detection, events are more natural symptoms of abnormality in communication systems—nodes generate events in response to abnormality—and therefore are more effective for the task at hand. Moreover, performance data are system specific and more difficult to generalize.

Third, we investigate the problem of identifying event storms. Our approach consists of dividing the system into non-overlapping regions and then using the main contributing regions of a storm as its signature.

Section 2 presents the system under study. In Section 3, we present a time synchronization algorithm. In Section 4, we discuss the data analysis approach to event logs. In Section 5, we present a simple rule for detecting outliers based on the model given in Section 3 and provide a technique for solving the non-stationary behavior of event logs. In Section 6, we outline an approach to correlate event storms in time and space. In Section 7, we discuss the experimental results of the application of our techniques to the system under study.