Square-Wave Like Performance Change Detection Using SPC Charts and ANFIS

Abstract

While developing software products, performance regressions are always big issues in enterprise software projects. To detect possible performance regressions earlier, many performance tests are executed during development phase for thousands or ten thousands of performance metrics. In the previous researches, we introduced an automated performance anomaly detection and management framework, and showed Statistical Process Control (SPC) charts can be successfully applied to anomaly detection. In this paper, we address the special performance trends in which the existing performance anomaly detection system hardly detects the performance change especially when a performance regression is introduced and recovered again. Generally the issue comes from that the fluctuation gets aggravated and the lower and upper control limits get relaxed with the fixed sampling window size while applying SPC charts. To resolve the issue, we propose to apply automatically tuned sampling size, and to build the optimized Fuzzy detection system. ANFIS is adopted as a Fuzzy inference system to determine the appropriate sampling window size. Using the randomly generated data sets, we tune fuzzy rules and fuzzy input/output membership functions of ANFIS by learning. Finally we show simulation results of the proposed anomaly detection system.