On robust fuzzy c-regression models

Abstract

One of the most popular clustering methods based on minimization of a criterion function is the fuzzy c-means one. Its generalization by application of hyperplane shaped prototypes of the clusters is known as the Fuzzy C-Regression Models (FCRM) method. Although with this generalization many new applications of clustering emerged, it appeared to be rather sensitive to poor initialization and to the presence of noise and outliers in data. In this paper we introduce a new objective function, using the Huber's M-estimators and the Yager's OWA operators to overcome the disadvantages of the approach considered. We derive and describe an algorithm for minimization of the objective function defined. We have called it the Fuzzy C-Ordered-Regression Models (FCORM) clustering algorithm. The algorithm is compared to a few other important reference ones. To this end experiments on synthetic data with various types of noise and different numbers of outliers are carried out. We investigate the methods performance in the conditions that can be encountered in signal analysis. Large-scale simulations demonstrate the competitiveness and usefulness of the method proposed.

Introduction

The unsupervised classification of data into groups is called clustering. The method plays an important role in many engineering fields, such as pattern recognition, computer vision, machine learning, image analysis, communication, knowledge discovery, data mining and so on [1], [10], [19]. In the traditional so-called hard clustering, the groups (clusters) are disjoint. Each data item belongs to one cluster only. In [43] Zadeh introduced the notion of a fuzzy membership function. It allowed to associate with each data item and each cluster a real number in the interval [0,1] representing the “grade of membership” of this item in the cluster considered. This way Zadeh formed the basis for the development of fuzzy (or soft) clustering. However, the idea itself has been introduced by Ruspini [35] and Dunn [9], and generalized by Bezdek who developed an approach based on criterion function minimization [1].

One of the most popular clustering methods based on criterion function minimization is the Fuzzy C-Means (FCM) method which has successfully been applied to a wide variety of problems [1], [5], [6], [21], [24], [25], [33]. Many modifications of the FCM method have been proposed in the literature. Most of them rely on an inclusion of additional information into the clustering process. In an important group of FCM modifications, the information about the shapes of clusters prototypes is exploited. This information is relayed to the algorithms as the constraints on these shapes: in [1] and [26] the prototypes were constrained to linear varieties or linear elliptotypes in a feature space, in [12] to hyperellipsoidal ones and in [11] to hyperspheres in a feature space. This work deals with a modification of the Fuzzy C-Regression Models (FCRM) method, which is also called as the method of Fuzzy Switching Regression Models (FSRM)¹ [13], where the prototypes are constrained to functions (usually but not necessarily the linear ones). Many modifications of the above algorithm have recently appeared in the literature. The method proposed by Menard introduces additionally a so called noise cluster and, besides, it takes into account uncertainty with regard to data membership in linear regression models, near the models intersection [29]. In [23] ε-insensitive loss function was applied to the determination of data distances from the regression lines. A combination of clustering methods and the method of support vector regression was presented in [3]. The concept of α-cut implementation of the fuzzy c-regression models was introduced by Yang et al. [39]. To obtain robustness against bad initialization, the mountain clustering method was used [38] for the determination of linear regression parameters. In [40] support vector machines method was combined with clustering in the kernel space to improve robustness in classification problems.

In another group of fuzzy clustering modifications, the information about the data non-Gaussian distribution and the presence of noise and outliers is taken into account. This group includes: possibilistic clustering [8], [19], fuzzy noise clustering [7], $L_p$ ($0<p<1$) norm clustering [14], $L_1$ and $L_{\infty }$ norm clustering [2], [16], [17], fuzzy c-ordered-means clustering [27], time-domain-constrained clustering [25], ε-insensitive fuzzy clustering [20] and ε-insensitive fuzzy c-regression models clustering [23].

Two different approaches to clustering algorithms robustifying against outliers are worth emphasizing. They are: application of reweighting scenario based Huber's M-estimators [15] and the use of the ordered weighted averaging (OWA) operation [41]. In [27] both approaches were combined for the first time (the aim was to robustify a modification of the FCM method). The goal of this paper is to show that the Huber's M-estimators and the Yager's OWA operators can be used together to improve robustness of the method of the fuzzy c-regression models significantly. The second goal of this work is to investigate the performance of the proposed method when applied to data in the presence of noise and outliers. We will first present the proposed fuzzy c-ordered-regression models method, which exploits both approaches to be more robust, and then we will investigate its performance.

The investigations are aimed not only to present the proposed method competitiveness with respect to the reference methods but also to show its high practical value. To this end its application to the Q wave onset determination in the ECG signals can be considered. The Q wave begins the depolarization of the heart ventricles. Depolarization is followed by repolarization which ends at the offset of the T wave. Thus the both phases of the heart contraction are covered by a so called QT interval, presented in Fig. 1. Precise determination of the interval borders is of high clinical importance, yet it is rather difficult when the analyzed signals are noisy [18]. Fitting two linear models to the signal segments preceding and following the Q wave, we can determine the wave onset at the point where the lines intersect. Thus the switching regression models can help us to solve this important problem from the field of biomedical signal processing.

The paper is organized as follows: A detailed description of the Fuzzy C-Ordered Regression Models (FCORM) method is presented in Section 2. The experiments on different synthetic data, simulating the real life signals, are presented and discussed in Section 3. Finally, conclusions are drawn in Section 4.