PR-FCM: A polynomial regression-based fuzzy C-means algorithm for attribute-associated data

Highlights

• A novel fuzzy c-means algorithm is proposed for attribute-associated data clustering.

• The parameters of proposed algorithms are fully investigated through synthetic datasets.

• The proposed algorithm performs better compared with others on synthetic, real-world, and tunnel boring machine datasets.

Abstract

Partitioning data into internally homogeneous parts is an important problem when mining in situ engineering data. In this paper, a polynomial regression-based fuzzy c-means (PR-FCM) clustering algorithm that utilizes the functional relationships among the attributes of the input dataset is proposed. In this algorithm, a polynomial regression equation is taken as the center of each cluster instead of the cluster prototype used in conventional FCM, and the difference between a sample and a cluster prototype is defined as the distance between the actual value of one attribute and the corresponding predicted value provided by its own polynomial regression equation. An alternating optimization method is designed to optimize the new clustering objective function of the proposed algorithm. A series of experiments on synthetic and real-world datasets are conducted to evaluate the performance of the PR-FCM algorithm, which exhibits higher effectiveness and possesses more advantages than the original FCM algorithm. The PR-FCM algorithm is applied to tunnel boring machine (TBM) operation data from a TBM project in China. The experimental results show that the proposed algorithm can effectively cluster TBM operation data.

Introduction

With the development of information technology, massive operational data have been measured and recorded for complex engineering systems, promoting the development of data-driven techniques for handling in situ engineering data in recent years. Substantial literature has demonstrated that the information mined from operation data can be used to improve the design, control, and execution of engineering systems [1], [2], [3], [4]. However, the operating state of an engineering system usually changes because it experiences different working conditions, which means that the corresponding operation data patterns change greatly as well. Thus, it is necessary to partition these operation data such that the characteristics in the same part of the dataset are more similar than those in other parts to facilitate the design and analysis of engineering systems, as discussed in Refs. [5], [6]. Data clustering is a strong tool for solving this problem.

Data clustering is an important branch of unsupervised learning; the purpose is to divide the input data according to some criteria so that the data in the same cluster are as similar as possible [7], [8]. One of the most widely used clustering methods is the fuzzy c-means (FCM) algorithm [9], [10]. The FCM algorithm assigns membership statuses to each datum, and these statuses are inversely related to the relative distances from the datum to the cluster prototypes that act as the cluster centers in FCM. The closer a datum is to the center of a cluster, the higher its degree of membership is with this cluster. Because of the concept of membership in FCM, many aspects of real life possess no clear boundaries. Therefore, this approach is widely used in various areas, such as pattern recognition [11], image segmentation [12], and fault detection [13]. For human emotion recognition, Liliana et al. [14] proposed an algorithm to detect facial expressions based on the active appearance model and semisupervised FCM. Liu et al. [15] used the FCM algorithm to detect defect edges in infrared images, providing better performance than that of classic edge detection operators. In [16], Zhao et al. proposed an enhanced gravitation search-based FCM algorithm to identify abnormal power system data. By adding two features representing pixels to the FCM algorithm, Kalti and Mohamed [17] improved the accuracy of image segmentation. Ramos et al. [18] used density-oriented FCM and kernel FCM algorithms to design data-driven fault diagnosis systems. Barraza et al. [19] applied the fireworks algorithm to the FCM algorithm to find the optimal number of clusters required for achieving a better clustering effect. In [20], the context-aware spatial constraints and local membership matrix information were incorporated into the classic FCM algorithm, forming a robust FCM algorithm for the segmentation of brain tissues in magnetic resonance imaging. Zhao et al. [21] clustered acoustic emission signals by a combination of the FCM algorithm and principal component analysis.

The abovementioned works have provided insights into the availability and potential benefits of FCM for engineering design and analysis. However, the engineering data belonging to different clusters usually overlap greatly, which means that the traditional FCM algorithm cannot provide accurate clustering results in some engineering practices since it partitions data according to their spatial distances. It is necessary to utilize other data patterns, especially nonlinear data patterns, to improve the clustering accuracy of the algorithm. The kernel method has been introduced to fuzzy clustering algorithms to map the input data into a higher-dimensional feature space [22]. The kernel-based fuzzy clustering (FKCM) algorithm is able to recognize the nonlinearity of data, and it has been successfully applied in image segmentation, incomplete data clustering, and noisy data clustering tasks in recent years [23], [24]. In addition, shell-based fuzzy clustering is another algorithm that can address clustering problems with nonlinear data. Unlike the kernel-based clustering method, shell-based clustering takes shells as prototypes and divides the input data into different shells in space [25], [26]. To represent clusters with contour formats [27], shell-based clustering has been applied as an alternative to the FCM algorithm in recent years [28]. In addition, other extensions of the FCM algorithm, such as the relational FCM algorithm [29] and the FCM algorithm that incorporates spatial information [30], are available for representing complex data structures. Some related works regarding extensions of the FCM algorithm are briefly listed in Table 1.

For in situ engineering data, however, it is commonly known that the relationships among the data attributes vary considerably under different working conditions or different operation states. Thus, some in situ engineering data, such as economic data, meteorological data, and equipment detection data, can be seen as functional data. To address this issue, the regression-based FCM algorithm has been proposed. Hathaway et al. [39] proposed a fuzzy c-regression model (FCRM) for functional data based on the ideas of expectation maximization (EM) and regression. Wedel and Steenkamp [40] proposed algorithms for fuzzy clusterwise regression by improving the target function within the framework of preference analysis. Yamakawa [41] added principal component analysis to the FCRM, solving the problem of the FCRM not performing well on high-dimensional datasets. Conversely, Sato-Ilk [42] introduced the kernel method to the FCRM to map it to a high dimension; this approach was aimed at data with complex spatial distributions. Zhao [43] improved the measurement process between samples and clusters by balancing the fitting deviation and spatial distance, endowing the developed clustering method with a richer physical meaning.

Previous works on regression-based FCM algorithms illustrate that these approaches can solve problems involving the partitioning of data functional relationships to some extent. In practice, however, engineering equipment often suffers from poor working environments, causing the working status of the equipment to vary. In this context, in situ engineering data tend to present complex nonlinear functional relationships, which are hard to partition with the previous linear regression-based FCM algorithms because of the limited approximation capability of the linear regression technique for data with complex nonlinear functional structures. To address this issue, this paper presents a polynomial regression-based FCM (PR-FCM) clustering algorithm to solve the problem of clustering complex in situ engineering data. The proposed algorithm clusters the input data based on the nonlinear functional relationships among the attributes of the data but not their spatial distributions. In the proposed algorithm, we use PR to describe these nonlinear functional relationships and utilize the PR of each cluster to replace the traditional clustering prototypes and construct a new fuzzy clustering-based objective function. Then, the corresponding optimization method for the proposed clustering objective function is designed. The PR-FCM algorithm can extract the information contained in the nonlinear functional relationships among the data attributes. Therefore, compared to the existing fuzzy clustering algorithm, the PR-FCM algorithm performs better for data with nonlinear functional relationships. The main contributions of the work are summarized as follows:

• PR is introduced into the FCM framework to represent complex functional structures, and a corresponding iterative optimization method is designed to optimize the parameters of the algorithm. The proposed PR-FCM algorithm partitions data with nonlinear functional relationships; this task is difficult to complete with the traditional clustering algorithm.

• The computational complexity of the proposal is analyzed theoretically. In addition, the factors that affect the performance of the PR-FCM algorithm are studied systematically, including the polynomial order and the sample size. These investigations provide meaningful guidance and great convenience for the setup of the algorithm and further improve its effectiveness.

• The PR-FCM algorithm is compared with several benchmark algorithms on not only synthetic datasets but also real-world and tunnel boring machine (TBM) datasets. The results indicate that the PR-FCM algorithm performs better than other approaches and imply the applicability of the PR-FCM algorithm in engineering data mining.

The remainder of the paper is distributed as follows. 2 Preliminaries, 3 Proposed algorithm introduce the details of FCM, the PR algorithm, and the proposed algorithm. In 4 Experiments on synthetic datasets, 5 Experiments on real-world datasets, several synthetic datasets and real-world datasets are used to test and compare the performance of the PR-FCM algorithm with that of several benchmark algorithms. In Section 6, the proposed algorithm is applied to a real TBM operation dataset to demonstrate its effectiveness and advances in engineering data clustering. Some conclusions are given in Section 7.