Long-term prediction of time series using fuzzy cognitive maps

Abstract

As a powerful recognized knowledge modeling tool, fuzzy cognitive maps (FCMs) have been investigated for time series modeling and forecasting problems. This methodology performs well in one-step-ahead or short-term prediction but poorly in terms of long-term prediction because of the potentially complex interaction between different ensuing steps. In this article, a sound conceptual method is proposed for long-term time series prediction with FCMs, which melds FCMs, time series segmentation and fuzzy clustering. A time series is divided into suitable and internally homogeneous segments. Dynamic time warping is introduced to evaluate the distance between segments. Subsequently, modified fuzzy c-means based on dynamic time warping is utilized to fuzzify these segments such that the segments are transformed into fuzzy time series and semantic vectors. The convex optimization based method is utilized with intent to rapidly and robustly learn FCMs. Consequently, the weight of FCMs can be obtained on the basis of the fuzzy time series. Eventually, the forecasting time segment will be capable of inference according to the formed FCMs and the semantic vectors. In addition, the semantic vectors can intuitively reflect the main characteristics and change tendencies of the time series. To demonstrate the long-term prediction ability of our method, we test it on both synthetic and real-life datasets in comparison with other representative and up-to-date forecasting methods; the superior performance of our method exhibits its excellent capability in forecasting future values.

Introduction

Time series forecasting is the endeavor of providing speculations by understanding the past (Makridakis, 1994), which has been widely applied in numerous practical applications such as economics, climatology and industries. The significance of time series forecasting is that the decision-making can be efficiently and effectively conducted in these areas. During the past decades, different time series models have been proposed, including traditional and fuzzy methods. The traditional time series models, such as autoregressive integrated moving average (ARIMA) (Box et al., 2015, Lee and Tong, 2011), artificial neural networks model and hybrid ARIMA model (Zhang, 2003, Kihoro et al., 2004, Khashei et al., 2012), and support vector machine (Cao and Tay, 2003, Chaâbane, 2014), have failed to address the prediction problem under the uncertain circumstances in which historical data are incomplete, imprecise or ambiguous. To resolve this problem, the fuzzy time series model provides a feasible alternative to guarantee robustness of the forecasting models (Song and Chissom, 1994, Singh, 2017, Bose and Mali, 2019, Singh, 2016). Most currently available time series forecasting models concentrate on one-step-ahead prediction. As a new challenge in time series modeling, long-term prediction (multistep-ahead prediction) is more pervasive in many cases (Sorjamaa et al., 2007, Helmi et al., 2018). Sorjamaa et al. (2008) proposed a combination of methodologies based on extreme learning machine, partial least squares and nonparametric noise estimation. The combination of the methods projects the high-dimensional input regressor into low-dimensional latent space, maximizing the prediction ability of any nonlinear approximator. Cortez et al. (2019) explored a large set of neural network methods that directly optimize prediction intervals for multistep time series forecasting. Galicia et al. (2019) explored the suitability of combining three methods (decision tree, gradient boosted trees and random forest) into ensembles to forecast big data time series. The ensemble method, which computes the weights for each ensemble member using a least square method, assigns higher weights to the more accurate ensemble members based on their past performance. Koesdwiady et al. (2018) proposed two methods to improve multistep time series prediction: time-step-augmented and conditional generative adversarial network based models. The first method augments the information about the current time step and follows a training process similar to that of the meta algorithm. The second model is developed using multioutput strategy and utilizes the ability of the generative adversarial network in mimicking a dataset distribution. Alameer et al. (2020) proposed an end-to-end deep learning architecture for accurately forecasting monthly coal price fluctuations at different horizons, which combines long-short term memory and a deep neural network. Liu et al. (2021) introduced a two-layers extreme learning machine with the new recurrent algorithm for multistep time series prediction. This method applied a new recurrent technique that not only removed the restriction of the prediction horizon problem but also used a mean squared error of the current step to update the output weights for the next step. Taieb and Atiya (2016) presented a review of the available literature and a comprehensive investigation into the bias and variance behavior of long-term forecasting strategies. In this survey, multistep-ahead forecasting strategies are classified into three major categories: recursive, direct and joint. In the recursive strategy, prediction models iteratively forecast one step at a time with previous predictions as the model inputs. In the direct strategy, separate models are trained for each step ahead. In the joint strategy, a single model that has multiple outputs predicts the whole prediction horizon with a single attempt. In addition, the joint methods exhibit better performance with respect to both forecasting bias and variance (Taieb and Atiya, 2016).

Compared with one-step-ahead prediction, the difficulties of long-term prediction are uncertainty and potentially complex interactions between the different ensuing steps. To overcome this difficulty, one viable solution is to concentrate on high level representations of time series rather than the specific individual values. Li et al. (2010) utilized a vector quantization technique to forecast long-term vector values in one step. This approach, called deterministic vector long-term forecasting, took advantage of the sliding window method to extract features of interest in a time series and fuzzy c-means clustering to fuzzify interval partitioning. In view of the levels of vagueness and uncertainty of medical problems, Wang et al. (2015) used fuzzy information granulation (Zadeh, 1979, Pedrycz and Vukovich, 2001) to segment time series and extract abstract features of the subsequences, and then a multiple fuzzy rules interpolation scheme (Chang et al., 2008) was applied for long-term prediction of time series. Analogously, Yang et al. (2017) structured a linear fuzzy information granulation to reflect the time-dependent trend of change, and Gaussian type fuzzy sets were selected to construct fuzzy information granulation. Subsequently, the constructed granular time series were utilized for training a fuzzy inference system that predicted long-term values of the original time series. To improve the interpretability of long-term prediction, Guo et al. (2018) exploited hidden Markov models to derive the relations existing in the granular time series, which were nonoverlapping segments of the time series based on the principle of justifiable granularity. Similarly to the above approaches, Luo et al. (2020) achieved the prediction of long-term fluctuation of time series in the light of the short-term fluctuation patterns by means of polar coordinate fuzzy information granules.

The above research works revealed that long-term forecast modeling based on fuzzy time series has attracted significant attention during the past years. As a soft computing tool, fuzzy cognitive maps (FCMs) introduced by Kosko et al. (1986) can be used to describe and model the complex system (Papageorgiou and Salmeron, 2013). The FCMs consist of concept nodes and directed weights. The nodes demonstrate different aspects in the behavior of the system and the weights reflect causality presented between concepts. The review article (Felix et al., 2019) presents an up-to-date and comprehensive presentation of the theory and applications of FCMs. On the basis of knowledge-based representation and realizing inference processes, FCMs are able to capture the behavior of a given dynamic system. When an FCMs is designed to model time series, it can be broken down into the following four major tasks: input fuzzification, FCMs learning, modeling, and output defuzzification (Stach et al., 2008). The weight matrix should be determined first when using FCMs. The learning problem of FCMs is concentrated on acquiring the weight matrix based on expert intervention, available historical data or both. The data-based strategy is more suitable to learn FCMs when multiple training sequences are available and there is a lack of a priori knowledge, which can also increase the robustness and generalization of FCMs models (Chen et al., 2015, Papageorgiou and Salmeron, 2013). In our previous work (Lu et al., 2019), a fast and effective learning method for FCMs based on convex optimization is proposed. Following this strategy, Stach et al. (2008) exploited fuzzy cognitive maps (FCMs), which is learned by real-coded genetic algorithms, to perform prediction both at numerical and linguistic levels. For further improving the prediction accuracy of the FCMs model, Lu et al. (2014) designed a high-order fuzzy cognitive map (HFCM) to model and predict time series, in which fuzzy c-means clustering algorithm was used to construct the framework of FCMs and genetic algorithm is applied to learn the weights. The single concept of HFCM depends not only on the last states of all concepts but also on states that are multiple steps ahead, so the approximation ability of FCMs was enhanced. Pedrycz et al. (2015) introduced a framework for description of a numeric time series aided with information granules, which is constructed in the space of amplitude and change in amplitude of the time series. Each information granule, formed with the help of the fuzzy c-means clustering algorithm, was mapped onto a concept of the FCMs. The influences of the first step and the last step of the learning part of time series on the resulting FCMs should be different. Based on this, Salmeron and Froelich (2016) investigated a dynamic optimization of FCMs for time series forecasting with the goal of increasing the accuracy of forecasting. In this approach, the weights of the FCMs and the length of the learning period were dynamically adjusted according to the local characteristics of the time series. For multivariate time series prediction, Papageorgiou and Poczeta (2017) developed a two-stage model that combined evolutionary FCMs and artificial neural networks predictors in a cascade form. For dealing with nonstationary time series, Yang and Liu (2018) resorted to wavelet transform to decompose original nonstationary time series into multivariate time series, and then the high-order FCMs was applied to model and predict multivariate time series. Papageorgiou and Froelich (2012) proposed a long-term predictive model based on FCMs to forecast the state of pneumonia, which was learned by a multistep enhancement of the evolutionary algorithm.

Together, the above-mentioned studies indicate that FCMs and its extension have been successfully applied for time series forecasting. However, most research on forecasting models has been carried out for one-step-ahead prediction. To date, there are few studies that have investigated the long-term predictive models referring to FCMs for time series.

In view of all the observations made above, the ultimate objective of this study is to propose a novel method based on FCMs to improve the prediction accuracy of long-term time series forecasting. Initially, the numerical time series is divided into sequential segments resorting to the piecewise linear representation. Afterwards, the segments are transformed into equalized lengths to the prediction horizon based on dynamic time warping (DTW). Then, the extended fuzzy c-means clustering algorithm is used to cluster all of these segments. As a result, the segments are transformed into multivariate fuzzy time series. Following this, the forecasting model based on FCMs can be constructed from these fuzzy time series, in which the learning of FCMs is realized by a convex optimization method. The key contributions of this study are summarized as follows:

• FCMs is innovatively applied to construct a long-term time series prediction model. In the FCMs model, each node of FCMs represents one latent variation modality of time series with regard to the prediction horizon and the weights depict the causal relationship that exists among these modalities. The proposed method handles the time series from the perspective of facilitating human understanding and cognition.

• The unequal-length segments are equalized based on DTW distance, and the modified fuzzy c-means clustering is applied in this method to fuzzify these time series segments.

The remainder of this paper is organized as follows. Section 2 introduces the learning problem of FCMs and time series clustering based on DTW. In Section 3, the complete framework of the proposed long-term prediction model is presented. In Section 4, the experiments based on some publicly available datasets are exploited to demonstrate the advantages of the proposed model. Section 5 provides some helpful conclusions.