Time series forecasting based on kernel mapping and high-order fuzzy cognitive maps

doi:10.1016/j.knosys.2020.106359

Knowledge-Based Systems

Volume 206, 28 October 2020, 106359

https://doi.org/10.1016/j.knosys.2020.106359 Get rights and content

Abstract

Fuzzy cognitive maps (FCMs) have emerged as a powerful tool for dealing with the task of time series prediction. Most existing research devoted to designing an effective method to extract feature time series from the original time series, which are used to construct FCMs and predict the time series. However, in existing methods, all extracted feature time series, including the redundant feature time series, were used to develop FCMs instead of selecting the key feature time series (KFTS) to construct FCMs, which limits the generalization and prediction accuracy of the models. In this paper, we propose a framework based on kernel mapping and high-order FCMs (HFCM) to forecast time series inspired by the kernel methods and support vector regression (SVR). The model is termed as Kernel-HFCM. Kernel mapping is designed to map the original one-dimensional time series into multidimensional feature time series, and then the feature selection algorithm is proposed to select the KFTS from the multidimensional feature time series to develop the HFCM. Finally, reverse kernel mapping is used to map the feature time series back to the predicted one-dimensional time series. In comparison to the existing methods, the experimental results on seven benchmark datasets demonstrate the effectiveness of Kernel-HFCM in time series prediction.

Introduction

Fuzzy cognitive maps (FCMs) [1], the powerful fuzzy models inheriting the main attributes of fuzzy logic and neural networks, are effective for simulating and modeling complex systems. FCMs are made up of vertices and directed weight edges connecting the vertices. The vertices in FCMs are utilized to represent real-world concepts, and the directed weight edges denote the relations between the vertices. FCMs possess advantages in terms of the fast numerical reasoning ability, good interpretability, and intuitive knowledge representation, and thus have wide applications in multiple fields, including disorder diagnosis and classification [2], decision making [3], [4], time series modeling [5], and time series prediction [6], [7]. Time series is simply a series of data points ordered in time, such as. Time series prediction is the use of models to predict the future value of $l_{t + 1}$ based on the observed historical time series. Accurate time series forecasting is of great significance in finance, economy, environmental protection, and agriculture, etc. Thus, numerous algorithms based on FCMs were designed to model and predict time series. Overall, the feature time series were extracted from the original time series to develop FCMs [8], [9], [10], [11], [12]. Then parameter optimization methods were used to learn FCMs. Over the past years, many parameter optimization algorithms have been proposed to develop the FCMs, such as Hebbian rule [13], particle swarm optimization (PSO) [14], real coded genetic algorithm (RCGA) [15], modified asexual reproduction optimization (MARO) [16], and imperialist competitive learning algorithm (ICLA) [17]. Given the definition of FCMs, their dynamics are of the first-order, meaning that the next state depends upon the one in the previous iteration, which limits the ability of FCMs to model complex systems. To enhance the modeling capabilities of FCMs, high-order FCMs (HFCM) were proposed to enhance the approximation ability of FCMs [18]. RCGA-based learning method [18] and the gradient optimization method [19] were also designed to develop the HFCM.

The kernel methods [20], [21], [22] are well-known methods for nonlinear pattern analysis, whose core idea is embedding the original data into the appropriate high-dimensional feature space by some kinds of nonlinear mapping, and then the general linear learner is used to analyze and process the pattern in this new space. The kernel methods assume that the data is nonlinearly distributed in the low-dimensional space, and the distribution is likely to be linear after being transformed into the high-dimensional feature space. Due to the powerful ability in analyzing the nonlinear pattern, the models based on the kernel methods have been used in classification [23], [24], regression [25], [26], prediction [27], [28], etc.

Although FCMs have been successfully applied to time series modeling and prediction, these FCM-based prediction models still have many shortcomings, mainly reflected in the following aspects. Existing feature time series extraction algorithms based on FCMs are not effective enough. For instance, the models in [8], [9], [10], [11], [12] transformed the original time series into fuzzy time series, which lost part of the trend, state, and spectrum features of the original time series. Therefore, the accuracy of time series prediction was insufficient. The model in [19] applied the Harr wavelet transform to extract the feature time series and the model in [29] applied the fuzzy c-means to extract the feature time series. All feature time series, including the redundant feature time series, were used to develop FCMs instead of selecting the key feature time series (KFTS) to construct FCMs, which made the model less accurate on different complex time series prediction problems. In other words, the feature time series used to develop FCMs were not representative enough.

Because of the limitations of existing FCMs-based time series prediction models, we propose a novel time series prediction framework by the hybrid combination of kernel mapping and HFCM inspired by the kernel methods and the support vector regression (SVR). The framework is termed as Kernel-HFCM. Kernel mapping is a highly effective and generalized feature time series extraction method, which is designed to embed the original one-dimensional time series into the multidimensional feature time series space, and then the feature selection algorithm is proposed to select the KFTS from the multidimensional feature time series space, and the KFTS are used to develop the structure of HFCM to model the pattern of the KFTS. After acquiring the pattern of the KFTS, the future time series values will be predicted precisely. In comparison to the existing methods, the experimental results on seven benchmark datasets demonstrate the effectiveness of our proposal in complex non-stationary time series prediction. The novelty of the proposed Kernel-HFCM is summarized as follows,

(1) Kernel-HFCM is first proposed for time series analysis inspired by the kernel methods and SVR. The experimental results validate the superiority of our framework in prediction accuracy compared with the state-of-the-art methods.

(2) The kernel mapping is proposed firstly to extract the feature time series from the original time series, and then the feature selection algorithm is designed to select the KFTS to develop the HFCM. To the best of our knowledge, feature selection is first introduced to the FCM-based time series prediction. Inheriting the properties of the kernel mapping and feature selection, Kernel-HFCM overcomes the shortcomings that the redundant feature time series were used to develop FCMs.

The rest of this paper is organized as follows: Section 2 provides an introduction on FCMs. Section 3 reviews the related work on FCMs based time series prediction. The proposed Kernel-HFCM is introduced in detail in Section 4. The experiments and discussion are given in Section 5. Finally, conclusions are given in Section 6.

Section snippets

Fuzzy cognitive maps

FCMs are models consisting of a set of conceptual nodes and fuzzy relationships between nodes, which can be presented as directed weighted graphs. Suppose the number of nodes of an FCM is $N_{c}$ , then the state value vector C of conceptual nodes is defined as follows, $C = [\begin{matrix} C_{1} & C_{2} & \dots & C_{N_{c}} \end{matrix}]$ where $C_{i}$ ranges in [−1, 1] (or [0, 1]), $i = 1, 2, \dots, N_{c}$ . The state value of each conceptual node represents the activation degree of the current node. The fuzzy relationships between nodes are defined as an $N_{c} \times N_{c}$ weight matrix W. $W$

Related work

Overall, the feature time series are extracted from the original time series, which are used to develop FCMs [6], such as the models in [8], [9], [10], [11], [12]. Stach et al. [8] transformed the original time series into information granules including change of the time series and amplitude of the time series, and granular time series were extracted from the information granules, which were used as the train data to construct and learn FCMs. Froelich et al. [9] proposed a new method to

Kernel-HFCM

In this section, a novel framework of time series prediction, termed as Kernel-HFCM, is introduced. Fig. 2 shows the overall framework of Kernel-HFCM. First, the original time series are normalized into the range of [0, 1] or [−1, 1]. Second, the kernel mapping is designed to map the original one-dimensional time series into the multidimensional feature time series, and then the key feature time series are selected from the multidimensional feature time series to develop the HFCM. Third, the

Datasets

Seven benchmark time series datasets are used to verify the effectiveness of our proposed Kernel-HFCM in time series prediction. All of them can be obtained from the Yahoo finance website [40]. The descriptions of the datasets are shown in Table 1.

Experimental setup

The activation function of Kernel-HFCM in this paper is set to the sigmoid function, whose output is in [0, 1]. Therefore, the original time series are normalized into the range of [0, 1] before feeding them into Kernel-HFCM. Besides, the

Conclusions

Inspired by SVR and the kernel methods, a time series prediction model named Kernel-HFCM is proposed based on the kernel mapping and the HFCM. The kernel mapping is designed to map the original one-dimensional time series to the high-dimension feature space, and then the KFTS are selected from the high-dimensional feature space to develop the HFCM. To avoid overfitting, ridge regression is applied to design the cost function. Finally, the reverse kernel mapping transforms the feature time

CRediT authorship contribution statement

Kaixin Yuan: Conceptualization, Methodology, Software, Validation, Formal analysis, Writing - original draft, Writing - review & editing. Jing Liu: Writing - review & editing, Supervision, Funding acquisition. Shanchao Yang: Writing - review & editing. Kai Wu: Writing - review & editing. Fang Shen: Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported in part by the Key Project of Science and Technology Innovation 2030 supported by the Ministry of Science and Technology of China under Grant 2018AAA0101302 and in part by the General Program of National Natural Science Foundation of China (NSFC) under Grant 61773300.

References (44)

MizumotoM. et al.
Comparison of fuzzy reasoning methods
Fuzzy Sets and Systems
(1982)
FroelichW. et al.
Fuzzy cognitive maps in the modeling of granular time series
Knowl.-Based Syst.
(2017)
FroelichW. et al.
Fuzzy cognitive maps in the modeling of granular time series
Knowl.-Based Syst.
(2017)
LuW. et al.
The modeling and prediction of time series based on synergy of high-order fuzzy cognitive map and fuzzy c-means clustering
Knowl.-Based Syst.
(2014)
SongH.J. et al.
Design of fuzzy cognitive maps using neural networks for predicting chaotic time series
Neural Netw.
(2010)
StachW. et al.
Genetic learning of fuzzy cognitive maps
Fuzzy Sets and Systems
(2005)
SalmeronJ.L. et al.
Learning fuzzy cognitive maps with modified asexual reproduction optimisation algorithm
Knowl.-Based Syst.
(2019)
MinJ.H. et al.
Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters
Expert Syst. Appl.
(2005)
TsadirasA.K.
Comparing the inference capabilities of binary, trivalent and sigmoid fuzzy cognitive maps
Inf. Sci. (N.Y.)
(2008)
WuK. et al.
Robust learning of large-scale fuzzy cognitive maps via the lasso from noisy time series
Knowl.-Based Syst.
(2016)

LiuZ. et al.

A robust time series prediction method based on empirical mode decomposition and high-order fuzzy cognitive maps

Knowl.-Based Syst.

(2020)

AmirkhaniA. et al.

A novel hybrid method based on fuzzy cognitive maps and fuzzy clustering algorithms for grading celiac disease

Neural Comput. Appl.

(2018)

BakhtavarE. et al.

Assessment of workplace accident risks in underground collieries by integrating a multi-goal cause-and-effect analysis method with MCDM sensitivity analysis

Stoch. Environ. Res. Risk Assess.

(2018)

PapageorgiouK. et al.

Decision-making process for photovoltaic solar energy sector development using fuzzy cognitive map technique

Energies

(2020)

WuK. et al.

Time series prediction using sparse autoencoder and high-order fuzzy cognitive maps

IEEE Trans. Fuzzy Syst.

(2020)

RezaeeM.J. et al.

A decision system using fuzzy cognitive map and multi-group data envelopment analysis to estimate hospitals’ outputs level

Neural Comput. Appl.

(2018)

StachW. et al.

Numerical and linguistic prediction of time series with the use of fuzzy cognitive maps

IEEE Trans. Fuzzy Syst.

(2008)

W. Lu, J. Yang, X. Liu, The linguistic forecasting of time series based on fuzzy cognitive maps, in: 2013 Joint IFSA...

E. Papageorgiou, C. Stylios, P. Groumpos, Fuzzy cognitive map learning based on nonlinear Hebbian rule, in:...

PapageorgiouE.I. et al.

Fuzzy cognitive maps learning using particle swarm optimization

J. Intell. Inf. Syst.

(2005)

S. Ahmadi, S. Alizadeh, N. Forouzideh, C.-H. Yeh, R. Martin, E. Papageorgiou, ICLA imperialist competitive learning...

W. Stach, L. Kurgan, W. Pedrycz, Higher-order fuzzy cognitive maps, in: NAFIPS 2006-2006 Annual Meeting of the North...

Cited by (30)

A comprehensive framework for designing and learning fuzzy cognitive maps at the granular level
2024, Applied Soft Computing
Fuzzy Cognitive Maps (FCM) possesses interpretability and fuzzy reasoning capability. It is widely applied in addressing time series prediction problems and various algorithms for learning FCM are proposed. Information granulation method can transfer knowledge from numerical data to granular knowledge. However, how to learn FCM better at the granular level is a challenge. Therefore, we develop a comprehensive framework, which consists of six stages: encoding, granulation, learning, prediction, degranulation and decoding. In this framework, we extract three-dimensional information data from univariate time series data for encoding. It is granulated by the fuzzy C-means clustering algorithm. Then the FCM is constructed at the granular level. In the learning stage, we propose a model based on the adaptive loss function to learn FCM which can be solved by alternating direction multiplier method and quadratic programming method. After learning the FCM, time series prediction is carried out. The degranulation function converts the information granulation back into numerical data. We do some experiments on twelve real datasets, the results demonstrate the effectiveness of the comprehensive framework.
A self-organizing deep network architecture designed based on LSTM network via elitism-driven roulette-wheel selection for time-series forecasting
2024, Knowledge-Based Systems
In this study, we propose a new self-organizing deep network architecture of fuzzy polynomial neural networks (FPNN) based on Fuzzy rule-based Polynomial Neurons (FPNs) and a long short-term memory (LSTM) network to solve the task of time-series forecasting. In the existing regression model based on polynomial neural networks (PNN), it is difficult to achieve high quality performance when predicting time series data, because this model lacks the ability to extract temporal and spatial information. Therefore, we propose a new architecture consisting of one LSTM (temporal) layer and several fuzzy polynomial (spatial) layers to overcome the above-mentioned shortcomings of PNN and enhance its predictive ability to approximate the data. The temporal layer consists of LSTM neurons that have inherently strong modeling capabilities to learn sequential information. The spatial layers are composed of Rule-based Polynomial Neurons (FPNs) that can effectively reflect the complex nonlinear structure found in the input space and granulate it using of the Fuzzy C-Means (FCM) clustering method. An elitism-driven roulette-wheel selection (E_RWS) is used to select appropriate neurons. E_RWS not only ensures that the neuron with the strongest fitting ability is selected but also increases the diversity of candidate neurons. According to the experimental results, the proposed model has a high prediction performance and outperforms many state-of-the-art prediction methods when applied to the real-world time-series.
Back temporal autoregressive matrix factorization for high-dimensional time series prediction
2024, Expert Systems with Applications
While prediction for large-scale multidimensional spatio-temporal data is widely used in applications such as business analytics and open source metrics, service data collected from these applications often have values that are missing from the data. The accuracy of prediction results requires that the prediction model used is highly robust and scalable, something that is typically lacking in traditional prediction models. To this end, we propose a back temporal autoregressive matrix factorization (BTAMF) framework, which can construct an efficient prediction model for time series data with missing values by mining the trend and periodic characteristics of the prediction results. It first uses the target data and its left neighbors to construct trend autoregressive and then uses the target data and its associated periodic data to construct periodic autoregressive to mine the trend and periodicity properties of time series data. We then integrate the tensor/matrix factorization and both autoregressive models into a single optimization model for missing value completion and prediction through optimization of the objective functions. This scheme, which exploits global and local information of the target data, can directly predict the data without pre-completing the missing values, which provides a strong guarantee of the temporal integrity of the time series data. To validate the efficient completion and prediction effects of BTAMF in real-world scenarios, we select ten real-world behavioral datasets from GitHub and seven comparison models for experiments. Our experimental results show that BTAMF has a stronger advantage over current state-of-the-art methods in terms of scalability and prediction accuracy.
Multi-output time series forecasting with randomized multivariate Fuzzy Cognitive Maps
2023, Chaos, Solitons and Fractals
Fuzzy Cognitive Maps (FCMs) have become a relevant technique for modeling and forecasting time series due to their advantages in dealing with uncertainty and simulating the dynamics of complex systems. Although numerous univariate and multivariate FCM-based forecasting models have been presented in the literature, one of the still open questions is how to enable FCMs to forecast multivariate time series for multiple-input, multiple-output (MIMO) systems with an efficient learning mechanism. from a computational point of view. This paper suggests a randomized MIMO FCM-based forecasting approach called MO-RHFCM to predict low-dimensional multivariate time series. More specifically, MO-RHFCM is a hybrid model merging the concepts of multivariate fuzzy time series, high order FCM (HFCM), and Echo State Networks (ESN). The structure of MO-RHFCM consists of three layers: input layer, reservoir (internal) layer, and output layer. Only the output layer is trainable using the Least Squares minimization algorithm; hence training the proposed MO-RHFCM method is fast and simple. The weights inside each sub-reservoir are selected randomly and remain fixed during the training process. The obtained results indicate the efficacy and validity of the proposed MO-RHFCM technique compared with some machine learning and deep learning baseline models.
Randomized high order fuzzy cognitive maps as reservoir computing models: A first introduction and applications
2022, Neurocomputing
Citation Excerpt :
Therefore, evolutionary learning has been replaced with other techniques in some references. For instance, the least squares in [18], the ridge regression in [3,13,19], the Bayesian ridge regression in [20], and the Moore–Penrose inverse in [12]. In spite of the advances in training procedures, they can still be very time-consuming due to the high number of parameters that must be adjusted.
Fuzzy Cognitive Maps (FCMs) have emerged as an interpretable signed weighted digraph method consisting of nodes (concepts) and weights which represent the dependencies among the concepts. Although FCMs have attained considerable achievements in various time series prediction applications, designing an FCM model with time-efficient training method is still an open challenge. Thus, this paper introduces a novel univariate time series forecasting technique, which is composed of a group of randomized high order FCM models labeled R-HFCM. The novelty of the proposed R-HFCM model is relevant to merging the concepts of FCM and Echo State Network (ESN) as an efficient and particular family of Reservoir Computing (RC) models, where the least squares algorithm is applied to train the model. From another perspective, the structure of R-HFCM consists of the input layer, reservoir layer, and output layer in which only the output layer is trainable while the weights of each sub-reservoir components are selected randomly and kept constant during the training process. As case studies, in this paper we consider solar energy forecasting with public data for Brazilian solar stations, hourly electric load of the power supply company of the city of Johor in Malaysia, solar energy dataset from United States National Renewable Energy Laboratory (NREL), electric load data from the Global Energy Forecasting Competition 2012 (GEFCom 2012), and PJM hourly energy consumption data. The experiments also include the effect of the map size, activation function, the number of order and the size of the reservoir on the accuracy of R-HFCM method. The obtained results confirm the outperformance of the proposed R-HFCM model in comparison to the other methods. This study provides evidence that FCM can be a new way to implement a reservoir of dynamics in time series modeling. The Python code of the model is publicly available for research replication.
Wind power forecasting based on variational mode decomposition and high-order fuzzy cognitive maps
2022, Applied Soft Computing
Accurate wind power forecasting can effectively reduce the adverse effects of wind power forecasting errors on wind power grid integration and power dispatch. However, current wind power forecasting technology, such as the method based on machine learning, belongs to the black box model and is not solvable. Variational mode decomposition (VMD) is a decomposition technique based on the time–frequency characteristics of the original time series, which has a mathematical theoretical foundation. Besides, fuzzy cognitive map (FCMs) is a kind of soft computing method with strong knowledge representation and reasoning ability. Therefore, to enhance the forecasting accuracy of wind power, in this paper, a novel time series forecasting method based on improved VMD (IVMD) and high-order FCM (HFCM), namely IVMD $_{HFCM}$ is proposed. IVMD can effectively extract the features in the raw time series depending on the time–frequency characteristics of the time series. Then, the subseries obtained by IVMD are modeled and forecasted by HFCM, and the Bayesian ridge regression method is adopted to learn the weight of HFCM. Finally, the differential evolution (DE) algorithm is used to get the optimal hyperparameters of IVMD $_{HFCM}$ . The performance of IVMD $_{HFCM}$ is verified by comparing it with that of state-of-the-art methods on ten publicly available datasets. Moreover, the proposed IVMD $_{HFCM}$ is compared with the existing HFCM based method on ten actual wind power datasets. The results show that the IVMD $_{HFCM}$ can effectively improve the accuracy of wind power forecasting and reduce the forecasting error. Besides, the IVMD $_{HFCM}$ can also effectively explore the fluctuations of wind power.

View all citing articles on Scopus

View full text

Time series forecasting based on kernel mapping and high-order fuzzy cognitive maps

Abstract

Introduction

Section snippets

Fuzzy cognitive maps

Related work

Kernel-HFCM

Datasets

Experimental setup

Conclusions

CRediT authorship contribution statement

Declaration of Competing Interest

Acknowledgments

Fuzzy Sets and Systems

Knowl.-Based Syst.

Knowl.-Based Syst.

Knowl.-Based Syst.

Neural Netw.

Fuzzy Sets and Systems

Knowl.-Based Syst.

Expert Syst. Appl.

Inf. Sci. (N.Y.)

Knowl.-Based Syst.

Knowl.-Based Syst.

A novel hybrid method based on fuzzy cognitive maps and fuzzy clustering algorithms for grading celiac disease

Neural Comput. Appl.

Assessment of workplace accident risks in underground collieries by integrating a multi-goal cause-and-effect analysis method with MCDM sensitivity analysis

Stoch. Environ. Res. Risk Assess.

Decision-making process for photovoltaic solar energy sector development using fuzzy cognitive map technique

Energies

Time series prediction using sparse autoencoder and high-order fuzzy cognitive maps

IEEE Trans. Fuzzy Syst.

A decision system using fuzzy cognitive map and multi-group data envelopment analysis to estimate hospitals’ outputs level

Neural Comput. Appl.

Numerical and linguistic prediction of time series with the use of fuzzy cognitive maps

IEEE Trans. Fuzzy Syst.

Fuzzy cognitive maps learning using particle swarm optimization

J. Intell. Inf. Syst.