A class of hybrid morphological perceptrons with application in time series forecasting

https://doi.org/10.1016/j.knosys.2011.01.001Get rights and content

Abstract

In this work a class of hybrid morphological perceptrons, called dilation–erosion perceptron (DEP), is presented to overcome the random walk dilemma in the time series forecasting problem. It consists of a convex combination of fundamental operators from mathematical morphology (MM) on complete lattices theory (CLT). A gradient steepest descent method is presented to design the proposed DEP (learning process), using the back propagation (BP) algorithm and a systematic approach to overcome the problem of nondifferentiability of morphological operators. The learning process includes an automatic phase fix procedure that is geared at eliminating time phase distortions observed in some time series. Finally, an experimental analysis is conducted with the proposed DEP using five real world time series, where five well-known performance metrics and an evaluation function are used to assess the forecasting performance of the proposed model. The obtained results are compared with those generated by classical forecasting models presented in the literature.

Introduction

Morphological perceptrons (MPs) [62], [73], [77] represent a particular class of artificial neurons based on the framework of mathematical morphology (MM) [69], [46], in which its algebraic foundations can be found in the complete lattices theory (CLT) [70], [66], [37]. This approach was successfully extended to artificial neural networks (ANNs) [35], [36], and were called morphological neural networks (MNNs) [11], [9], [10], [12], [13], whose neuron computation is based on morphological operators in the context of CLT [70], [66], [37].

In this context, several MNNs have been proposed in the literature, including morphological perceptrons [62], [73], [76], morphological perceptrons with dendrites [63], morphological associative memories [61], [78], [79], [80], [83], morphological-rank neural networks [52], fuzzy lattice neural networks [54], [39], [75], hybrid morphological-rank-linear neural networks [53], modular morphological neural networks [27], [24], [25], [23], [19], morphological shared weight and regularization neural networks [41], [38], amongst other.

These MNNs were successfully applied to a large number of problems, such as pattern recognition [39], [42], automatic target recognition [41], handwritten character recognition [53], landmine detection [29], self-localization and hyperspectral image analysis [57], [32]. Recently, some kinds of MNNs have been developed as solution of the time series forecasting problem [24], [19], [26], [22], [20], [21], [16], [15], [17], [18]. However, a dilemma arises from all these models regarding some time series, known as random walk dilemma [47], [72], where the predictions generated show a characteristic one step delay regarding the real time series data, that is, a time phase distortion in the reconstruction of phase space of the time series phenomenon [14], [28], [20], [21], [16], [15], [17], [18].

In this sense, this work presents a class of hybrid morphological perceptrons to overcome the random walk dilemma in the time series forecasting problem. The proposed model, called dilation–erosion perceptron (DEP), consists of a convex combination of dilation and erosion operators from MM [69], [46] on CLT [70], [66], [37]. Also, a gradient steepest descent method is presented to design the proposed DEP (learning process) based on ideas from the back propagation (BP) algorithm [35], [36] and using a systematic approach to overcome the problem of nondifferentiability of morphological operations, based on ideas from Pessoa and Maragos [53]. An automatic correction step [14], [28], [20], [21], [16], [15], [17], [18] was included into DEP learning process, in the attempt to eliminating time phase distortions that occur in some time series.

Furthermore, an experimental analysis is conducted with the proposed DEP using five complex nonlinear problems of time series forecasting: Dow Jones Industrial Average Index, Standard & Poor 500 Stock Index, National Association of Securities Dealers Automated Quotation Index, Petrobras PN Stock Prices and Sunspot. Also, five well-known performance metrics and an evaluation function are used to assess the performance of the proposed model.

This work is organized as follows. In Section 2 are presented the fundamentals of the time series forecasting problem and the random walk dilemma. Section 3 shows characterization of the time series used in this work. Section 4 presents the fundamentals of morphological neural networks. Section 5 describes the proposed DEP model. Section 6 describes the performance measures used to assess the proposed DEP model. In Section 7 are presented simulations and experimental results with the proposed DEP model, as well as a comparison between the obtained results and those given by classical forecasting models presented in the literature. Finally, in Section 8, it is presented the conclusions of this work.

Section snippets

The time series forecasting

A time series is a sequence of observations about a given phenomenon, where it is observed in a discrete or continuous space. In this work all time series will be considered time discrete and equidistant.

Commonly, a time series can be defined byx=xtR|t=1,2,,N,where t is the temporal index and N is the number of observations. The term x is the set of temporal observations equally spaced and temporally ordered by a chronological index t, which is called time and defines the granularity of

Analyzed time series

It is necessary to define a set of relevant time series to build a benchmark for the comparative analysis with the investigated forecasting models. Therefore, it uses important and relevant time series commonly investigated in the literature [14], [28], [20], [21], [16], [15], [17], [18]: Dow Jones Industrial Average Index, Standard & Poor 500 Stock Index, National Association of Securities Dealers Automated Quotation Index, Petrobras PN Stock Prices and Sunspot. In these series it was found

Background Information on morphological neural networks

According to Ritter [65], [64], the image algebra represents a mathematical theory of analysis and transformation of images, which can be used to the development of conventional artificial neural networks (ANNs) [59]. From this observations initiated the development of the so-called morphological neural networks (MNNs) [11], [8], [13], [60]. Note that image algebra represents an heterogeneous algebra, according to Birkhoff and Lipson [3], which includes linear and minimax algebra as

The proposed dilation–erosion perceptron (DEP)

This section presents a class of hybrid morphological perceptrons to solve the time series forecasting problem. The proposed model, called dilation–erosion perceptron (DEP), consists of a convex combination of dilation and erosion operators from mathematical morphology on complete lattice theory.

Performance evaluation

Many performance evaluation criteria are found in literature. However, most of the existing literature on time series forecasting frequently employ only one performance criterion for forecasting evaluation. The most widely used performance criterion is the mean squared error (MSE), given byMSE=1Nj=1Nxj-x^j2,where N is the number of patterns, xj is the desired output for pattern j and xˆj is the predicted value for pattern j.

The MSE measure may be used to drive the forecasting model in the

Simulations and experimental results

A set of five real world time series (Dow Jones Industrial Average Index, Standard & Poor 500 Stock Index, National Association of Securities Dealers Automated Quotation Index, Petrobras PN Stock Prices and Sunspot) was used as a test bed for evaluation of the proposed model. All time series were normalized to lie within the range [0, 1] and divided in three sets according to Prechelt [56]: training set (50% of the data points), validation set (25% of the data points) and test set (25% of the

Conclusion

In this paper a class of hybrid morphological perceptrons was presented for dealing with time series forecasting problems. The proposed model, referred to as dilation–erosion perceptron (DEP), consists of a convex combination of fundamental operators from mathematical morphology on complete lattices theory. Also, it was presented a gradient steepest descent method to design the proposed DEP (learning process) based on ideas from the back propagation algorithm and using a systematic approach to

References (84)

  • G.X. Ritter et al.

    Image algebra: an overview

    Computer Vision, Graphics, and Image Processing

    (1990)
  • C. Ronse

    Why mathematical morphology needs complete lattices

    Signal Processing

    (1990)
  • R. Savit et al.

    Time series and dependent variables

    Physica D

    (1991)
  • N. Tanaka et al.

    Estimating the active dimension of the dynamics in a time series based on a information criterion

    Physica D

    (2001)
  • M.E. Valle et al.

    A general framework for fuzzy morphological associative memories

    Fuzzy Sets and Systems

    (2008)
  • G.P. Zhang

    A neural network ensemble method with jittered training data for time series forecasting

    Information Sciences

    (2007)
  • G. Birkhoff

    Lattice Theory

    (1993)
  • G.E.P. Box et al.

    Time Series Analysis: Forecasting and Control

    (1994)
  • O. Castillo et al.

    Hybrid intelligent systems for time series prediction using neural networks, fuzzy logic, and fractal theory

    IEEE Transactions on Neural Networks

    (2002)
  • M.P. Clements et al.

    On the limitations of comparing mean square forecast errors

    Journal of Forecasting

    (1993)
  • R. Cuninghame-Green

    Minimax Algebra

  • J.L. Davidson, Template learning in morphological neural nets, in: Image Algebra and Morphological Image Processing II,...
  • J.L. Davidson, Simulated annealing and morphological neural networks, in: Proceedings of SPIE, vol. 1769, 1992, pp....
  • J.L. Davidson et al.

    Morphology neural networks: an introduction with applications

    IEEE Transactions on Signal Processing

    (1993)
  • J.L. Davidson, G.X. Ritter, A theory of morphological neural networks, in: Proceedings of the SPIE, vol. 1215, 1990,...
  • J.L. Davidson, R. Srivastava, Fuzzy image algebra neural network for template identification, in: AMES, IA, 1993, pp....
  • J.L. Davidson, A. Talukder, Template identification using simulated annealing in morphology neural networks, in:...
  • R. de A. Araújo

    Swarm-based hybrid intelligent forecasting method for financial time series prediction

    Learning and Nonlinear Models

    (2007)
  • R. de A. Araújo

    Hybrid intelligent methodology to design translation invariant morphological operators for Brazilian stock market prediction

    Neural Networks

    (2010)
  • R. de A. Araújo

    A hybrid intelligent morphological approach for stock market forecasting

    Neural Processing Letters

    (2010)
  • R. de A. Araújo

    Translation invariant morphological time-lag added evolutionary forecasting method for stock market prediction

    Expert Systems with Applications

    (2011)
  • R. de A. Araújo et al.

    An intelligent hybrid approach for designing increasing translation invariant morphological operators for time series forecasting

  • R. de A. Araújo, T.A.E. Ferreira, A morphological-rank-linear evolutionary method for stock market prediction,...
  • R. de A. Araújo et al.

    Morphological-rank-linear time-lag added evolutionary forecasting method for financial time series forecasting

  • R. de A. Araújo, F. Madeiro, R.P. de Sousa, L.F.C. Pessoa, Modular morphological neural network training via adaptive...
  • R. de A. Araújo, F. Madeiro, R.P. de Sousa, L.F.C. Pessoa, T.A.E. Ferreira, An evolutionary morphological approach for...
  • R. de A. Araújo, F. Madeiro, T.A.F. Ferreira, R.P. de Sousa, L.F.C. Pessoa, Improved evolutionary hybrid method for...
  • R. de A. Araújo et al.

    An evolutionary morphological-rank-linear approach for time series prediction

  • R.P. de Sousa, J.M. Carvalho, F.M. Assis, L.F.C. Pessoa, Designing translation invariant operations via neural network...
  • T.A.E. Ferreira et al.

    A new intelligent system methodology for time series forecasting with artificial neural networks

    Neural Processing Letters

    (2008)
  • P.D. Gader et al.

    Morphological regularization neural networks

    Pattern Recognition

    (2000)
  • T.V. Gestel et al.

    Financial time series prediction using least squares support vector machines within the evidence framework

    IEEE Transactions on Neural Networks

    (2001)
  • Cited by (0)

    View full text