Abstract
Success of neural networks depends on an important parameter, initialization of weights and bias connections. This paper proposes modified quaternion firefly algorithm (MQFA) for initial optimal weight and bias connection to neural networks. The proposed modified quaternion firefly method is based on updating population, moving fireflies and best solution in quaternion space. The combination of modified quaternion firefly and neural network is developed with the scope of creating an improved balance between premature convergence and stagnation. The performance of the proposed method is tested on two nonlinear discrete time systems, Box–Jenkins time series data and exchange rate prediction of Indian currency. Results of the MQFA with back-propagation neural network (MQFA-BPNN) compared with existing differential evolution-based neural network and opposite differential evolution-based neural network. Results obtain using MQFA-BPNN envisage that this method is effective and provides better identification accuracy. Computational complexity of MQFA-BPNN is deliberated, and validation of proposed method is tested by statistical methods.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adebiyi A, Adewumi A, Ayo CK (2014) Comparison of ARIMA and artificial neural networks model for stock price prediction. J Appl Math 2(1):1–7
Adjrad M, Belouchrani A (2007) Estimation of multi component polynomial phase signals impinging on a multisensory array using state-space modeling. IEEE Trans Signal Process 55(1):32–45
Beligiannis GN, Skarlas LV, Likothanassis SD, Perdikouri KG (2005) Nonlinear model structure identification of complex biomedical data using a genetic programming-based technique. IEEE Trans Instrum Meas 54(6):2184–2190
Bennett C, Stewart RA, Beal CD (2013) ANN-based residential water end used demand forecasting model. Expert Syst Appl 40(4):1014–1023
Box GEP, Jenkins GM (1970) Time series analysis. Forecasting and control. Holden Day, San Francisco
Chen S, Billings SA (1989) Representation of nonlinear systems: the NARMAX model. Int J Control 49:1013–1032
Chen L, Narendra K (2004) Identification and control of nonlinear discrete time system based on its linearization: a unified framework. IEEE Trans Neural Netw 15(3):663–673
Chen M, Ge SS, How BVE (2010) Robust adaptive neural network control for a class of uncertain MIMO nonlinear systems with input nonlinearities. IEEE Trans Neural Netw 21(5):796–812
Chon KH, Cohen RJ (1997) Linear and nonlinear ARMA model parameter estimation using an artificial neural network. IEEE Trans Boimed Eng 44:168–174
Conway JH, Smith DA (2003) On quaternions and octonions: their geometry, arithmetic and symmetry. A.K. Peters, Wellesley
Crepinsek M, Mernik M, Liu SH (2011) Analysis of exploration and exploitation in evolutionary algorithm by ancestry trees. Int J Innov Comput Appl 3:11–19
Eberly D (2002) Quaternion algebra and calculus. Magic Software Inc. http://www.magic-software.com
Fister I, Yang XS, Brest J, Brest I Jr (2013) Modified firefly algorithm using quaternion representation. Expert Syst Appl 40:7220–7230
Girard PR (1984) The quaternion group and modern physics. Eur J Phys 5:25–32
Gurusen E, Kayakulu G, Daim T (2011) Using artificial neural network model in stock market index prediction. Expert Syst Appl 38(8):10389–10397
Harnandez E, Arkun Y (1993) Control of nonlinear systems using polynomial ARMA models. AICHE J 39(3):446–460
Hosseini HG, Luo D, Reynolds KJ (2006) The comparison of different feedforward neural network architecture of ECG signal diagnosis. Med Eng Phys 28(4):372–378
Irani R, Nasimi R (2011) Evolving neural network using real coded genetic algorithm for permeability estimation of reservoir. Expert Syst Appl 38(8):9862–9866
Kwon YK, Moon BR (2007) A hybrid neurogenetic approach for stock forecasting. IEEE Trans Neural Netw 18(3):851–864
Lee TT, Jeng JT (1998) The Chebyshev polynomial based unified model neural network for functional approximation. IEEE Trans Syst Man Cybern B 28:925–935
Li YM, Tong SC, Liu Y, Li T (2014) Adaptive fuzzy robust output feedback control of nonlinear systems with unknown dead zones based on a small-gain approach. IEEE Trans Fuzzy Syst 22(1):164–176
Narendra KS, Parthaasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1:4–27
Sadegh N (1993) A perceptron based neural network for identification and control of nonlinear systems. IEEE Trans Neural Netw 4:982–988
Sakhre V, Singh UP, Jain S (2016) FCPN approach for uncertain nonlinear dynamical system with unknown disturbance. Int J Fuzzy Syst 18:1–18. doi:10.1007/s40815-016-0145-5
Schetzmen M (1980) The voltera and winner theories on nonlinear systems. Wiley, New York
Simon Haykin S (2009) Neural networks and learning machines, vol 3. Pearson Education, Upper Saddle River. http://www.mif.vu.lt/~valdas/DNT/Literatura/Haykin09/Haykin09.pdf
Subudhi B, Jena D (2011) A differential evolution based neural network approach to nonlinear system identification. Appl Soft Comput 11:861–871
Theofilatos K, Beligiannis G, Likothanassis S (2009) Combining evolutionary and stochastic gradient techniques for system identification. J. Comput. Appl. Math. 227(1):147–160
Ticknor JL (2013) A Bayesian regularized artificial network for stock market forecasting. Expert Syst Appl 40(14):5501–5506
Tong SC, Li YM, Zhang HG (2011) Adaptive neural network decentralized backstepping output-feedback control for nonlinear large scale systems with time delays. IEEE Trans Neural Netw 22(7):1073–1086
Wang J, Zeng YR, Zhang JL, Huang W, Bao YK (2006) The criticality of spare parts evaluating model using an artificial neural network approach. Lect Notes Comput Sci 3991:728–735
Wang L, Zeng Y, Chen T (2015) Backpropagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Syst Appl 42:855–863
Watanbe K, Matsuura I, Abe M, Kebota M, Himelblau DM (1989) Incipient fault diagnosis of chemical processes via artificial neural networks. AICHE J 35(11):1803–1812
Xie Y, Guo B, Xu L, Li J, Stoica P (2006) Multistatic adaptive microwave imaging for early breast cancer detection. IEEE Trans Boimed Eng 53(8):1647–1657
Yam JY, Chow TWS (2000) A weight initialized method for improving training speed in feedforward neural network. Neurocomputing 30(1):219–232
Yang XS (2010) Engineering optimization: an introduction with metaheuristic applications. Wiley, Hoboken
Yang XS (2014) Nature-inspired optimization algorithms, Elsevier Insights Series, 1st edn. pp 111–123
Zhang L, Subbarayan G (2002) An evaluation of back-propagation neural network for the optimal design of structural systems: part I training procedures. Comput Methods Appl Mech Eng 191(25):2873–2886
Zhang YQ, Wan X (2007) Statistical fuzzy interval neural network for currency exchange rate time series prediction. Appl Soft Comput 7(4):1149–1156
Zhang GP, Patuwo BE, Hu MY (2001) A simulation study of artificial neural network for nonlinear time series forecasting. Comput Oper Res 28(4):381–396
Zhang JR, Zhang J, Lok TM, Lyu MR (2007) A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. Appl Math Comput 185(2):1026–1037
Zhang Y, Chai T, Wang H (2011) A nonlinear control method based on ANFIS and multiple models for a class of SISO nonlinear systems and its application. IEEE Trans Neural Netw 22(11):1783–1795
Acknowledgements
We would like to thank the editors and anonymous reviewers for their valuable suggestions and constructive comments which will really help us improve presentation and quality of paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by V. Loia.
Appendix A: Quaternion algebra
Appendix A: Quaternion algebra
Quaternions are number systems, which extends complex numbers. Quaternion’s representation’s is used for balance between the exploration and exploitation. In quaternions are number systems each real value element of solution vector is mapped with four dimensional quaternions. In comparison with actual search space, quaternion search space is very vast, but exploration for quaternion space is easier (Conway and Smith 2003; Girard 1984). Quaternion can be represented by \(q= x_0 + x_1 i + x_2 j + x_3 k\) where \(x_0 , x_1 , x_2 , x_3\) are real numbers. Quaternion algebra is based on Hamilton equations:
Let \(q_0 = x_0 + x_1 i + x_2 j + x_3 k\) and \(q_1 = y_0 + y_1 i + y_2 j + y_3 k\) be any two quaternions. Quaternion algebra defines the following operations on quaternion’s (Eberly 2002):
-
Addition: of quaternions is defined by
$$\begin{aligned}&q_0 + q_1 = \left( {x_0 + x_1 i + x_2 j + x_3 k} \right) \nonumber \\&\qquad + \left( {y_0 + y_1 i + y_2 j + y_3 k} \right) \nonumber \\&\quad = (x_0 + y_0 ) + (x_1 + y_1 ) i+ (x_2 + y_2 ) j\nonumber \\&\qquad + (x_3 + y_3 )\hbox {k} \end{aligned}$$(51) -
Subtraction: of quaternions is defined by
$$\begin{aligned}&q_0 - q_1 = \left( {x_0 + x_1 i + x_2 j + x_3 k} \right) \nonumber \\&\qquad - \left( {y_0 + y_1 i + y_2 j + y_3 k} \right) \nonumber \\&\quad = (x_0 - y_0 ) + (x_1 - y_1 ) i+ (x_2 - y_2 ) j\nonumber \\&\qquad + (x_3 - y_3 )\hbox {k} \end{aligned}$$(52) -
Multiplication: of quaternions is defined by
$$\begin{aligned}&q_0 q_1 = \left( {x_0 {+} x_1 i{+} x_2 j{+} x_3 k} \right) \left( {y_0 + y_1 i + y_2 j + y_3 k} \right) \\&\quad = \left( {x_0 y_0 - x_1 y_1 - x_2 y_2 - x_3 y_3 } \right) + \left( x_0 y_1 + x_1 y_0 \right. \\&\quad \left. + x_2 y_3- x_3 y_2 \right) i\\&\quad + \left( {x_0 y_2 - x_1 y_3 + x_2 y_0 + x_3 y_1 } \right) j + \left( x_0 y_3 + x_1 y_2 \right. \\&\quad \left. - x_2 y_1+\,x_3 y_0 \right) k \end{aligned}$$multiplication of quaternions does not satisfy commutative law, since \(q_0 q_1 \ne q_1 q_0 \).
-
Scalar multiplication:
$$\begin{aligned} \alpha q_0= & {} \alpha \left( {x_0 + x_1 i + x_2 j + x_3 k} \right) \nonumber \\= & {} \left( {\alpha x_0 } \right) + \left( {\alpha x_1 } \right) i + \left( {\alpha x_2 } \right) j + \left( {\alpha x_3 } \right) k \end{aligned}$$(53) -
Conjugate: is unary arithmetical unary operation defined by
$$\begin{aligned} q^{*}= (x_0 + x_1 i + x_2 j + x_3 k)^{*} = x_0 - x_1 i {-} x_2 j {-} x_3 k\nonumber \\ \end{aligned}$$(54)Quaternions satisfy following properties:
$$\begin{aligned} (q^{*})^{*}= q \hbox { and } (q_0 q_1 )^{*}= q_0 ^{*}q_1 ^{*}. \end{aligned}$$ -
Norm of a quaternion is defined by
$$\begin{aligned} \left\| q \right\|= & {} \left\| {x_0 + x_1 i + x_2 j + x_3 k} \right\| \nonumber \\= & {} \sqrt{x_0 ^{2} + x_1 ^{2} + x_2 ^{2} + x_3 ^{2}} \end{aligned}$$(55)The norm is real valued function that satisfies the properties: \(\left\| q \right\| = \left\| {q^{*}} \right\| \) and \(\left\| {q_0 q_1 } \right\| = \left\| {q_0 } \right\| \left\| {q_1 } \right\| \). This function is suitable for mapping of elements of vector from 4-dimensional space to a 1-dimensional space.
-
Multiplicative inverse of a quaternion is denoted by \(q^{-1}\) and has properties \(qq^{-1} = q^{-1}q=1\), \(\left( {q^{-1}} \right) ^{-1} = q\) and \(\left( {q_0 q_1 } \right) ^{-1} = q_1 ^{-1}q_0 ^{-1}\). It is constructed as
$$\begin{aligned} q^{-1}= {q^{*}}/{\left\| q \right\| }. \end{aligned}$$(56) -
Division: of quaternions \(q_0 \) and \(q_1 \)is defined by
$$\begin{aligned} {q_0 }/{q_1 } = q_0 q_1 ^{-1} \end{aligned}$$(57) -
Distance: of quaternions \(q_0 \)and \(q_1 \)is defined by
$$\begin{aligned}&\mathrm{dist}(q_0 ,q_1 )\nonumber \\&\quad = \sqrt{(x_0 ^{2}-y_0 ^{2}) + (x_1 ^{2}-y_1 ^{2}) + (x_2 ^{2}-y_2 ^{2}) {+} (x_3 ^{2}{-}y_3 ^{2}) }\nonumber \\ \end{aligned}$$(58) -
Qrand: unary function of quaternion is defined as
$$\begin{aligned} qrand() = \left\{ {x_i = N(0,1)| \hbox { for }i =0,1,2,3} \right\} \end{aligned}$$(59)where N(0, 1) is normal distribution with mean 0 and standard deviation 1.
-
Qzero: unary function of quaternion is defined as
$$\begin{aligned} qzero() = \left\{ {x_i = 0| \hbox { for i}=0,1,2,3} \right\} \end{aligned}$$(60)
i.e., each component of quaternion is initialized with zero. These quaternion algebraic operations are used in QFA.
Rights and permissions
About this article
Cite this article
Singh, U.P., Jain, S. Optimization of neural network for nonlinear discrete time system using modified quaternion firefly algorithm: case study of Indian currency exchange rate prediction. Soft Comput 22, 2667–2681 (2018). https://doi.org/10.1007/s00500-017-2522-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-017-2522-x