Optimization of neural network for nonlinear discrete time system using modified quaternion firefly algorithm: case study of Indian currency exchange rate prediction

Abstract

Success of neural networks depends on an important parameter, initialization of weights and bias connections. This paper proposes modified quaternion firefly algorithm (MQFA) for initial optimal weight and bias connection to neural networks. The proposed modified quaternion firefly method is based on updating population, moving fireflies and best solution in quaternion space. The combination of modified quaternion firefly and neural network is developed with the scope of creating an improved balance between premature convergence and stagnation. The performance of the proposed method is tested on two nonlinear discrete time systems, Box–Jenkins time series data and exchange rate prediction of Indian currency. Results of the MQFA with back-propagation neural network (MQFA-BPNN) compared with existing differential evolution-based neural network and opposite differential evolution-based neural network. Results obtain using MQFA-BPNN envisage that this method is effective and provides better identification accuracy. Computational complexity of MQFA-BPNN is deliberated, and validation of proposed method is tested by statistical methods.

Appendix A: Quaternion algebra

Quaternions are number systems, which extends complex numbers. Quaternion’s representation’s is used for balance between the exploration and exploitation. In quaternions are number systems each real value element of solution vector is mapped with four dimensional quaternions. In comparison with actual search space, quaternion search space is very vast, but exploration for quaternion space is easier (Conway and Smith 2003; Girard 1984). Quaternion can be represented by $q= x_0 + x_1 i + x_2 j + x_3 k$ where $x_0 , x_1 , x_2 , x_3$ are real numbers. Quaternion algebra is based on Hamilton equations:

$$\begin{aligned} ij= & {} k, jk=i, ki=j, \quad ji=-k, kj=-i, \nonumber \\ ik= & {} -j \hbox { and } i^{2}= j^{2} = k^{2} = 1 \end{aligned}$$

(50)

Let $q_0 = x_0 + x_1 i + x_2 j + x_3 k$ and $q_1 = y_0 + y_1 i + y_2 j + y_3 k$ be any two quaternions. Quaternion algebra defines the following operations on quaternion’s (Eberly 2002):

• Addition: of quaternions is defined by

  $$\begin{aligned}&q_0 + q_1 = \left( {x_0 + x_1 i + x_2 j + x_3 k} \right) \nonumber \\&\qquad + \left( {y_0 + y_1 i + y_2 j + y_3 k} \right) \nonumber \\&\quad = (x_0 + y_0 ) + (x_1 + y_1 ) i+ (x_2 + y_2 ) j\nonumber \\&\qquad + (x_3 + y_3 )\hbox {k} \end{aligned}$$

  (51)

• Subtraction: of quaternions is defined by

  $$\begin{aligned}&q_0 - q_1 = \left( {x_0 + x_1 i + x_2 j + x_3 k} \right) \nonumber \\&\qquad - \left( {y_0 + y_1 i + y_2 j + y_3 k} \right) \nonumber \\&\quad = (x_0 - y_0 ) + (x_1 - y_1 ) i+ (x_2 - y_2 ) j\nonumber \\&\qquad + (x_3 - y_3 )\hbox {k} \end{aligned}$$

  (52)

• Multiplication: of quaternions is defined by

  $$\begin{aligned}&q_0 q_1 = \left( {x_0 {+} x_1 i{+} x_2 j{+} x_3 k} \right) \left( {y_0 + y_1 i + y_2 j + y_3 k} \right) \\&\quad = \left( {x_0 y_0 - x_1 y_1 - x_2 y_2 - x_3 y_3 } \right) + \left( x_0 y_1 + x_1 y_0 \right. \\&\quad \left. + x_2 y_3- x_3 y_2 \right) i\\&\quad + \left( {x_0 y_2 - x_1 y_3 + x_2 y_0 + x_3 y_1 } \right) j + \left( x_0 y_3 + x_1 y_2 \right. \\&\quad \left. - x_2 y_1+\,x_3 y_0 \right) k \end{aligned}$$

  multiplication of quaternions does not satisfy commutative law, since $q_0 q_1 \ne q_1 q_0$.

• Scalar multiplication:

  $$\begin{aligned} \alpha q_0= & {} \alpha \left( {x_0 + x_1 i + x_2 j + x_3 k} \right) \nonumber \\= & {} \left( {\alpha x_0 } \right) + \left( {\alpha x_1 } \right) i + \left( {\alpha x_2 } \right) j + \left( {\alpha x_3 } \right) k \end{aligned}$$

  (53)

• Conjugate: is unary arithmetical unary operation defined by

  $$\begin{aligned} q^{*}= (x_0 + x_1 i + x_2 j + x_3 k)^{*} = x_0 - x_1 i {-} x_2 j {-} x_3 k\nonumber \\ \end{aligned}$$

  (54)

  Quaternions satisfy following properties:

  $$\begin{aligned} (q^{*})^{*}= q \hbox { and } (q_0 q_1 )^{*}= q_0 ^{*}q_1 ^{*}. \end{aligned}$$

• Norm of a quaternion is defined by

  $$\begin{aligned} \left\| q \right\|= & {} \left\| {x_0 + x_1 i + x_2 j + x_3 k} \right\| \nonumber \\= & {} \sqrt{x_0 ^{2} + x_1 ^{2} + x_2 ^{2} + x_3 ^{2}} \end{aligned}$$

  (55)

  The norm is real valued function that satisfies the properties: $\left\| q \right\| = \left\| {q^{*}} \right\|$ and $\left\| {q_0 q_1 } \right\| = \left\| {q_0 } \right\| \left\| {q_1 } \right\|$. This function is suitable for mapping of elements of vector from 4-dimensional space to a 1-dimensional space.

• Multiplicative inverse of a quaternion is denoted by $q^{-1}$ and has properties $qq^{-1} = q^{-1}q=1$, $\left( {q^{-1}} \right) ^{-1} = q$ and $\left( {q_0 q_1 } \right) ^{-1} = q_1 ^{-1}q_0 ^{-1}$. It is constructed as

  $$\begin{aligned} q^{-1}= {q^{*}}/{\left\| q \right\| }. \end{aligned}$$

  (56)

• Division: of quaternions $q_0$ and $q_1$is defined by

  $$\begin{aligned} {q_0 }/{q_1 } = q_0 q_1 ^{-1} \end{aligned}$$

  (57)

• Distance: of quaternions $q_0$and $q_1$is defined by

  $$\begin{aligned}&\mathrm{dist}(q_0 ,q_1 )\nonumber \\&\quad = \sqrt{(x_0 ^{2}-y_0 ^{2}) + (x_1 ^{2}-y_1 ^{2}) + (x_2 ^{2}-y_2 ^{2}) {+} (x_3 ^{2}{-}y_3 ^{2}) }\nonumber \\ \end{aligned}$$

  (58)

• Qrand: unary function of quaternion is defined as

  $$\begin{aligned} qrand() = \left\{ {x_i = N(0,1)| \hbox { for }i =0,1,2,3} \right\} \end{aligned}$$

  (59)

  where N(0, 1) is normal distribution with mean 0 and standard deviation 1.

• Qzero: unary function of quaternion is defined as

  $$\begin{aligned} qzero() = \left\{ {x_i = 0| \hbox { for i}=0,1,2,3} \right\} \end{aligned}$$

  (60)

i.e., each component of quaternion is initialized with zero. These quaternion algebraic operations are used in QFA.