A Variational Bayesian Approach for Identification of Time-Delay Markov Jump Autoregressive Exogenous Systems

Abstract

This paper is concerned with the identification problem of Markov jump autoregressive exogenous systems with unknown slow-varying time-delay. With the implementation of variational Bayesian inference approach, the distribution characteristics of local model parameters are obtained. Meanwhile, the transition probability matrix and the unknown time-delay are determined. Furthermore, a numerical example and a simulated continuous fermentation reactor example are performed to verify the effectiveness of the proposed algorithm.

Appendices

A Appendix

The optimization problem expressed in Eq. (9) is solved in this section. The lower bound of the evidence can be derived as

$$\begin{aligned} \begin{aligned} F\left[ {q\left( {I,\tau } \right) ,q\left( \varTheta \right) } \right]&= \int {q\left( {I,\tau } \right) q\left( \varTheta \right) \log \frac{{p\left( {Y,U,I,\tau ,\varTheta } \right) }}{{q\left( {I,\tau } \right) q\left( \varTheta \right) }}} \mathrm{d}I\mathrm{d}\tau \mathrm{d}\varTheta \\&= \int {q\left( {I,\tau } \right) q\left( \varTheta \right) \log \frac{{p\left( {Y,U,I,\tau |\varTheta } \right) p\left( \varTheta \right) }}{{q\left( {I,\tau } \right) q\left( \varTheta \right) }}} \mathrm{d}I\mathrm{d}\tau \mathrm{d}\varTheta \\&= \int {q\left( {I,\tau } \right) q\left( \varTheta \right) \log \frac{{p\left( {Y,U,I,\tau |\varTheta } \right) }}{{q\left( {I,\tau } \right) }}} \mathrm{d}I\mathrm{d}\tau \mathrm{d}\varTheta \\&\quad + \int {q\left( \varTheta \right) \log \frac{{p\left( \varTheta \right) }}{{q\left( \varTheta \right) }}} \mathrm{d}\varTheta . \end{aligned} \end{aligned}$$

(37)

Then, a Lagrange multiplier $\lambda _1$ is introduced as

$$\begin{aligned} \frac{\partial }{{\partial q\left( {I,\tau } \right) }}\left\{ {F\left[ {q\left( {I,\tau } \right) ,q\left( \varTheta \right) } \right] + \lambda _1 \left( {\int {q\left( {I,\tau } \right) \mathrm{d}I\mathrm{d}\tau } - 1} \right) } \right\} = 0. \end{aligned}$$

(38)

Employing the variational derivative, the above Lagrange function is derived as follows:

$$\begin{aligned} \begin{aligned} {\left\langle {\log p\left( {Y,U,I,\tau |\varTheta } \right) } \right\rangle _{q\left( \varTheta \right) }} - \log q\left( {I,\tau } \right) - 1 + \lambda _1 = 0. \end{aligned} \end{aligned}$$

(39)

Solving Eq. (39), the expression of $q\left( {I,\tau } \right)$ can be obtained as

$$\begin{aligned} q\left( {I,\tau } \right) = {e^{\lambda _1 - 1}}\exp \left[ {{{\left\langle {\log p\left( {Y,U,I,\tau |\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }}} \right] . \end{aligned}$$

(40)

When Eq. (40) is substituted into the constraint $\int \limits {q\left( {I,\tau } \right) \mathrm{d}I\mathrm{d}\tau } = 1$, the term relating to the Lagrange multiplier $\lambda _1$ is obtained as

$$\begin{aligned} {e^{\lambda _1 - 1}} = {\left\{ {\int \limits {\exp \left[ {{{\left\langle {\log p\left( {Y,U,I,\tau |\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }}} \right] } \mathrm{d}I\mathrm{d}\tau } \right\} ^{ - 1}}. \end{aligned}$$

(41)

Finally, the expression of the joint variational posterior of the latent variable I and the time-delay $\tau$ is obtained as

$$\begin{aligned} q\left( {I,\tau } \right) = \frac{{\exp \left[ {{{\left\langle {\log p\left( {Y,U,I,\tau |\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }}} \right] }}{{\int \limits {\exp \left[ {{{\left\langle {\log p\left( {Y,U,I,\tau |\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }}} \right] } \mathrm{d}I\mathrm{d}\tau }}. \end{aligned}$$

(42)

B Appendix

The derivation of Eq. (11) is detailed here. Considering the law of total probability, the joint variational posterior of mode identity $I_k$ and $I_{k-1}$ is expressed as

$$\begin{aligned} \begin{aligned}&q\left( {{I_k} = i,{I_{k - 1}} = j} \right) = \int \limits _{{I_{1:k - 2}}} {q\left( {{I_{1:k - 2}},{I_k} = i,{I_{k - 1}} = j} \right) } \mathrm{d}{I_{1:k - 2}}. \end{aligned} \end{aligned}$$

(43)

Then, Eq. (10) is substituted into the above equation, in which only the instants 1 to k are taken into consideration.

$$\begin{aligned} \begin{aligned}&q\left( {{I_k} = i,{I_{k - 1}} = j} \right) \\&\quad = \frac{{\int \limits _{{I_{1:k - 2}}} {\int \limits _{{\tau _{1:k}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k}},{u_{1:k}},{I_{1:k - 2}},{I_k} = i,{I_{k - 1}} = j,{\tau _{1:k}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] } } \mathrm{d}{\tau _{1:k}}\mathrm{d}{I_{1:k - 2}}}}{{\int \limits _{{I_{1:k}}} {\int \limits _{{\tau _{1:k}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k}},{u_{1:k}},{I_{1:k}},{\tau _{1:k}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] } } \mathrm{d}{\tau _{1:k}}\mathrm{d}{I_{1:k}}}}. \end{aligned} \end{aligned}$$

(44)

The numerator of Eq. (44) can be decomposed into the following expression.

$$\begin{aligned}&\int \limits _{{I_{1:k - 2}}} {\int \limits _{{\tau _{1:k}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k}},{u_{1:k}},{I_{1:k - 2}},{I_k} = i,{I_{k - 1}} = j,{\tau _{1:k}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] } } \mathrm{d}{\tau _{1:k}}\mathrm{d}{I_{1:k - 2}}\nonumber \\&\quad = \int \limits _{{\tau _k}} {\exp } \big [ {\left\langle {\log p\left( {{y_k}|{x_k},{I_k} = i,{\tau _k},\varTheta } \right) p\left( {{\tau _k}|{I_k} {=} i} \right) p\left( {{I_k} {=} i|{I_{k - 1}} {=} j} \right) K_2} \right\rangle _{q\left( \varTheta \right) }} \big ] \nonumber \\&\qquad \quad \bigg \{ \int \limits _{{\tau _{1:k - 1}}} \int \limits _{{I_{1:k - 2}}} \exp \big [ \left\langle \log p\left( {y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}} \right. \right. \nonumber \\&\left. \left. \qquad \qquad = j,{\tau _{1:k - 1}}|\varTheta \right) \right\rangle _{q\left( \varTheta \right) } \big ] \mathrm{d}{I_{1:k - 2}}\mathrm{d}{\tau _{1:k - 1}} \bigg \} \mathrm{d}{\tau _k}, \end{aligned}$$

(45)

where $K_2 = p\left( {{u_k}|{y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}} = j,\tau _{1:{k-1}},\varTheta } \right)$ is a constant that unrelated to the integral. The denominator of Eq. (44) has a similar expression as

$$\begin{aligned}&\int \limits _{{I_{1:k}}} {\int \limits _{{\tau _{1:k}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k}},{u_{1:k}},{I_{1:k}},{\tau _{1:k}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] } } \mathrm{d}{\tau _{1:k}}\mathrm{d}{I_{1:k}}\nonumber \\&\quad = \int \limits _{{I_k}} \int \limits _{{I_{k - 1}}} \int \limits _{{\tau _k}} \exp \big [ \left\langle \log p\left( {y_k}|{x_k},{I_k} \right. \right. \nonumber \\&\left. \left. \quad = m,{\tau _k},\varTheta \right) p\left( {{\tau _k}|{I_k} = m} \right) p\left( {{I_k} = m|{I_{k - 1}} = n} \right) K_2 \right\rangle _{q\left( \varTheta \right) } \big ] \nonumber \\&\qquad \quad \bigg \{ \int \limits _{{\tau _{1:k - 1}}} \int \limits _{{I_{1:k - 2}}} \exp \big [ \left\langle \log p\left( {y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}}\right. \right. \nonumber \\&\left. \left. \qquad \quad = n,{\tau _{1:k - 1}}|\varTheta \right) \right\rangle _{q\left( \varTheta \right) } \big ] \mathrm{d}{I_{1:k - 2}}\mathrm{d}{\tau _{1:k - 1}} \bigg \} \mathrm{d}{\tau _k}\mathrm{d}{I_k}\mathrm{d}{I_{k - 1}}. \end{aligned}$$

(46)

Then, $\int \limits _{{\tau _{1:k - 1}}} {\int \limits _{{I_{1:k - 2}}} {\exp {{\left\langle {\log p\left( {{y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}} = j,{\tau _{1:k - 1}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }}} } \mathrm{d}{I_{1:k - 2}}\mathrm{d}{\tau _{1:k - 1}}$, which appears both in the numerator and denominator of Eq. (44), is simplified in the following context.

When only the instants 1 to $k - 1$ are taken into consideration for Eq. (10), the variational posterior $q\left( {{I_{1:k - 2}},{I_{k - 1}} = j} \right)$ can be expressed as

$$\begin{aligned} \begin{aligned}&q\left( {{I_{1:k - 2}},{I_{k - 1}} = j} \right) \\&\quad = \frac{{\int \limits _{{\tau _{1:k - 1}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}} = j,{\tau _{1:k - 1}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ]} \mathrm{d}{\tau _{1:k - 1}}}}{{\int \limits _{{I_{1:k - 1}}} {\int \limits _{{\tau _{1:k - 1}}} {\exp } \big [ {{\left\langle {\log p\left( {{y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}},{\tau _{1:k - 1}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] \mathrm{d}{\tau _{1:k - 1}}\mathrm{d}{I_{1:k - 1}}} }}. \end{aligned} \end{aligned}$$

(47)

An integral of the variable ${I_{1:k - 2}}$ is performed on the above equation, then,

$$\begin{aligned} \begin{aligned}&q\left( {{I_{k - 1}} = j} \right) \\&\quad = \int \limits _{{I_{1:k - 2}}} {q\left( {{I_{1:k - 2}},{I_{k - 1}} = j} \right) } \mathrm{d}{I_{1:k - 2}}\\&\quad = \frac{{\int \limits _{{I_{1:k - 2}}} {\int \limits _{{\tau _{1:k - 1}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}} = j,{\tau _{1:k - 1}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] } \mathrm{d}{\tau _{1:k - 1}}} \mathrm{d}{I_{1:k - 2}}}}{{\int \limits _{{I_{1:k - 1}}} {\int \limits _{{\tau _{1:k - 1}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}},{\tau _{1:k - 1}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }}\big ] } \mathrm{d}{\tau _{1:k - 1}}\mathrm{d}{I_{1:k - 1}}} }}. \end{aligned} \end{aligned}$$

(48)

Equation (48) can be rearranged as follows:

$$\begin{aligned} \begin{aligned}&\int \limits _{{I_{1:k - 2}}} \int \limits _{{\tau _{1:k - 1}}} \exp \big [ \left\langle \log p\left( {y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 2}},{I_{k - 1}} \right. \right. \\&\left. \left. \quad = j,{\tau _{1:k - 1}}|\varTheta \right) \right\rangle _{q\left( \varTheta \right) } \big ] \mathrm{d}{\tau _{1:k - 1}} \mathrm{d}{I_{1:k - 2}}\\&\quad = q\left( {{I_{k - 1}} = j} \right) {K_3}, \end{aligned} \end{aligned}$$

(49)

where $K_3=\int \limits _{{\tau _{1:k - 1}}} {\int \limits _{{I_{1:k - 1}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k - 1}},{u_{1:k - 1}},{I_{1:k - 1}},{\tau _{1:k - 1}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }}\big ] \mathrm{d}}}{{{I_{1:k - 1}}\mathrm{d}{\tau _{1:k - 1}}} }$ is a normalization term that unrelated to the integrals of ${\tau _k}$, ${I_k}$, and ${I_{k-1}}$. Therefore, a common factor can be removed from the numerator and denominator of Eq. (44). Finally, the joint variational posterior of ${I_k}$ and ${I_{k-1}}$ can be simplified as

$$\begin{aligned} \begin{aligned}&q\left( {{I_k} = i,{I_{k - 1}} = j} \right) \\&\quad = \frac{{\int \limits _{{\tau _k}} {\exp } \big [ {{\left\langle {\log p\left( {{y_k}|{x_k},{I_k} = i,{\tau _k},\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] }}{{\int \limits _{{I_k}} {\int \limits _{{I_{k - 1}}} {\int \limits _{{\tau _k}} {\exp \big [ {{\left\langle {\log p\left( {{y_k}|{x_k},{I_k} = m,{\tau _k},\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }}\big ] } } } }}\\&\qquad \frac{{p\left( {{\tau _k}|{I_k} = i} \right) p\left( {{I_k} = i|{I_{k - 1}} = j} \right) q\left( {{I_{k - 1}} = j} \right) \mathrm{d}{\tau _k}}}{{p\left( {{\tau _k}|{I_k} = m} \right) p\left( {{I_k} = m|{I_{k - 1}} = n} \right) q\left( {{I_{k - 1}} = n} \right) \mathrm {d}{I_k}\mathrm {d}{I_{k - 1}}\mathrm {d}{\tau _k}}}. \end{aligned} \end{aligned}$$

(50)

When the integrals in Eq. (50) are transformed into summations, Eq. (11) is obtained.

C Appendix

The derivation of Eq. (14) is presented in this section. According to the property of conditional probability, the variational posterior distribution of the time-delay $\tau _k$ conditional on the mode identity $I_k$ can be derived as

$$\begin{aligned} \begin{aligned} q\left( {{\tau _k} = d|{I_k} = i} \right)&= \frac{{q\left( {{\tau _k} = d,{I_k} = i} \right) }}{{q\left( {{I_k} = i} \right) }}\\&= \frac{{\int \limits _{{I_{1:k - 1}}} {\int \limits _{{\tau _{1:k - 1}}} {q\left( {{I_{1:k - 1}},{I_k} = i,{\tau _{1:k - 1}},{\tau _k} = d} \right) } \mathrm{d}{\tau _{1:k - 1}}\mathrm{d}{I_{1:k - 1}}} }}{{\int \limits _{{I_{1:k - 1}}} {\int \limits _{{\tau _{1:k}}} {q\left( {{I_{1:k - 1}},{I_k} = i,{\tau _{1:k}}} \right) } \mathrm{d}{\tau _{1:k}}\mathrm{d}{I_{1:k - 1}}} }}. \end{aligned} \end{aligned}$$

(51)

Taking advantage of the expression of the joint posterior distribution in Eq. (10), the conditional variational posterior distribution of the time-delay can be further derived as

$$\begin{aligned} \begin{aligned}&q\left( {{\tau _k} = d|{I_k} = i} \right) \\&\quad = \frac{{\int \limits _{{I_{1:k - 1}}} {\int \limits _{{\tau _{1:k - 1}}} {\exp \big [ {{\left\langle {\log p\left( {{y_{1:k}},{u_{1:k}},{I_{1:k - 1}},{I_k} = i,{\tau _{1:k - 1}},{\tau _k} = d|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] } } \mathrm{d}{\tau _{1:k - 1}}\mathrm{d}{I_{1:k - 1}}}}{{\int \limits _{{I_{1:k - 1}}} {\int \limits _{{\tau _{1:k}}} {\exp } \big [ {{\left\langle {\log p\left( {{y_{1:k}},{u_{1:k}},{I_{1:k - 1}},{I_k} = i,{\tau _{1:k}}|\varTheta } \right) } \right\rangle }_{q\left( \varTheta \right) }} \big ] } \mathrm{d}{\tau _{1:k}}\mathrm{d}{I_{1:k - 1}}}}\\&\quad = \frac{{\exp {{\left\langle {\log \left[ {p\left( {{y_k}|{x_k},{I_k} = i,{\tau _k} = d,\varTheta } \right) p\left( {{\tau _k} = d|{I_k} = i} \right) } \right] } \right\rangle }_{q\left( \varTheta \right) }}}}{{\int \limits _{{\tau _k}} {\exp {{\left\langle {\log \left[ {p\left( {{y_k}|{x_k},{I_k} = i,{\tau _k},\varTheta } \right) p\left( {{\tau _k}|{I_k} = i} \right) } \right] } \right\rangle }_{q\left( \varTheta \right) }}} \mathrm{d}{\tau _k}}}. \end{aligned} \end{aligned}$$

(52)

Then, the above equation can be expressed into a summation form as Eq. (14).

D Appendix

Details of the derivation of Eq. (16) are provided in this section. The Lagrange multiplier $\lambda _2$ is adopted with respect to the lower bound of the evidence of Eq. (8) as

$$\begin{aligned} \frac{\partial }{{\partial q\left( \varTheta \right) }}\left\{ {F\left[ {q\left( {I,\tau } \right) ,q\left( \varTheta \right) } \right] + \lambda _2 \left( {\int \limits {q\left( \varTheta \right) } \mathrm{d}\varTheta - 1} \right) } \right\} = 0. \end{aligned}$$

(53)

The variational derivative is then implemented, and the above Lagrange function can be derived as follows:

$$\begin{aligned} \begin{aligned} {\left\langle {\log p\left( {Y,U,I,\tau |\varTheta } \right) } \right\rangle _{q\left( {I,\tau } \right) }} + p\left( {\varTheta } \right) - \log q\left( \varTheta \right) - 1 + {\lambda _2} = 0. \end{aligned} \end{aligned}$$

(54)

Employing a similar procedure as Appendix A, the solution of the variational posterior of parameters is obtained as

$$\begin{aligned} q\left( \varTheta \right) = \frac{{p\left( {\varTheta } \right) \exp \left[ {{{\left\langle {\log p\left( {Y,U,I,\tau |\varTheta } \right) } \right\rangle }_{q\left( {I,\tau } \right) }}} \right] }}{{\int \limits {p\left( {\varTheta } \right) \exp \left[ {{{\left\langle {\log p\left( {Y,U,I,\tau |\varTheta } \right) } \right\rangle }_{q\left( {I,\tau } \right) }}} \right] } \mathrm{d}\varTheta }}. \end{aligned}$$

(55)

E Appendix

In this section, the derivation of Eq. (30) is detailed. As no valid prior exists, direct optimization of the lower bound with respect to the transition probability $\alpha _{ij}$, instead of its distribution, is implemented. A partial derivative of Eq. (29) with respect to $\alpha _{ij}$ is solved and assigned to be zero as

$$\begin{aligned} \frac{\partial }{{\partial {\alpha _{ij}}}}L\left( {{\alpha _{ij}}} \right) = \sum \limits _{k = 2}^N {q\left( {{I_k} = i,{I_{k - 1}} = j} \right) } \frac{1}{{{\alpha _{ij}}}} + {\lambda _\alpha } = 0. \end{aligned}$$

(56)

Then, the transition probability $\alpha _{ij}$ can be derived as

$$\begin{aligned} {\alpha _{ij}} = - \frac{1}{{{\lambda _\alpha }}}\sum \limits _{k = 2}^N {q\left( {{I_k} = i,{I_{k - 1}} = j} \right) }. \end{aligned}$$

(57)

When the above equation is substituted into the constraint of the transition probability $\sum \nolimits _{i = 1}^M {{\alpha _{ij}}} = 1$, the Lagrange multiplier $\lambda _{\alpha }$ can be obtained as

$$\begin{aligned} {\lambda _\alpha } = - \sum \limits _{i = 1}^M {\sum \limits _{k = 2}^N {q\left( {{I_k} = i,{I_{k - 1}} = j} \right) } }. \end{aligned}$$

(58)

Then, the solution of the transition probability is derived as

$$\begin{aligned} {\alpha _{ij}} = \frac{{\sum \nolimits _{k = 2}^N {q\left( {{I_k} = i,{I_{k - 1}} = j} \right) } }}{{\sum \nolimits _{i = 1}^M {\sum \nolimits _{k = 2}^N {q\left( {{I_k} = i,{I_{k - 1}} = j} \right) } } }}. \end{aligned}$$

(59)