Bayesian Learning of Finite Asymmetric Gaussian Mixtures

Abstract

Asymmetric Gaussian mixture (AGM) model has been proven to be more flexible than the classic Gaussian mixture model from many aspects. In contrast with previous efforts that have focused on maximum likelihood estimation, this paper introduces a fully Bayesian learning approach using Metropolis-Hastings (MH) within Gibbs sampling method to learn AGM model. We show the merits of the proposed model using synthetic data and a challenging intrusion detection application.

Appendix A

A.1 Derivation of Acceptance Ratio r by Eq. (12)

The derivation of acceptance ratio r is based on the assumption that mixture parameters are independent from each other which means that:

$$\begin{aligned} \begin{aligned} \pi (\varTheta )&= \pi (p,\xi ) = \pi (\xi ) \\&= \prod _{j=1}^M\pi (\mu _j)\pi (\sigma _{lj})\pi (\sigma _{rj}) \qquad \qquad \qquad \qquad \qquad \\&= \prod _{j=1}^M\mathcal {N}_d(\mu _j|\eta ,\varSigma )\mathcal {N}_d(\sigma _{lj}|\tau ,\varSigma )\mathcal {N}_d(\sigma _{rj}|\tau ,\varSigma )\qquad \qquad \qquad \end{aligned} \end{aligned}$$

(14)

in Eq. (14), since the mixture weigh p is generated following Gibbs sampling method whose acceptance ratio is always 1, it should be excluded from Metropolis-Hastings estimation step. Accordingly, apply the same rule to the proposal distribution as well:

$$\begin{aligned} \begin{aligned}&q(\varTheta ^{(t)}|\varTheta ^{(t-1)}) = q(\xi ^{(t)}|\xi ^{(t-1)}) \\&\quad = \prod _{j=1}^M\mathcal {N}_d(\mu _j^{(t)}|\mu _j^{(t-1)},\varSigma )\mathcal {N}_d(\sigma _{lj}^{(t)}|\sigma _{lj}^{(t-1)},\varSigma )\mathcal {N}_d(\sigma _{rj}^{(t)}|\sigma _{rj}^{(t-1)},\varSigma ) \end{aligned} \end{aligned}$$

(15)

by combining Eqs. (2), (4), (9), (10), (11), (14) and (15), Eq. (12) can be written as follows:

$$\begin{aligned} \begin{aligned}&r = \frac{p(\mathcal {X}|\varTheta ^{(t)})\pi (\varTheta ^{(t)})q(\varTheta ^{(t-1)}|\varTheta ^{(t)})}{p(\mathcal {X}|\varTheta ^{(t-1)})\pi (\varTheta ^{(t-1)})q(\varTheta ^{(t)}|\varTheta ^{(t-1)})} \\&\qquad = \prod _{i=i}^N \prod _{j=1}^M(\frac{p(X_i|\mu _j^{(t)},\sigma _{lj}^{(t)},\sigma _{rj}^{(t)})}{p(X_i|\mu _j^{(t-1)},\sigma _{lj}^{(t-1)},\sigma _{rj}^{(t-1)})} \qquad \qquad \\&\quad \times \frac{\mathcal {N}_d(\mu _j^{(t)}|\eta ,\varSigma )\mathcal {N}_d(\sigma _{lj}^{(t)}|\tau ,\varSigma )\mathcal {N}_d(\sigma _{rj}^{(t)}|\tau ,\varSigma )}{\mathcal {N}_d(\mu _j^{(t-1)}|\eta ,\varSigma )\mathcal {N}_d(\sigma _{lj}^{(t-1)}|\tau ,\varSigma )\mathcal {N}_d(\sigma _{rj}^{(t-1)}|\tau ,\varSigma )} \\&\quad \,\, \times \frac{\mathcal {N}_d(\mu _j^{(t-1)}|\mu _j^{(t)},\varSigma )\mathcal {N}_d(\sigma _{lj}^{(t-1)}|\sigma _{lj}^{(t)},\varSigma )\mathcal {N}_d(\sigma _{rj}^{(t-1)}|\sigma _{rj}^{(t)},\varSigma )}{\mathcal {N}_d(\mu _j^{(t)}|\mu _j^{(t-1)},\varSigma )\mathcal {N}_d(\sigma _{lj}^{(t)}|\sigma _{lj}^{(t-1)},\varSigma )\mathcal {N}_d(\sigma _{rj}^{(t)}|\sigma _{rj}^{(t-1)},\varSigma )} \end{aligned} \end{aligned}$$

(16)