Abstract
Student-\(t\) mixture of Gaussian processes (TMGP) extends the conventional mixture of Gaussian processes (MGP) by using Student-\(t\) mixture as the input distribution instead of Gaussian mixture for robust regression and the un \(\nu\)-Hardcut EM algorithm has already been established for Student-\(t\) mixtures of Gaussian processes. However, this Hardcut EM algorithm takes an approximation of the \(Q\)-function with the maximum a posteriori estimate of the hidden variables in the E-step, and separately optimizes the degrees of freedom of Student-\(t\) distributions and the hyperparameters of the covariance functions in the M-step. In order to get rid of these problems, we propose a variational EM (VEM) algorithm for Student-\(t\) mixtures of Gaussian processes from the viewpoint of variational inference. It is demonstrated by the experimental results on three datasets that our proposed VEM algorithm is effective and outperforms the un \(\nu\)-Hardcut EM algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge (2006)
Tresp, V.: Mixtures of gaussian processes. Adv. Neural. Inf. Process. Syst. 13, 654–660 (2000)
Chen, Z., Ma, J., Zhou, Y.: A precise hard-cut EM algorithm for mixtures of gaussian processes. In: Huang, D.-S., Jo, K.-H., Wang, L. (eds.) ICIC 2014. LNCS (LNAI), vol. 8589, pp. 68–75. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09339-0_7
Luo, C., Sun, S.: Variational mixtures of Gaussian processes for classification. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, vol. 26, pp. 4603–4609 (2017)
Sun, S., Xu, X.: Variational inference for infinite mixtures of Gaussian processes with applications to traffic flow prediction. IEEE Trans. Intell. Transp. Syst. 12(2), 466–475 (2011)
Wu, D., Chen, Z., Ma, J.: An MCMC based EM algorithm for mixtures of gaussian processes. In: Hu, X., Xia, Y., Zhang, Y., Zhao, D. (eds.) ISNN 2015. LNCS, vol. 9377, pp. 327–334. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-25393-0_36
Wu, D., Ma, J.: A two-layer mixture model of Gaussian processes functional regressions and its MCMC EM algorithm. In: IEEE Transactions on Neural Networks and Learning Systems, vol. 29 (2018)
Yang, Y., Ma, J.: An efficient EM approach to parameter learning of the mixture of Gaussian processes. In: Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H. (eds.) ISNN 2011. LNCS, vol. 6676, pp. 165–174. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21090-7_20
Yuan, C., Neubauer, C.: Variational mixture of Gaussian process experts. Adv. Neural. Inf. Process. Syst. 21, 1897–1904 (2009)
Qiang, Z., Ma, J.: Automatic model selection of the mixtures of Gaussian processes for regression. In: Hu, X., Xia, Y., Zhang, Y., Zhao, D. (eds.) ISNN 2015. LNCS, vol. 9377, pp. 335–344. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-25393-0_37
Li, X., Li, T., Ma, J.: The un -hardcut EM algorithm for non-central Student- mixtures of Gaussian processes. In: 15th IEEE International Conference on Signal Processing (ICSP), pp. 289–294 (2020)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)
Gibbs, M., Mackay, D.J.C.: Efficient implementation of Gaussian processes. Technical report, Cavendish Laboratory, Cambridge, UK (1997)
Smola, A.J., Bartlett, P.: Sparse greedy Gaussian process regression. Adv. Neural Inform. Proc. Syst. 13, 619–625 (2001)
Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. Adv. Neural. Inf. Process. Syst. 18, 1257–1264 (2006)
Acknowledgement
This work is supported by the National Key R & D Program of China (2018YFC0808305).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, X., Li, X., Ma, J. (2021). Variational EM Algorithm for Student-\({\varvec{t}} \) Mixtures of Gaussian Processes. In: Huang, DS., Jo, KH., Li, J., Gribova, V., Hussain, A. (eds) Intelligent Computing Theories and Application. ICIC 2021. Lecture Notes in Computer Science(), vol 12837. Springer, Cham. https://doi.org/10.1007/978-3-030-84529-2_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-84529-2_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84528-5
Online ISBN: 978-3-030-84529-2
eBook Packages: Computer ScienceComputer Science (R0)