Abstract
System identification comprises a number of linear and nonlinear tools for black-box modeling of dynamical systems, with applications in several areas of engineering, control, biology and economy. However, the usual Gaussian noise assumption is not always satisfied, specially if data is corrupted by impulsive noise or outliers. Bearing this in mind, the present paper aims at evaluating how Gaussian Process (GP) models perform in system identification tasks in the presence of outliers. More specifically, we compare the performances of two existing robust GP-based regression models in experiments involving five benchmarking datasets with controlled outlier inclusion. The results indicate that, although still sensitive in some degree to the presence of outliers, the robust models are indeed able to achieve lower prediction errors in corrupted scenarios when compared to conventional GP-based approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning, 1st edn. MIT Press, Cambridge (2006)
Williams, C.K.I., Barber, D.: Bayesian classification with Gaussian processes. IEEE Trans. Pattern Anal. 20(12), 1342–1351 (1998)
Lawrence, N.D.: Gaussian process latent variable models for visualisation of high dimensional data. In: Advances in Neural Information Processing Systems, pp. 329–336 (2004)
Kocijan, J., Girard, A., Banko, B., Murray-Smith, R.: Dynamic systems identification with Gaussian processes. Math. Comput. Model. Dyn. 11(4), 411–424 (2005)
Murray-Smith, R., Johansen, T.A., Shorten, R.: On transient dynamics, off-equilibrium behaviour and identification in blended multiple model structures. In: European Control Conference (ECC 1999), Karlsruhe, BA-14. Springer (1999)
Solak, E., Murray-Smith, R., Leithead, W.E., Leith, D.J., Rasmussen, C.E.: Derivative observations in Gaussian process models of dynamic systems. In: Advances in Neural Information Processing Systems 16 (2003)
Rottmann, A., Burgard, W.: Learning non-stationary system dynamics online using Gaussian processes. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) Pattern Recognition. LNCS, vol. 6376, pp. 192–201. Springer, Heidelberg (2010)
Ažman, K., Kocijan, J.: Dynamical systems identification using Gaussian process models with incorporated local models. Eng. Appl. Artif. Intel. 24(2), 398–408 (2011)
Petelin, D., Grancharova, A., Kocijan, J.: Evolving Gaussian process models for prediction of ozone concentration in the air. Simul. Model. Pract. Theory 33, 68–80 (2013)
Frigola, R., Chen, Y., Rasmussen, C.: Variational Gaussian process state-space models. In: Advances in Neural Information Processing Systems (NIPS), vol. 27, pp. 3680–3688 (2014)
Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183–233 (1999)
Minka, T.P.: Expectation propagation for approximate Bayesian inference. In: Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence (UAI 2001), pp. 362–369. Morgan Kaufmann (2001)
Nakayama, H., Arakawa, M., Sasaki, R.: A computational intelligence approach to optimization with unknown objective functions. In: Dorffner, G., Bischof, H., Hornik, K. (eds.) ICANN 2001. LNCS, vol. 2130, pp. 73–80. Springer, Heidelberg (2001)
Kuss, M., Pfingsten, T., Csató, L., Rasmussen, C.E.: Approximate inference for robust Gaussian process regression. Technical report 136, Max Planck Institute for Biological Cybernetics, Tubingen, Germany (2005)
Tipping, M.E., Lawrence, N.D.: Variational inference for student-\(t\) models: robust Bayesian interpolation and generalised component analysis. Neurocomputing 69(1), 123–141 (2005)
Jylänki, P., Vanhatalo, J., Vehtari, A.: Robust gaussian process regression with a student-\(t\) likelihood. J. Mach. Learn. Res. 12, 3227–3257 (2011)
Berger, B., Rauscher, F.: Robust Gaussian process modelling for engine calibration. In: Proceedings of the 7th Vienna International Conference on Mathematical Modelling (MATHMOD 2012), pp. 159–164 (2012)
Kuss, M.: Gaussian process models for robust regression, classification, and reinforcement learning. Ph.D. thesis, TU Darmstadt (2006)
Rasmussen, C.E.: Evaluation of Gaussian processes and other methods for non-linear regression. Ph.D. thesis, University of Toronto, Toronto, Canada (1996)
Narendra, K.S., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Networks 1(1), 4–27 (1990)
Majhi, B., Panda, G.: Robust identification of nonlinear complex systems using low complexity ANN and particle swarm optimization technique. Expert Syst. Appl. 38(1), 321–333 (2011)
Acknowledgments
The authors thank the financial support of FUNCAP, IFCE, NUTEC and CNPq (grant no. 309841/2012-7).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Mattos, C.L.C., Santos, J.D.A., Barreto, G.A. (2015). An Empirical Evaluation of Robust Gaussian Process Models for System Identification. In: Jackowski, K., Burduk, R., Walkowiak, K., Wozniak, M., Yin, H. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2015. IDEAL 2015. Lecture Notes in Computer Science(), vol 9375. Springer, Cham. https://doi.org/10.1007/978-3-319-24834-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-24834-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24833-2
Online ISBN: 978-3-319-24834-9
eBook Packages: Computer ScienceComputer Science (R0)