Abstract
The last five years have seen the emergence of a powerful new framework for building sophisticated real-world applications based on machine learning. The cornerstones of this approach are (i) the adoption of a Bayesian viewpoint, (ii) the use of graphical models to represent complex probability distributions, and (iii) the development of fast, deterministic inference algorithms, such as variational Bayes and expectation propagation, which provide efficient solutions to inference and learning problems in terms of local message passing algorithms. This paper reviews the key ideas behind this new framework, and highlights some of its major benefits. The framework is illustrated using an example large-scale application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baum, L.E.: An inequality and associated maximization technique in statistical estimation of probabilistic functions of Markov processes. Inequalities 3, 1–8 (1972)
Berger, J.O.: Statistical Decision Theory and Bayesian Analysis, 2nd edn. Springer, Heidelberg (1985)
Bernardo, J.M., Smith, A.F.M.: Bayesian Theory. Wiley, Chichester (1994)
Berrou, C., Glavieux, A., Thitimajshima, P.: Near Shannon limit error-correcting coding and decoding: Turbo-codes (1). In: Proceedings ICC 1993, pp. 1064–1070 (1993)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)
Bishop, C.M., Spiegelhalter, D., Winn, J.: VIBES: A variational inference engine for Bayesian networks. In: Becker, S., Thrun, S., Obermeyer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 793–800. MIT Press, Cambridge (2003)
Dangauthier, P., Herbrich, R., Minka, T., Graepel, T.: Trueskill through time: Revisiting the history of chess. In: Advances in Neural Information Processing Systems, vol. 20 (2007), http://books.nips.cc/nips20.html
Frey, B.J.: Graphical Models for Machine Learning and Digital Communication. MIT Press, Cambridge (1998)
Frey, B.J., MacKay, D.J.C.: A revolution: Belief propagation in graphs with cycles. In: Jordan, M.I., Kearns, M.J., Solla, S.A. (eds.) Advances in Neural Information Processing Systems, vol. 10. MIT Press, Cambridge (1998)
Gallager, R.G.: Low-Density Parity-Check Codes. MIT Press, Cambridge (1963)
Glickman, M.E.: Parameter estimation in large dynmaical paird comparison experiments. Applied Statistics 48, 377–394 (1999)
Herbrich, R., Minka, T., Graepel, T.: Trueskilltm: A Bayesian skill rating system. In: Advances in Neural Information Processing Systems, vol. 19, pp. 569–576. MIT Press, Cambridge (2007)
Jaynes, E.T.: Probability Theory: The Logic of Science. Cambridge University Press, Cambridge (2003)
Kalman, R.E.: A new approach to linear filtering and prediction problems. Transactions of the American Society for Mechanical Engineering, Series D, Journal of Basic Engineering 82, 35–45 (1960)
MacKay, D.J.C., Neal, R.M.: Good error-correcting codes based on very sparse matrices. IEEE Transactions on Information Theory 45, 399–431 (1999)
McEliece, R.J., MacKay, D.J.C., Cheng, J.F.: Turbo decoding as an instance of Pearl’s ‘Belief Ppropagation’ algorithm. IEEE Journal on Selected Areas in Communications 16, 140–152 (1998)
Minka, T.: Expectation propagation for approximate Bayesian inference. In: Breese, J., Koller, D. (eds.) Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 362–369. Morgan Kaufmann, San Francisco (2001)
Minka, T.: A family of approximate algorithms for Bayesian inference. Ph.D. thesis, MIT (2001)
Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–285 (1989)
Rauch, H.E., Tung, F., Striebel, C.T.: Maximum likelihood estimates of linear dynamical systems. AIAA Journal 3, 1445–1450 (1965)
Zarchan, P., Musoff, H.: Fundamentals of Kalman Filtering: A Practical Approach, 2nd edn. AIAA (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Bishop, C.M. (2008). A New Framework for Machine Learning. In: Zurada, J.M., Yen, G.G., Wang, J. (eds) Computational Intelligence: Research Frontiers. WCCI 2008. Lecture Notes in Computer Science, vol 5050. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68860-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-68860-0_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68858-7
Online ISBN: 978-3-540-68860-0
eBook Packages: Computer ScienceComputer Science (R0)