Abstract
Many problems such as voice recognition, speech recognition and many other tasks have been tackled with Hidden Markov Models (HMMs). These problems can also be dealt with an extension of the Naive Bayesian Classifier (NBC) known as Dynamic NBC (DNBC). From a dynamic Bayesian network (DBN) perspective, in a DNBC at each time there is a NBC. NBCs work well in data sets with independent attributes. However, they perform poorly when the attributes are dependent or when there are one or more irrelevant attributes which are dependent of some relevant ones. Therefore, to increase this classifier accuracy, we need a method to design network structures that can capture the dependencies and get rid of irrelevant attributes. Furthermore, when we deal with dynamical processes there are temporal relations that should be considered in the network design. In order to learn automatically these models from data and increase the classifier accuracy we propose an evolutionary optimization algorithm to solve this design problem. We introduce a new encoding scheme and new genetic operators which are natural extensions of previously proposed encoding and operators for grouping problems. The design methodology is applied to solve the recognition problem for nine hand gestures. Experimental results show that the evolved network has higher average classification accuracy than the basic DNBC and a HMM.
This is a preview of subscription content, log in via an institution to check access.
References
Aviles-Arriaga, H.H., Sucar, L.E., Mendoza, C.E., Vargas, B.: Visual recognition of gestures using dynamic naive bayesian classifiers. In: The 12th IEEE International Workshop on Robot and Human Interactive Communication, 2003, pp. 133–138. IEEE Computer Society, Washington, DC, USA (2003)
Aviles-Arriaga, H.H., Sucar, L.E., Mendoza, C.E.: Visual recognition of similar gestures. In: ICPR ’06: Proceedings of the 18th International Conference on Pattern Recognition, pp. 1100–1103. IEEE Computer Society, Washington, DC, USA (2006)
Cameron, P.J.: Combinatorics: Topics, Techniques, Algorithms. Cambridge University Press, Cambridge (1994)
Devore, J.L.: Probability and Statistics for Engineering and the Sciences. Wadsworth, Belmont (1995)
Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer, Germany (2003)
Falkenauer, E.: A new representation and operators for genetic algorithms applied to grouping problems. Evol. Comput. 2(2), 123–144 (1994)
Friedman, N.: The bayesian structural em algorithm. In: Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence (UAI-98), pp. 129–138. Morgan Kaufmann, San Francisco, CA (1998)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2–3), 131–163 (1997)
Friedman, N., Murphy, K., Russell, S.: Learning the structure of dynamic probabilistic networks. In: Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence (UAI-98), pp. 139–147. Morgan Kaufmann, San Francisco, CA (1998)
Goldberg, D.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Massachusetts (1989)
Larrañaga, P., Poza, M.: Structure learning of bayesian networks by genetic algorithms: A. performance analysis of control parameters. IEEE J. Pattern Anal. Mach. Intell. 18(9), 912–926 (1996) citeseer.ist.psu.edu/larranaga94structure.html
Martínez, M., Sucar, L.E.: Learning dynamic naive bayesian classifier. In: Florida Artificial Intelligence Research Symposium (FLAIRS–21), pp. 655–659. AAAI, Menlo Park, California (2008)
Martinez-Arroyo, M., Sucar, L.E.: Learning an optimal naive bayes classifier. In: ICPR ’06: Proceedings of the 18th International Conference on Pattern Recognition, pp. 1236–1239. IEEE Computer Society, Washington, DC, USA (2006)
Myers, J.W., Laskey, K.B., DeJong, K.A.: Learning bayesian networks from incomplete data using evolutionary algorithms. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, vol. 1, pp. 458–465. Morgan Kaufmann, Orlando, Florida, USA (1999)
Palacios-Alonso, M.A., Brizuela, C.A., Sucar, L.: Evolutionary learning of dynamic naive bayesian classifiers. In: Florida Artificial Intelligence Research Symposium (FLAIRS–21), pp. 660–665. AAAI, Menlo Park, California (2008)
Pazzani, M.: Searching for dependencies in bayesian classifiers. In: Learning from Data: Artificial Intelligence and Statistics V., pp. 239–248. Springer, New York (1996)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, CA, USA (1988)
Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. In: Readings in Speech Recognition, IEEE Proceedings, pp. 257–284. Morgan Kaufmann Publishers, San Francisco (1989)
Rechenberg, I.: Evolutionstrategie: Optimieriung Technisher Systeme Nach Prinzipien der Biologisten Evolution. Frommann-Holzboog, Stuggart, Germany (1973)
Ross, B.J., Zuviria, E.: Evolving dynamic bayesian networks with multi-objective genetic algorithms. Appl. Intell. 26(1), 13–23 (2007)
Sucar, L.E., Gillies, D.F., Gillies, D.A.: Probabilistic reasonig in higth-level vision. Image Vis. Comput. 12(1), 42–60 (1994)
Wong, M.L., Lee, S.Y., Leung, K.S.: A hybrid approach to learn bayesian networks using evolutionary programming. In: CEC ’02: Proceedings of the Evolutionary Computation on 2002. CEC ’02. Proceedings of the 2002 Congress, pp. 1314–1319. IEEE Computer Society, Washington, DC, USA (2002)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Palacios-Alonso, M.A., Brizuela, C.A. & Sucar, L.E. Evolutionary Learning of Dynamic Naive Bayesian Classifiers. J Autom Reasoning 45, 21–37 (2010). https://doi.org/10.1007/s10817-009-9130-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10817-009-9130-0