Abstract
Simple recurrent networks (SRN) can learn languages generated by finite state automata (FSA) [5]. The reverse process, i.e., extracting rules from neural networks in order to get FSAs, has also been explored. Rules from neural networks are generally extracted by partitioning the hidden state space of the network. Hidden Markov models (HMM) can also be used to extract FSAs from neural networks. The difference with other approaches is that it is not necessary to use the hidden state space activities of the network to extract the FSA: only the input-output relations of the network are required in fitting a HMM. Nonetheless, equivalent automata can be extracted. HMMs can thus be used to provide interpretations for the representations of neural networks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bakker, B. & Jong, M. de. (In press). The epsilon state count. In From animals to animats 6: Proceedings of the sixth international conference on the simulation of adaptive behavior, SAB 2000.
Becker, J. D., Honerkamp, J., Hirsch, J., Fröbe, U., Schlatter, E. & Greger, R. (1994). Analysing ion channels with hidden Markov models. European Journal of Physiology, 426, 328–332.
Chien, J. T. & Wang, H. C. (1997). Telephone speech recognition based on bayesian adaptation of hidden Markov models. Speech Communication, 22(4), 369–384.
Cleeremans, A. & McClelland, J. L. (1991). Learning the structure of event sequences. JEP: General, 120, 235–253.
Cleeremans, A., Servan-Schreiber, D. & McClelland, J. L. (1989). Finite state automata and simple recurrent networks. Neural Computation, 1, 372–381.
Durbin, M. A., Earwood, J. & Golden, R. M. (2000). Hidden Markov models for coding story recall data. In L. R. Gleitman & A. K. Joshi (editors), Proceedings of the twentysecond annual conference of the cognitive science society, 113–117. Lawrence Erlbaum Associates.
Elliott, R. J., Aggoun, L. & Moore, J. B. (1995). Hidden Markov models: Estimation and control. New York: Springer Verlag.
Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179–211.
Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z. & Lee, Y. C. (1992). Learning and extracting finite state automata with second-order recurrent neural networks. Neural Computation, 4, 393–405.
Hopcroft, J. & Ullman, J. (1979). Introduction to automata theory, languages and computation. Redwood City (CA): Addison-Wesley.
Krogh, A. (1998). An introduction to hidden Markov models for biological sequences. In S. L. Salzberg, D. B. Searls & S. Kasif (editors), Computational methods in molecular biology, 45–63. Elsevier.
Nissen, M. J. & Bullemer, P. (1987). Attentional requirements of learning: Evidence from performance measures. Cognitive Psychology, 19, 1–32.
Omlin, C. & Giles, C. (1996). Extraction of rules from discrete-time recurrent neural networks. Neural Networks, 9(1), 41–51.
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of IEEE, 77(2), 267–295.
Schmidbauer, O., Casacuberta, F., Castro, M. J. & Hegerl, G. (1993). Articulatory representation and speech technology. Language and Speech, 36(2), 331–351.
Tino, P. & Koteles, M. (2000). Extracting finite state representations from recurrent neural networks trained on chaotic symbolic sequences. IEEE Transactions on Neural Networks. (In press)
Visser, I., Raijmakers, M. E. J. & Molenaar, P. C. M. (In press). Confidence intervals for hidden Markov model parameters. Britishjournal of mathematical and statistical psychology. (Preprint available from: ingmar@dds.nl)
Yang, J., Xu, Y. & Chen, C. S. (1997). Human action learning via hidden Markov model. IEEE Transactions on Systems, Man and Cybernetics, 27(1), 34–44.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag London
About this paper
Cite this paper
Visser, I., Raijmakers, M.E.J., Molenaar, P.C.M. (2001). Hidden Markov Model Interpretations of Neural Networks. In: French, R.M., Sougné, J.P. (eds) Connectionist Models of Learning, Development and Evolution. Perspectives in Neural Computing. Springer, London. https://doi.org/10.1007/978-1-4471-0281-6_20
Download citation
DOI: https://doi.org/10.1007/978-1-4471-0281-6_20
Publisher Name: Springer, London
Print ISBN: 978-1-85233-354-6
Online ISBN: 978-1-4471-0281-6
eBook Packages: Springer Book Archive