Abstract
In this paper, it will be shown that it is feasible to extract finite state machines in a domain of, for rule extraction, previously unencountered complexity. The algorithm used is called the Crystallizing Substochastic Sequential Machine Extractor, or CrySSMEx. It extracts the machine from sequence data generated from the RNN in interaction with its domain. CrySSMEx is parameter free, deterministic and generates a sequence of increasingly deterministic extracted stochastic models until a fully deterministic machine is found.
An erratum to this chapter can be found at http://dx.doi.org/10.1007/11550907_163 .
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jacobsson, H.: Rule extraction from recurrent neural networks: A taxonomy and review. Neural Computation 17, 1223–1263 (2005)
Giles, C.L., Miller, C.B., Chen, D., Chen, H.H., Sun, G.Z.: Learning and extracting finite state automata with second-order recurrent neural networks. Neural Computation 4, 393–405 (1992)
Tiňo, P., Vojtek, V.: Extracting stochastic machines from recurrent neural networks trained on complex symbolic sequences. Neural Network World 8, 517–530 (1998)
Casey, M.: The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation 8, 1135–1178 (1996)
Wiles, J., Elman, J.L.: Learning to count without a counter: A case study of dynamics and activation landscapes in recurrent neural networks. In: Proceedings of the Seventeenth Annual Conference of the Cognitive Science Society, pp. 482–487. MIT Press, Cambridge (1995)
Tonkes, B., Blair, A., Wiles, J.: Inductive bias in context-free language learning. In: Proceedings of the Ninth Australian Conference on Neural Networks (1998)
Rodriguez, P., Wiles, J., Elman, J.L.: A recurrent network that learns to count. Connection Science 11, 5–40 (1999)
Tonkes, B., Wiles, J.: Learning a context-free task with a recurrent neural network: An analysis of stability. In: Heath, R., Hayes, B., Heathcote, A., Hooker, C. (eds.): Dynamical Cognitive Science: Proceedings of the Fourth Biennial Conference of the Australasian Cognitive Science Society (1999)
Bodén, M., Wiles, J., Tonkes, B., Blair, A.: Learning to predict a context-free language: Analysis of dynamics in recurrent hidden units. In: Proceedings of ICANN 1999, pp. 359–364. IEEE, Edinburgh (1999)
Bodén, M., Wiles, J.: Context-free and context-sensitive dynamics in recurrent neural networks. Connection Science 12, 196–210 (2000)
Gers, F.A., Schmidhuber, J.: LSTM recurrent networks learn simple context free and context sensitive languages. IEEE Transactions on Neural Networks 12, 1333–1340 (2001)
Jacobsson, H., Ziemke, T.: Improving procedures for evaluation of connectionist context-free language predictors. IEEE Transactions on Neural Networks 14, 963–966 (2003)
Zeng, Z., Goodman, R.M., Smyth, P.: Learning finite state machines with self-clustering recurrent networks. Neural Computation 5, 976–990 (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jacobsson, H., Ziemke, T. (2005). CrySSMEx, a Novel Rule Extractor for Recurrent Neural Networks: Overview and Case Study. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds) Artificial Neural Networks: Formal Models and Their Applications – ICANN 2005. ICANN 2005. Lecture Notes in Computer Science, vol 3697. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11550907_79
Download citation
DOI: https://doi.org/10.1007/11550907_79
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28755-1
Online ISBN: 978-3-540-28756-8
eBook Packages: Computer ScienceComputer Science (R0)