Abstract
Recurrent neural networks have proven useful in natural language processing. For example, they can be trained to predict, and even generate plausible text with few or no spelling and syntax errors. However, it is not clear what grammar a network has learned, or how it keeps track of the syntactic structure of its input. In this paper, we present a new method to extract a finite state machine from a recurrent neural network. A FSM is in principle a more interpretable representation of a grammar than a neural net would be, however the extracted FSMs for realistic neural networks will also be large. Therefore, we also look at ways to group the states and paths through the extracted FSM so as to get a smaller, easier to understand model of the neural network. To illustrate our methods, we use them to investigate how a neural network learns noun-verb agreement from a simple grammar where relative clauses may appear between noun and verb.
J. Bateman—This work was partially funded by Deutsche Forschungsgemeinschaft (DFG) through the Collaborative Research Center 1320, EASE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., Goldberg, Y.: Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. CoRR abs/1608.04207 (2016)
Angluin, D.: Identifying languages from stochastic examples. Technical report, YALEU/DCS/RR-614, Yale University, Department of Computer Science, New Haven, CT (1988)
Carrasco, R.C., Oncina, J.: Learning deterministic regular grammars from stochastic samples in polynomial time. ITA 33(1), 1–20 (1999)
Choo, J., Liu, S.: Visual analytics for explainable deep learning. CoRR abs/1804.02527 (2018)
Cohen, M., Caciularu, A., Rejwan, I., Berant, J.: Inducing regular grammars using recurrent neural networks. CoRR abs/1710.10453 (2017)
Gers, F.A., Schmidhuber, E.: LSTM recurrent networks learn simple context-free and context-sensitive languages. Trans. Neural Netw. 12(6), 1333–1340 (2001)
Giles, C.L., Miller, C.B., Chen, D., Sun, G.Z., Chen, H.H., Lee, Y.C.: Extracting and learning an unknown grammar with recurrent neural networks. In: Proceedings of the 4th International Conference on Neural Information Processing Systems. . NIPS 1991, pp. 317–324. Morgan Kaufmann Publishers Inc., San Francisco (1991)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hohman, F., Kahng, M., Pienta, R., Chau, D.H.: Visual analytics in deep learning: an interrogative survey for the next frontiers. CoRR abs/1801.06889 (2018)
Joulin, A., Mikolov, T.: Inferring algorithmic patterns with stack-augmented recurrent nets. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) NIPS, pp. 190–198 (2015)
Karpathy, A.: The unreasonable effectiveness of recurrent neural networks (2015). http://karpathy.github.io/2015/05/21/rnn-effectiveness/. Accessed 29 Jan 2018
Karpathy, A., Johnson, J., Fei-Fei, L.: Visualizing and understanding recurrent networks. CoRR abs/1506.02078 (2015)
Li, J., Chen, X., Hovy, E., Jurafsky, D.: Visualizing and understanding neural models in NLP. In: Proceedings of NAACL-HLT, pp. 681–691 (2016)
Li, J., Monroe, W., Jurafsky, D.: Understanding neural networks through representation erasure. CoRR abs/1612.08220 (2016)
Linzen, T., Dupoux, E., Goldberg, Y.: Assessing the ability of lstms to learn syntax-sensitive dependencies. TACL 4, 521–535 (2016)
Lipton, Z.C.: The mythos of model interpretability. In: 2016 ICML workshop on human interpretability in machine learning. CoRR abs/1606.03490 (2016)
Ming, Y., et al.: Understanding hidden memories of recurrent neural networks. CoRR abs/1710.10777 (2017)
Strobelt, H., Gehrmann, S., Huber, B., Pfister, H., Rush, A.M.: Visual analysis of hidden state dynamics in recurrent neural networks. CoRR abs/1606.07461 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Pomarlan, M., Bateman, J. (2018). Understanding NLP Neural Networks by the Texts They Generate. In: Trollmann, F., Turhan, AY. (eds) KI 2018: Advances in Artificial Intelligence. KI 2018. Lecture Notes in Computer Science(), vol 11117. Springer, Cham. https://doi.org/10.1007/978-3-030-00111-7_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-00111-7_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00110-0
Online ISBN: 978-3-030-00111-7
eBook Packages: Computer ScienceComputer Science (R0)