Abstract
Learning from Interpretation Transition (LFIT) is an unsupervised learning algorithm which learns the dynamics just by observing state transitions. LFIT algorithms have mainly been implemented in the symbolic method, but they are not robust to noisy or missing data. Recently, research works combining logical operations with neural networks are receiving a lot of attention, with most works taking an extraction based approach where a single neural network model is trained to solve the problem, followed by extracting a logic model from the neural network model. However most research work suffer from the combinatorial explosion problem when trying to scale up to solve larger problems. In particular a lot of the invariance that hold in the symbolic world are not getting utilized in the neural network field. In this work, we present a model that exploits symbolic invariance in our problem. We show that our model is able to scale up to larger tasks than previous work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization (2016)
Brown, T.B., Mann, B., et al.: Language models are few-shot learners. CoRR, abs/2005.14165 (2020)
d’Avila Garcez, A.S., Gori, M., Lamb, L.C., Serafini, L., Spranger, M., Tran, S.N.: Neural-symbolic computing: an effective methodology for principled integration of machine learning and reasoning. CoRR, abs/1905.06088 (2019)
Garcez, A.D.A., Broda, K., Gabbay, D.M.: Symbolic knowledge extraction from trained neural networks: a sound approach. Artif. Intell. 125(1), 155–207 (2001)
Dong, H., Mao, J., Lin, T., Wang, C., Li, L., Zhou, D.: Neural logic machines. CoRR, abs/1904.11694 (2019)
Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. CoRR, abs/1711.04574 (2017)
Avila Garcez, A.S., Zaverucha, G.: The connectionist inductive learning and logic programming system. Appl. Intell. 11(1), 59–77 (1999)
d’Avila Garcez, A.S., Gabbay, D.M., Broda, K.B.: Neural-symbolic learning system: foundations and applications. Springer, Heidelberg (2002). https://doi.org/10.1007/978-1-4471-0211-3
Gentet, E., Tourret, S., Inoue, K.: Learning from interpretation transition using feed-forward neural network. In: Proceedings of ILP 2016, CEUR Proceedings 1865, pp. 27–33 (2016)
Hendrycks, D., Gimpel, K.: Bridging nonlinearities and stochastic regularizers with gaussian error linear units. CoRR, abs/1606.08415 (2016)
Inoue, K.: Logic programming for boolean networks. In: Walsh, T. (ed.) IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, 16–22 July 2011, pp. 924–930. IJCAI/AAAI (2011)
Inoue, K., Ribeiro, T., Sakama, C.: Learning from interpretation transition. Mach. Learn. 94(1), 51–79 (2013). https://doi.org/10.1007/s10994-013-5353-8
Klarner, H., Streck, A., Siebert, H.: PyBoolNet: a python package for the generation, analysis and visualization of boolean networks. Bioinformatics 33(5), 770–772 (2016)
Lee, J., Lee, Y., Kim, J., Kosiorek, A., Choi, S., Teh, Y.W.: Set transformer. CoRR, abs/1810.00825 (2018)
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates Inc (2019)
Phua, Y.J., Inoue, K.: Learning logic programs from noisy state transition data. In: Kazakov, D., Erten, C. (eds.) ILP 2019. LNCS (LNAI), vol. 11770, pp. 72–80. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49210-6_7
Reed, S., De Freitas, N.: Neural programmer-interpreters. arXiv preprint arXiv:1511.06279 (2015)
Ribeiro, T., Inoue, K.: Learning prime implicant conditions from interpretation transition. In: Davis, J., Ramon, J. (eds.) ILP 2014. LNCS (LNAI), vol. 9046, pp. 108–125. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23708-4_8
Ribeiro, T., Magnin, M., Inoue, K., Sakama, C.: Learning delayed influences of biological systems. Front. Bioeng. Biotechnol. 2, 81 (2015)
Rintala, T., et al.: Using boolean network extraction of trained neural networks to reverse-engineer gene-regulatory networks from time-series data (2019)
Vaswani, A., et al.: Attention is all you need. CoRR, abs/1706.03762 (2017)
Xiao, Y., Dougherty, E.R.: The impact of function perturbations in Boolean networks. Bioinformatics 23(10), 1265–1273 (2007)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. CoRR, abs/1212.5701 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Phua, Y.J., Inoue, K. (2022). Learning Logic Programs Using Neural Networks by Exploiting Symbolic Invariance. In: Katzouris, N., Artikis, A. (eds) Inductive Logic Programming. ILP 2021. Lecture Notes in Computer Science(), vol 13191. Springer, Cham. https://doi.org/10.1007/978-3-030-97454-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-97454-1_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-97453-4
Online ISBN: 978-3-030-97454-1
eBook Packages: Computer ScienceComputer Science (R0)