Abstract
Deep Relational Machines (or DRMs) present a simple way for incorporating complex domain knowledge into deep networks. In a DRM this knowledge is introduced through relational features: in the original formulation of [1], the features are selected by an ILP engine using domain knowledge encoded as logic programs. More recently, in [2], DRMs appear to achieve good performance without the need of feature-selection by an ILP engine (the features are simply drawn randomly from a space of relevant features). The reports so far on DRMs though have been deficient on three counts: (a) They have been tested on very small amounts of data (7 datasets, not all independent, altogether with few 1000s of instances); (b) The background knowledge involved has been modest, involving few 10s of predicates; and (c) Performance assessment has been only on classification tasks. In this paper we rectify each of these shortcomings by testing on datasets from the biochemical domain involving 100s of 1000s of instances; industrial-strength background predicates involving multiple hierarchies of complex definitions; and on classification and regression tasks. Our results provide substantially reliable evidence of the predictive capabilities of DRMs; along with a significant improvement in predictive performance with the incorporation of domain knowledge. We propose the new datasets and results as updated benchmarks for comparative studies in neural-symbolic modelling.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
Due to the page limit, we don’t show the hierarchy figure. This hierarchy is available on the web. Refer the Dataset availability section for more information.
- 5.
Size refers to the number of neurons.
- 6.
- 7.
We are adopting this more conservative stand despite low P values for two reasons. First, we note that LRNNs only use the equivalent of the AB representation: we would, therefore, expect their performance to improve if provided with relations in the ABFR representation. Secondly, the reader is no doubt aware of the usual precautions when interpreting P-values obtained from multiple comparisons.
- 8.
We have contacted the authors proposing ways to conduct this comparison.
References
Lodhi, H.: Deep relational machines. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8227, pp. 212–219. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42042-9_27
Vig, L., Srinivasan, A., Bain, M., Verma, A.: An Investigation into the role of domain-knowledge on the use of embeddings. In: Lachiche, N., Vrain, C. (eds.) ILP 2017. LNCS (LNAI), vol. 10759, pp. 169–183. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-78090-0_12
Towell, G.G., Shavlik, J.W.: Knowledge-based artificial neural networks. Artif. Intell. 70(1–2), 119–165 (1994)
d’Avila Garcez, S., Broda, K.B., Gabbay, D.M.: Neural-symbolic Learning Systems: Foundations and Applications. Springer, London (2012). https://doi.org/10.1007/978-1-4471-0211-3
Gust, H., Hagmayer, Y., Kuhnberger, K.U., Sloman, S.: Learning symbolic inferences with neural networks. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 27 (2005)
Sourek, G., Aschenbrenner, V., Zelezny, F., Kuzelka, O.: Lifted relational neural networks. arXiv preprint arXiv:1508.05128 (2015)
Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. J. Artif. Intell. Res. 61, 1–64 (2018)
Lavrač, N., Džeroski, S., Grobelnik, M.: Learning nonrecursive definitions of relations with linus. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 265–281. Springer, Heidelberg (1991). https://doi.org/10.1007/BFb0017020
Faruquie, T.A., Srinivasan, A., King, R.D.: Topic models with relational features for drug design. In: Riguzzi, F., Železný, F. (eds.) ILP 2012. LNCS (LNAI), vol. 7842, pp. 45–57. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38812-5_4
Joshi, S., Ramakrishnan, G., Srinivasan, A.: Feature construction using theory-guided sampling and randomised search. In: Železný, F., Lavrač, N. (eds.) ILP 2008. LNCS (LNAI), vol. 5194, pp. 140–157. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85928-4_14
Ramakrishnan, G., Joshi, S., Balakrishnan, S., Srinivasan, A.: Using ILP to construct features for information extraction from semi-structured text. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS (LNAI), vol. 4894, pp. 211–224. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-78469-2_22
Saha, A., Srinivasan, A., Ramakrishnan, G.: What kinds of relational features are useful for statistical learning? In: Riguzzi, F., Železný, F. (eds.) ILP 2012. LNCS (LNAI), vol. 7842, pp. 209–224. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38812-5_15
Specia, L., Srinivasan, A., Joshi, S., Ramakrishnan, G., das Graças Volpe Nunes, M.: An investigation into feature construction to assist word sense disambiguation. Mach. Learn. 76(1), 109–136 (2009). https://doi.org/10.1007/s10994-009-5114-x
Srinivasan, A., King, R.D.: Feature construction with inductive logic programming: a study of quantitative predictions of biological activity by structural attributes. In: Muggleton, S. (ed.) ILP 1996. LNCS, vol. 1314, pp. 89–104. Springer, Heidelberg (1997). https://doi.org/10.1007/3-540-63494-0_50
França, M.V.M., Zaverucha, G., Garcez, A.: Neural relational learning through semi-propositionalization of bottom clauses. In: AAAI Spring Symposium Series (2015)
Muggleton, S.: Inverse entailment and progol. New Gener. Comput. 13(3–4), 245–286 (1995)
Plotkin, G.: Automatic Methods of Inductive Inference. Ph.D. thesis, Edinburgh University, August 1971
Marx, K.A., O’Neil, P., Hoffman, P., Ujwal, M.: Data mining the NCI cancer cell line compound GI50 values: identifying quinone subtypes effective against melanoma and leukemia cell classes. J. Chem. Inf. Comput. Sci. 43(5), 1652–1667 (2003)
Ralaivola, L., Swamidass, S.J., Saigo, H., Baldi, P.: Graph kernels for chemical informatics. Neural Netw. 18(8), 1093–1110 (2005)
Olier, I., Sadawi, N., Bickerton, G.R., Vanschoren, J., Grosan, C., Soldatova, L., King, R.D.: Meta-QSAR: a large-scale application of meta-learning to drug design and discovery. Mach. Learn., pp. 1–27 (2018)
Van Craenenbroeck, E.; Vandecasteele, H.D.L.: Dmax’s functional group and ring library (2002). https://dtai.cs.kuleuven.be/software/dmax/
Ando, H.Y., Dehaspe, L., Luyten, W., Van Craenenbroeck, E., Vandecasteele, H., Van Meervelt, L.: Discovering h-bonding rules in crystals with inductive logic programming. Mol. Pharm. 3(6), 665–674 (2006)
Grave, K.D., Costa, F.: Molecular graph augmentation with rings and functional groups. J. Chem. Inf. Model. 50(9), 1660–1668 (2010)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Bianchini, M., Scarselli, F.: On the complexity of shallow and deep neural network classifiers. In: ESANN (2014)
Rogers, D., Hahn, M.: Extended-connectivity fingerprints. J. Chem. Inf. Model. 50(5), 742–754 (2010)
Chollet, F., et al.: Keras (2015). https://keras.io
Abadi, M., Agarwal, A., et al.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015), Software available from tensorflow.org. https://www.tensorflow.org/
McCreath, E., Sharma, A.: Lime: a system for learning relations. In: Richter, M.M., Smith, C.H., Wiehagen, R., Zeugmann, T. (eds.) ALT 1998. LNCS (LNAI), vol. 1501, pp. 336–374. Springer, Heidelberg (1998). https://doi.org/10.1007/3-540-49730-7_25
Lodhi, H., Muggleton, S.: Is mutagenesis still challenging. In: Proceedings of the 15th International Conference on Inductive Logic Programming, ILP, pp. 35–40. Citeseer (2005)
Dash, T., Joshi, R.S., Baskar, A., Srinivasan, A.: Some distributional results for discrete stochastic search. In: Submitted to Asian Conference on Machine Learning (ACML) (2018)
Srinivasan, A., Ramakrishnan, G.: Parameter screening and optimisation for ILP using designed experiments. J. Mach. Learn. Res. 12, 627–662 (2011). http://portal.acm.org/citation.cfm?id=1953067
Acknowledgments
A.S. is a Visiting Professorial Fellow, School of CSE, UNSW Sydney. A.S. is supported by the SERB grant EMR/2016/002766. RDK is supported by Indian National Science Academy’s Dr. V. Ramalingaswamy Chair award. We thank the following for their invaluable assistance: researchers at the DTAI, University of Leuven, for suggestions on how to use the background knowledge within DMAX; Ing. Gustav Sourek (Czech Technical University, Prague) and Dr. Ivan Olier Caparroso (Liverpool John Moores University, UK) for providing the dataset information and scores.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Dash, T., Srinivasan, A., Vig, L., Orhobor, O.I., King, R.D. (2018). Large-Scale Assessment of Deep Relational Machines. In: Riguzzi, F., Bellodi, E., Zese, R. (eds) Inductive Logic Programming. ILP 2018. Lecture Notes in Computer Science(), vol 11105. Springer, Cham. https://doi.org/10.1007/978-3-319-99960-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-99960-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-99959-3
Online ISBN: 978-3-319-99960-9
eBook Packages: Computer ScienceComputer Science (R0)