Skip to main content
SpringerLink
Log in

Policy-based memoization for ILP-based concept discovery systems

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Inductive Programming Logic (ILP)-based concept discovery systems aim to find patterns that describe a target relation in terms of other relations provided as background knowledge. Such systems usually work within first order logic framework, build large search spaces, and have long running times. Memoization has widely been incorporated in concept discovery systems to improve their running times. One of the problems that memoization brings to such systems is the memory overhead which may be a bottleneck. In this work we propose policies that decide what types of concept descriptors to store in memotables and for how long to keep them. The proposed policies have been implemented as extensions to a concept discovery system called Tabular CRIS wEF, and the resulting system is named Policy-based Tabular CRIS. Effects of the proposed policies are evaluated on several datasets. The experimental results show that the proposed policies greatly improve the memory consumption while preserving the benefits introduced by memoization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. For detailed information about the dataset users may refer to King et al. (1996)

  2. Our observation from our past studies is that 3 is a commonly used value for the concept descriptor length; however, the maximum clause length depends greatly on the granularity of the background knowledge. Therefore, clause length limit of 3 may not be sufficient to find good concept descriptors for some domains. By using this limit one can incur less accurate concept descriptors. One reason for using this length limit in our experiments was to obtain concept descriptors under less amount of execution time.

  3. Although it is commonly used to evaluate the performance of parallelized computer programs, several ILP-based concept discovery systems that incorporate memoization followed this metric to evaluate their proposed methods (Blockeel et al. 2002; Costa et al. 2003; Struyf and Blockeel 2003).

References

  • Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., Verkamo, A.I. (1996). Fast discovery of association rules. In: Advances in Knowledge Discovery and Data Mining, pp. 307–328. AAAI/MIT Press.

  • Blaták, J., & Popelínskỳ, L. (2006). Drap: A framework for distributed mining first-order frequent patterns. In: Proceedings of the 16th conference on inductive logic programming, pp. 25–27.

  • Blockeel, H., Dehaspe, L., Demoen, B., Janssens, G., Vandecasteele, H. (2002). Improving the efficiency of inductive logic programming through the use of query packs. Journal of Artificial Intelligence Research, 16, 135–166.

    MATH  Google Scholar 

  • Chung, S., & Luo, C. (2008). Efficient mining of maximal frequent itemsets from databases on a cluster of workstations. Knowledge and Information Systems, 16, 359–391. doi:10.1007/s10115-007-0115-1.

    Article  Google Scholar 

  • Cohen, W.W. (1995). Learning to classify english text with ilp methods. Advances in inductive logic programming, 32, 124–143.

    Google Scholar 

  • Costa, V.S., Srinivasan, A., Camacho, R., Blockeel, H., Demoen, B., Janssens, G., Struyf, J., Vandecasteele, H., Laer, W.V. (2003). Query transformations for improving the efficiency of ILP systems. Journal of Machine Learning Research, 4, 465–491.

    MATH  Google Scholar 

  • Davis, M., Liu, W., Miller, P., Redpath, G. (2011). Detecting anomalies in graphs with numeric labels. In: CIKM, pp. 1197–1202.

  • Dehaspe, L., & Raedt, L.D. (1995). Parallel inductive logic programming. In Proceedings of the MLnet Familiarization Workshop on Statistics, Machine Learning and Knowledge Discovery in Databases, pp. 112–117.

  • Dehaspe, L., & Raedt, L.D. (1997). Mining association rules in multiple relations. In: N. Lavrac, S. Dzeroski (eds.) ILP, Lecture Notes in Computer Science, vol. 1297, pp. 125–132. Springer.

  • Di Mauro, N., Taranto, C., Esposito, F. (2014). Link classification with probabilistic graphs. Journal of Intelligent Information Systems, 1–26. 10.1007/s10844-013-0293-0.

  • Divina, F., Keijzer, M., Marchiori, E. (2003). A method for handling numerical attributes in ga-based inductive concept learners. In: E. Cantú-Paz, J.A. Foster, K. Deb, L. Davis, R. Roy, U.M. O’Reilly, H.G. Beyer, R.K. Standish, G. Kendall, S.W. Wilson, M. Harman, J. Wegener, D. Dasgupta, M.A. Potter, A.C. Schultz, K.A. Dowsland, N. Jonoska, J.F. Miller (eds.) GECCO, Lecture Notes in Computer Science, vol. 2723, pp. 898–908. Springer.

  • Dolsak, B. (2002). Finite element mesh design expert system. Knowledge-based Systems, 15 (8), 315–322.

    Article  Google Scholar 

  • Dolsak, B., & Muggleton, S. (1992). The application of Inductive Logic Programming to finite element mesh design. In: Inductive Logic Programming. Academic Press.

  • Doncescu, A., Waissman, J., Richard, G., Roux, G. (2002). Characterization of bio-chemical signals by inductive logic programming. Knowledge-Based Systems, 15 (1–2), 129–137.

    Article  Google Scholar 

  • Dong, Y., Du, X., Ramakrishna, Y., Ramakrishnan, C., Ramakrishnan, I., Smolka, S., Sokolsky, O., Stark, E., Warren, D. (1999). Fighting livelock in the i-Protocol: A comparative study of verification tools. In TACAS’99: Proceedings of the 5th International Conference on Tools and Algorithms for Construction and Analysis of Systems, Lecture Notes in Computer Science, vol. 1579, pp. 74–88. Springer Berlin / Heidelberg.

  • Džeroski, S. (1993). Handling imperfect data in inductive logic programming. In: Proceedings of the Fourth Scandinavian Conference on Artificial intelligence—93, SCAI93, pp. 111–125. IOS Press, Amsterdam, The Netherlands, The Netherlands.

  • Džeroski, S., Dehaspe, L., Ruck, B., Walley, W. (1994). Classification of river water quality data using machine learning. In: Proceedings of the 5th International Conference on the Development and Application of Computer Techniques to Environmental Studies, Vol. I: Pollution modelling, pp. 129–137.

  • Dzeroski, S. (2003). Multi-relational data mining: An introduction. SIGKDD Explorations, 5 (1), 1–16.

    Article  Google Scholar 

  • Dzeroski, S., Jacobs, N., Molina, M., Moure, C., Muggleton, S., Laer, W.V. (1998). Detecting traffic problems with ILP. In: ILP’98: Proceedings of the 8th International Workshop on Inductive Logic Programming, pp. 281–290.

  • Eager, D., Zahorjan, J., Lazowska, E. (1989). Speedup versus efficiency in parallel systems. IEEE Transactions on Computers, 38 (3), 408 –423. 10.1109/12.21127.

    Article  Google Scholar 

  • Fayyad, U.M., & Irani, K.B. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. In: R. Bajcsy (ed) IJCAI, pp. 1022–1029. Morgan Kaufmann.

  • Fonseca, N., Silva, F., Camacho, R. (2006). April An inductive logic programming system. In: JELIA’06: Proceedings of the 10th European Conference on Logics in Artificial Intelligence, Lecture Notes in Computer Science, vol. 4160, pp. 481–484. Springer Berlin / Heidelberg.

  • Goutte, C., & Gaussier, E. (2005). A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. In: ECIR’05: Proceedings of the 27th European Conference on Information Retrieval, pp. 345–359. Springer.

  • Graefe, G., & McKenna, W. (1993). The volcano optimizer generator: extensibility and efficient search. In: ICDE’93: Proceedings of the 9th International Conference on Data Engineering, pp. 209–218.

  • Hinton, G. (1990). UCI machine learning repository kinship data set. http://archive.ics.uci.edu/ml/datasets/Kinship.

  • Holt, J.D., & Chung, S.M. (2001). Multipass algorithms for mining association rules in text databases. Knowledge and Information Systems, 3, 168–183. doi:10.1007/PL00011664.

    Article  MATH  Google Scholar 

  • James, C. (1996). Part-of-speech disambiguation using ilp. Tech. rep., PRG-TR-25-96 Oxford University Computing Laboratory.

  • Jia, Y., Zhang, J., Huan, J. (2011). An efficient graph-mining method for complicated and noisy data with real-world applications. Knowledge Information System, 28 (2), 423–447.

    Article  Google Scholar 

  • Kavurucu, Y., Senkul, P., Toroslu, I.H. (2009). ILP-based concept discovery in multi-relational data mining. Expert Systems with Applications, 36 (9), 11,418–11,428.

    Article  Google Scholar 

  • Kavurucu, Y., Senkul, P., Toroslu, I.H. (2009). ILP-based concept discovery in multi-relational data mining. Expert Systems with Applications, 36 (9), 11,418–11,428.

    Article  Google Scholar 

  • Kavurucu, Y., Senkul, P., Toroslu, I.H. (2010). Concept discovery on relational databases: New techniques for search space pruning and rule quality improvement. Knowledge-Based Systems, 23 (8), 743–756.

    Article  Google Scholar 

  • King, R.D. (2004). Applying inductive logic programming to predicting gene function. AI Magazine, 25 (1), 57.

    Google Scholar 

  • King, R.D., Muggleton, S.H., Srinivasan, A., Sternberg, M. (1996). Structure-activity relationships derived by machine learning: The use of atoms and their bond connectivities to predict mutagenicity by inductive logic programming. Proceedings of the National Academy of Sciences, 93 (1), 438–442.

    Article  Google Scholar 

  • Koga, H., Ishibashi, T., Watanabe, T. (2007). Fast agglomerative hierarchical clustering algorithm using locality-sensitive hashing. Knowledge and Information Systems, 12, 25–53. doi:10.1007/s10115-006-0027-5.

    Article  MATH  Google Scholar 

  • Krogel, M.A., & Wrobel, S. (2001). Transformation-based learning using multirelational aggregation. In: C. Rouveirol, M. Sebag (eds.) ILP, Lecture Notes in Computer Science, vol. 2157, pp. 142–155. Springer.

  • Kuželka, O., Szabóová, A., železnỳ, F. (2013). A method for reduction of examples in relational learning. Journal of Intelligent Information Systems, 1–27. 10.1007/s10844-013-0294-z.

  • Lahiri, M., & Berger-Wolf, T. (2010). Periodic subgraph mining in dynamic networks. Knowledge and Information Systems, 24, 467–497. doi:10.1007/s10115-009-0253-8.

  • Lavrac, N., & Dzeroski, S. (1993). Inductive Logic Programming: Techniques and Applications. Routledge, New York, NY, 10001.

  • Lavrač, N., Džeroski, S., Grobelnik, M. (1991). Learning nonrecursive definitions of relations with LINUS. In: Y. Kodratoff (ed.) Proceedings of the 5th European Working Session on Learning, Lecture Notes in Artificial Intelligence, vol. 482, pp. 265–281. Springer-Verlag.

  • Li, H.F., Huang, H.Y., Lee, S.Y. (2011). Fast and memory efficient mining of high-utility itemsets from data streams: with and without negative item profits. Knowledge and Information Systems, 28, 495–522.

    Article  Google Scholar 

  • Liu, H., Lin, Y., Han, J. (2011). Methods for mining frequent items in data streams: an overview. Knowledge and Information Systems, 26, 1–30. doi:10.1007/s10115-009-0267-2.

    Article  Google Scholar 

  • Michalski, R., & Larson, J. (1997). Inductive inference of VL decision rules. In: Workshop on Pattern-Directed Inference Systems, vol. 63, pp. 33–44. SIGART Newsletter, ACM.

  • Mooney, R.J., & Califf, M.E. (1995). Induction of first-order decision lists: Results on learning the past tense of english verbs. Journal of Artificial Intelligence Research, 3, 1–24. 10.1613/jair.148.

    Google Scholar 

  • Muggleton, S. (1990). Inductive logic programming. In: ALT, pp. 42–62.

  • Muggleton, S. (1995). Inverse entailment and Progol. New Generation Computing. Special issue on Inductive Logic Programming, 13 (3-4), 245–286.

    Google Scholar 

  • Muggleton, S. (1999). Inductive Logic Programming. In: The MIT Encyclopedia of the Cognitive Sciences (MITECS). MIT Press.

  • Muggleton, S., & Buntine, W. (1988). Machine invention of first order predicates by inverting resolution. In: ML’88: Proceedings of the 5th International Conference on Machine Learning, pp. 339–351.

  • Muggleton, S., & Feng, C. (1990). Efficient induction of logic programs. In: ALT’90: Proceedings of the 1st Conference on Algorithmic Learning Theory, pp. 368–381.

  • Muggleton, S., & Raedt, L.D. (1994). Inductive logic programming: Theory and methods. Journal of Logic Programming, 19/20, 629–679. 10.1016/0743-1066(94)90035-3.

    Article  MathSciNet  MATH  Google Scholar 

  • Mutlu, A., Berk, M.A., Senkul, P. (2010). Improving the time efficiency of ILP-based multi-relational concept discovery with dynamic programming approach. In: ISCIS’10: Proceedings of the 25th International Symposium on Computer and Information Sciences, pp. 43–50.

  • Mutlu, A., & Senkul, P. (2012). Improving hash table hit ratio of an ilp-based concept discovery system with memoization capabilities. In: ISCIS’12: Proceedings of the 27th International Symposium on Computer and Information Sciences.

  • Mutlu, A., & Senkul, P. (2014). Improving hit ratio of ILP-based concept discovery system with memoization. Computer Journal, 57 (1), 138–153.

    Article  Google Scholar 

  • Mutlu, A., Senkul, P., Kavurucu, Y. (2011) Improving the scalability of ILP-based multi-relational concept discovery system through parallelization. Knowledge-Based Systems. doi:10.1016/j.knosys.2011.11.001

  • Nassif, H., Al-Ali, H., Khuri, S., Keirouz, W., Page, D. (2010). An inductive logic programming approach to validate hexose binding biochemical knowledge. In: ILP’09: Proceedings of the 19th International Conference on Inductive Logic Programming, pp. 149–165. Springer-Verlag.

  • Nědellec, C., Adě, H., Bergadano, F., Tausend, B. (1996). Declarative bias in ILP.

  • Pazzani, M.J., Brunk, C., Silverstein, G. (1991). A knowledge-intensive approach to learning relational concepts. In: ML, pp. 432–436.

  • Penn, G., & Munteanu, C. (2003). A tabulation-based parsing method that reduces copying. In: ACL’03: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 1, pp. 200–207.

  • Pompe, U., & Kononenko, I. (1995). Linear space induction in first order logic with relieff. Mathematical and Statistical Methods in Artificial Intelligence. CISM Course and Lecture Notes, 363, 185–220.

    MathSciNet  MATH  Google Scholar 

  • Quinlan, J.R. (1990). Learning logical definitions from relations. Machine Learning, 5 (3), 239–266.

    Google Scholar 

  • Robnik-Sikonja, M., & Kononenko, I. (2003). Theoretical and empirical analysis of relieff and rrelieff. Machine Learning, 53 (1–2), 23–69.

    Article  MATH  Google Scholar 

  • Rocha, R. (2007). On improving the efficiency and robustness of table storage mechanisms for tabled evaluation. In: PADL, pp. 155–169.

  • Rocha, R., Fonseca, N.A., Costa, V.S. (2005). On applying tabling to inductive logic programming. In: ECML’05: Proceeedings of the 16th European Conference on Machine Learning, pp. 707–714.

  • Rocha, R., Silva, F., Costa, V.S. (2000). YapTab: A Tabling Engine Designed to Support Parallelism. In: TAPD’00: Proceedings of the 2nd Conference on Tabulation in Parsing and Deduction, pp. 77–87.

  • Romero, O.E., Gonzalez, J.A., Holder, L.B. (2011). Handling of numeric ranges with the subdue system. In: FLAIRS Conference.

  • Sagonas, K.F., & Stuckey, P.J. (2004). Just enough tabling. In: PPDP, pp. 78–89.

  • Sato, T. (2008). A glimpse of symbolic-statistical modeling by prism. Journal of Intelligent Information Systems, 31 (2), 161–176.

    Article  Google Scholar 

  • Sebag, M., & Rouveirol, C. (1997). Tractable induction and classification in first order logic via stochastic matching. In: IJCAI’97: Proceedings of the 15th International Joint Conferences on Artificial Intelligence, pp. 888–893.

  • Shapiro, E. (1983). Algorithmic Program Debugging. The MIT Press.

  • Skillicorn, D.B., & Wang, Y. (2001). Parallel and sequential algorithms for data mining using inductive logic. Knowledge and Information Systems, 3, 405–421.

    Article  MATH  Google Scholar 

  • Srinivasan, A. (1999). The Aleph Manual. http://www.comlab.ox.ac.uk/activities/machinelearning/Aleph/.

  • Srinivasan, A. (1999). A study of two sampling methods for analyzing large datasets with ILP. Data Mining and Knowledge Discovery, 3 (1), 95–123.

    Article  Google Scholar 

  • Srinivasan, A., King, R.D., Muggleton, S.H., Sternberg, M. (1997). The predictive toxicology evaluation challenge. In: IJCAI-97: Proceedings of the 15th International Joint Conference on Artificial Intelligence, pp. 1–6.

  • Srinivasan, A., Muggleton, S.H., King, R., Sternberg, M. (1994). Mutagenesis: Ilp experiments in a non-determinate biological domain. In: Proceedings of the 4th International Workshop on Inductive Logic Programming, volume 237 of GMD-Studien, pp. 217–232.

  • Struyf, J., & Blockeel, H. (2003). Query optimization in inductive logic programming by reordering literals. In: ILP’03: Proceedings of the 13th International Conference on Inductive Logic Programming, pp. 329–346. Springer-Verlag.

  • Tran, T.N., Satou, K., Ho, T.B. (2005). Using inductive logic programming for predicting protein-protein interactions from multiple genomic data. PKDD, pp. 321–330.

  • Troncon, R., Demoen, B., Janssens, G. (2006). When tabling does not work. In: Proceedings of Colloquium on Implementation of Constraint Logic Programming Systems.

  • Tveit, A., & Hetland, M. (2003). Multicategory incremental proximal support vector classifiers. In: V. Palade, R. Howlett, L. Jain (eds.) Knowledge-Based Intelligent Information and Engineering Systems, Lecture Notes in Computer Science, vol. 2773, pp. 386–392. Springer Berlin / Heidelberg.

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Selcuk University, 42075, Konya, Turkey

    Alev Mutlu

  2. Computer Engineering Department, Middle East Technical University, 06800, Ankara, Turkey

    Pinar Karagoz

Authors
  1. Alev Mutlu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pinar Karagoz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alev Mutlu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mutlu, A., Karagoz, P. Policy-based memoization for ILP-based concept discovery systems. J Intell Inf Syst 46, 99–120 (2016). https://doi.org/10.1007/s10844-015-0354-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-015-0354-7

Keywords

Search

Navigation