Skip to main content
SpringerLink
Log in

Select First, Transfer Later: Choosing Proper Datasets for Statistical Relational Transfer Learning

  • Conference paper
  • First Online:
Inductive Logic Programming (ILP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14363))

Included in the following conference series:

Abstract

Statistical Relational Learning (SRL) relies on statistical and probabilistic modeling to represent, learn, and reason about domains with complex relational and rich probability structures. Although SRL techniques have succeeded in many real-world applications, they follow the same assumption as most ML techniques by assuming training and testing data have the same distribution and are sampled from the same feature space. Changes between these distributions might require training a new model using new data. Transfer Learning adapts knowledge already learned to other tasks and domains to help create new models, particularly in a low-data regime setting. Many recent works have succeeded in applying Transfer Learning to relational domains. However, most focus on what and how to transfer. When to transfer is still an open research problem as a pre-trained model is not guaranteed to help or improve performance for learning a new model. Besides, testing every possible pair of source and target domains to perform transference is costly. In this work, we focus on when by proposing a method that relies on probabilistic representations of relational databases and distributions learned by models to indicate the most suitable source domain for transferring. To evaluate our approach, we analyze the performances of two transfer learning-based algorithms given the most similar target domain to a source domain according to our proposal. In the experimental results, our method has succeeded as both algorithms reach their best performance when transferring between the most similar pair of source and target domains.

Supported by CAPES, FAPERJ, and CNPq.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 49.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahmadvand, M., Tahmoresnezhad, J.: Metric transfer learning via geometric knowledge embedding. Appl. Intell. 51, 921–934 (2021)

    Article  Google Scholar 

  2. Azevedo Santos, R., Paes, A., Zaverucha, G.: Transfer learning by mapping and revising boosted relational dependency networks. Mach. Learn. 109, 1435–1463 (2020)

    Article  MathSciNet  Google Scholar 

  3. Bilenko, M., Mooney, R.J.: Adaptive duplicate detection using learnable string similarity measures. In: Proceeding of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2003, pp. 39–48. Association for Computing Machinery, New York (2003)

    Google Scholar 

  4. Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. In: Proceedings of the 23rd International Conference on Machine Learning, ICML 2006, pp. 233–240. Association for Computing Machinery, New York (2006)

    Google Scholar 

  5. De Raedt, L.: Logical and relational learning. In: Zaverucha, G., da Costa, A.L. (eds.) Advances in Artificial Intelligence - SBIA 2008, pp. 1–1. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68856-3

    Chapter  Google Scholar 

  6. de Figueiredo, L.F., Paes, A., Zaverucha, G.: Transfer learning for boosted relational dependency networks through genetic algorithm. In: Katzouris, N., Artikis, A. (eds.) ILP 2021. LNCS, vol. 13191, pp. 125–139. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-97454-1_9

    Chapter  Google Scholar 

  7. Flach, P., Lachiche, N.: 1BC: a first-order bayesian classifier. In: Džeroski, S., Flach, P. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 92–103. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48751-4_10

    Chapter  Google Scholar 

  8. França, M.V., Zaverucha, G., D’avila Garcez, A.S.: Fast relational learning using bottom clause propositionalization with artificial neural networks. Mach. Learn. 94(1), 81–104 (2014)

    Article  MathSciNet  Google Scholar 

  9. Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning. MIT press, Cambridge (2007)

    Book  Google Scholar 

  10. Han, X., Huang, Z., An, B., Bai, J.: Adaptive transfer learning on graph neural networks, pp. 565–574. Association for Computing Machinery, New York (2021)

    Google Scholar 

  11. Khosravi, H., Schulte, O., Hu, J., Gao, T.: Learning compact markov logic networks with decision trees. Mach. Learn. 89(3), 257–277 (2012)

    Article  MathSciNet  Google Scholar 

  12. Kramer, S., Lavrač, N., Flach, P.: Propositionalization Approaches to Relational Data Mining, pp. 262–291. Springer, Heidelberg (2001)

    Google Scholar 

  13. Kumaraswamy, R., Odom, P., Kersting, K., Leake, D., Natarajan, S.: Transfer learning via relational type matching. In: 2015 IEEE International Conference on Data Mining, pp. 811–816. IEEE (2015)

    Google Scholar 

  14. Lachiche, N., Flach, P.A.: 1BC2: a true first-order bayesian classifier. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 133–148. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-36468-4_9

    Chapter  Google Scholar 

  15. Lee, C.K.: Transfer learning with graph neural networks for optoelectronic properties of conjugated oligomers. J. Chem. Phys. 154(2), 024906 (2021)

    Article  Google Scholar 

  16. Luca, T., Paes, A., Zaverucha, G.: Mapping across relational domains for transfer learning with word embeddings-based similarity. In: Katzouris, N., Artikis, A. (eds.) International Conference on Inductive Logic Programming, pp. 167–182. Springer, Heidelberg (2021). https://doi.org/10.1007/978-3-030-97454-1_12

  17. Luca, T., Paes, A., Zaverucha, G.: Combining word embeddings-based similarity measures for transfer learning across relational domains. In: International Conference on Inductive Logic Programming. Springer, Heidelberg (2023)

    Google Scholar 

  18. Luca, T., Paes, A., Zaverucha, G.: Word embeddings-based transfer learning for boosted relational dependency networks. Mach. Learn. 1–34 (2023)

    Google Scholar 

  19. Luo, Y., Wen, Y., Duan, L.Y., Tao, D.: Transfer metric learning: algorithms, applications and outlooks. arXiv preprint arXiv:1810.03944 (2018)

  20. MacKay, D.J.: Information Theory, Inference and Learning Algorithms. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  21. Menéndez, M., Pardo, J., Pardo, L., Pardo, M.: The Jensen-Shannon divergence. J. Franklin Inst. 334(2), 307–318 (1997)

    Article  MathSciNet  Google Scholar 

  22. Mihalkova, L., Huynh, T., Mooney, R.J.: Mapping and revising markov logic networks for transfer learning. In: AAAI, vol. 7, pp. 608–614 (2007)

    Google Scholar 

  23. Mihalkova, L., Mooney, R.J.: Bottom-up learning of markov logic network structure. In: Proceedings of the 24th International Conference on Machine Learning, ICML 2007, pp. 625–632. Association for Computing Machinery, New York (2007)

    Google Scholar 

  24. Muggleton, S.: Inductive logic programming. New Gener. Comput. 8, 295–318 (1991)

    Article  Google Scholar 

  25. Muggleton, S.H.: Inverse entailment and progol. New Gener. Comput. 13(3 &4), 245–286 (1995)

    Article  Google Scholar 

  26. Natarajan, S., Khot, T., Kersting, K., Gutmann, B., Shavlik, J.: Gradient-based boosting for statistical relational learning: the relational dependency network case. Mach. Learn. 86(1), 25–56 (2012)

    Article  MathSciNet  Google Scholar 

  27. Neville, J., Jensen, D.: Relational dependency networks. J. Mach. Learn. Res. 8(3) (2007)

    Google Scholar 

  28. Pan, J.: Review of metric learning with transfer learning. In: AIP Conference Proceedings, vol. 1864. AIP Publishing (2017)

    Google Scholar 

  29. Pan, S.J., Kwok, J.T., Yang, Q., et al.: Transfer learning via dimensionality reduction. In: AAAI, vol. 8, pp. 677–682 (2008)

    Google Scholar 

  30. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)

    Article  Google Scholar 

  31. Rish, I., et al.: An empirical study of the naive bayes classifier. In: IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, vol. 3, pp. 41–46 (2001)

    Google Scholar 

  32. Tamaddoni-Nezhad, A., Muggleton, S.: The lattice structure and refinement operators for the hypothesis space bounded by a bottom clause. Mach. Learn. 76, 37–72 (2009)

    Article  Google Scholar 

  33. Tang, X., Li, Y., Sun, Y., Yao, H., Mitra, P., Wang, S.: Transferring robustness for graph neural network against poisoning attacks, pp. 600–608. Association for Computing Machinery, New York (2020)

    Google Scholar 

  34. Torrey, L., Shavlik, J.: Transfer learning. In: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, pp. 242–264. IGI global (2010)

    Google Scholar 

  35. Van Haaren, J., Kolobov, A., Davis, J.: Todtler: two-order-deep transfer learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 29 (2015)

    Google Scholar 

  36. Wan, C., Pan, R., Li, J.: Bi-weighting domain adaptation for cross-language text classification. In: Twenty-Second International Joint Conference on Artificial Intelligence. Citeseer (2011)

    Google Scholar 

  37. Wrobel, S.: First order theory refinement. Adv. Inductive Logic Program. 32, 14–33 (1996)

    Google Scholar 

  38. Yang, Q., Zhang, Y., Dai, W., Pan, S.J.: Transfer Learning. Cambridge University Press, Cambridge (2020)

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Systems Engineering and Computer Science, COPPE Universidade Federal do Rio de Janeiro, Rio de Janeiro, RJ, Brazil

    Thais Luca & Gerson Zaverucha

  2. Institute of Computing, Universidade Federal Fluminense, Niteroi, RJ, Brazil

    Aline Paes

Authors
  1. Thais Luca
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aline Paes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerson Zaverucha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thais Luca .

Editor information

Editors and Affiliations

  1. Università degli Studi di Ferrara, Ferrara, Italy

    Elena Bellodi

  2. Università degli Studi di Bari "Aldo Moro", Bari, Italy

    Francesca Alessandra Lisi

  3. Università degli Studi di Ferrara, Ferrara, Italy

    Riccardo Zese

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Luca, T., Paes, A., Zaverucha, G. (2023). Select First, Transfer Later: Choosing Proper Datasets for Statistical Relational Transfer Learning. In: Bellodi, E., Lisi, F.A., Zese, R. (eds) Inductive Logic Programming. ILP 2023. Lecture Notes in Computer Science(), vol 14363. Springer, Cham. https://doi.org/10.1007/978-3-031-49299-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49299-0_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49298-3

  • Online ISBN: 978-3-031-49299-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation