Skip to main content
SpringerLink
Log in

Attention recurrent cross-graph neural network for selecting premises

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Premise selection is a task that selects likely useful premises from a large-scale premise set, which is one of the challenges for automated theorem proving. Nowadays, using graph neural networks to encode logical formulae becomes more and more appealing, as graph representations demonstrate the ability to preserve the semantic and syntactic information of logical formulae. However, graph neural networks in the prior works iteratively update each node’s embedding via information aggregated solely from its neighbors and always ignore the types of nodes. Besides, evidence shows that sharing and exchanging some information between formula pairs can ensure neural-based models more robust in measuring graph relevance. Unluckily, previous graph neural networks generate final graph embeddings independently in the premise selection task. To overcome these shortages, we propose a novel graph neural network, called ARCG-NN, for embedding the first-order logical formulae. The embedding model firstly takes the node types into full consideration and utilizes an attention mechanism based on newly proposed node types to compute weights in the message aggregation. Except for the message from the local neighborhood, ARCG-NN also dynamically exchange the cross-graph information of particular nodes along with the propagation rounds. Besides, an extra gate function on node types is used to filter out irrelevant information in the graph aggregation phase. To train, validate, and test our approach, we build balanced and unbalanced datasets based on the MPTP2078 benchmark. The experimental results demonstrate that the proposed graph neural architecture achieves state-of-the-art classification accuracy on the first-order premise selection task and helps the automated theorem prover E prove more problems in the test dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. The command is ./eprover –satauto-schedule –free-numbers -s -R –delete-bad-limit=2000000000 –definitional-cnf=24 –print-statistics –print-version –proof-object –cpu-limit=60 –sine problem_file.

  2. The command is ./eprover –satauto-schedule –free-numbers -s -R –delete-bad-limit=2000000000 –definitional-cnf=24 –print-statistics –print-version –proof-object –cpu-limit=10 problem_file.

References

  1. Alama J, Heskes T, Kühlwein D, Tsivtsivadze E, Urban J (2014) Premise selection for mathematics by corpus analysis and kernel methods. J Autom Reason 52(2):191–213

    Article  MathSciNet  Google Scholar 

  2. Chvalovskỳ K (2018) Top-down neural model for formulae. In: International conference on learning representations

  3. Crouse M, Abdelaziz I, Cornelio C, Thost V, Wu L, Forbus K, Fokoue A (2019) Improving graph neural network representations of logical formulae with subgraph pooling. arXiv preprint arXiv:1911.06904

  4. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. arXiv preprint arXiv:1606.09375

  5. Evans R, Saxton D, Amos D, Kohli P, Grefenstette E (2018) Can neural networks understand logical entailment? arXiv preprint arXiv:1802.08535

  6. Faizollahzadeh Ardabili S, Najafi B, Shamshirband S, Minaei Bidgoli B, Deo RC, Chau Kw (2018) Computational intelligence approach for modeling hydrogen production: a review. Eng Appl Comput Fluid Mech 12(1):438–458

    Google Scholar 

  7. Fey M, Lenssen JE (2019) Fast graph representation learning with PyTorch geometric. In: ICLR workshop on representation learning on graphs and manifolds

  8. Glimer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212

  9. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. arXiv preprint arXiv:1706.02216

  10. Hoder K, Voronkov A (2011) Sine qua non for large theory reasoning. In: International conference on automated deduction, Springer, pp 299–314

  11. Huang D, Dhariwal P, Song D, Sutskever I (2018) Gamepad: a learning environment for theorem proving. arXiv preprint arXiv:1806.00608

  12. Irving G, Szegedy C, Alemi AA, Eén N, Chollet F, Urban J (2016) Deepmath-deep sequence models for premise selection. In: Advances in neural information processing systems, pp 2235–2243

  13. Jakubův J, Urban J (2017) ENIGMA: efficient learning-based inference guiding machine. In: International conference on intelligent computer mathematics, Springer, pp 292–302

  14. Kaliszyk C, Urban J (2015) Mizar 40 for mizar 40. J Autom Reason 55(3):245–256

    Article  MathSciNet  Google Scholar 

  15. Kaliszyk C, Urban J, Vyskocil J (2015) Efficient semantic features for automated reasoning over large theories. In: Twenty-fourth international joint conference on artificial intelligence

  16. Kingma D, Ba J (2015) Adam: a method for stochastic optimization. In: International conference on learning representations

  17. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International conference on learning representations

  18. Kovács L, Voronkov A (2013) First-order theorem proving and vampire. In: International conference on computer aided verification, Springer, pp 1–35

  19. Kucik AS, Korovin K (2018) Premise selection with neural networks and distributed representation of features. arXiv preprint arXiv:1807.10268

  20. Li Y, Gu C, Dullien T, Vinyals O, Kohli P (2019) Graph matching networks for learning the similarity of graph structured objects. arXiv:1904.12787

  21. Li Y, Tarlow D, Brockschmidt M, Zemel R (2015) Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493

  22. Loos S, Irving G, Szegedy C, Kaliszyk C (2017) Deep network guided proof search. arXiv preprint arXiv:1701.06972

  23. Meng J, Paulson LC (2009) Lightweight relevance filtering for machine-generated resolution problems. J Appl Log 7(1):41–57

    Article  MathSciNet  Google Scholar 

  24. Olšák M, Kaliszyk C, Urban J (2019) Property invariant embedding for automated reasoning. arXiv preprint arXiv:1911.12073

  25. Paliwal A, Loos S, Rabe M, Bansal K, Szegedy C (2019) Graph representations for higher-order logic and theorem proving. arXiv preprint arXiv:1905.10006

  26. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch. In: NIPS workshop

  27. Piotrowski B, Urban J (2018) ATPboost: learning premise selection in binary setting with ATP feedback. In: International joint conference on automated reasoning, Springer, pp 566–574

  28. Robinson AJ, Voronkov A (2001) Handbook of automated reasoning, vol 1. Gulf Professional Publishing, Oxford

    MATH  Google Scholar 

  29. Schulz S (2012) E - a brainiac theorem prover. Ai Commun 15(2):111–126

    MATH  Google Scholar 

  30. Shamshirband S, Rabczuk T, Chau KW (2019) A survey of deep learning techniques: application in wind and solar energy resources. IEEE Access 7:164650–164666

    Article  Google Scholar 

  31. Sutcliffe G (2017) The TPTP problem library and associated infrastructure: from CNF to TH0, TPTP v6. 4.0. J Autom Reason 1–20

  32. Taormina R, Chau KW (2015) Ann-based interval forecasting of streamflow discharges using the lube method and mofips. Eng Appl Artif Intell 45:429–440

    Article  Google Scholar 

  33. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2017) Graph attention networks. arXiv preprint arXiv:1710.10903

  34. Wang M, Tang Y, Wang J, Deng J (2017) Premise selection for theorem proving by deep graph embedding. In: Advances in neural information processing systems, pp 2786–2796

  35. Wu C, Chau KW (2013) Prediction of rainfall time series using modular soft computing methods. Eng Appl Artif Intell 26(3):997–1007

    Article  Google Scholar 

  36. Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International conference on machine learning, PMLR, pp 6861–6871

  37. Wu Z, Pan S, Chen F, Long G, Zhang C, Philip SY (2020) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24

    Article  MathSciNet  Google Scholar 

  38. Yang K, Deng J (2019) Learning to prove theorems via interacting with proof assistants. arXiv preprint arXiv:1905.09381

  39. Zhao L, Akoglu L (2019) Pairnorm: tackling oversmoothing in gnns. arXiv preprint arXiv:1909.12223

Download references

Acknowledgements

This research is supported by the National Natural Science Foundation of China (Grant Nos. 61603307, 61673320 and 61473239) and the Grant from MOE (Ministry of Education in China) Project of Humanities and Social Sciences (Grant No. 19YJCZH048).

Author information

Authors and Affiliations

  1. School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China

    Qinghua Liu

  2. School of Mathematics, Southwest Jiaotong University, Chengdu, China

    Yang Xu & Xingxing He

Authors
  1. Qinghua Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingxing He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qinghua Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Xu, Y. & He, X. Attention recurrent cross-graph neural network for selecting premises. Int. J. Mach. Learn. & Cyber. 13, 1301–1315 (2022). https://doi.org/10.1007/s13042-021-01448-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01448-9

Keywords

Search

Navigation