Abstract
Representing symbols by high-dimensional vectors makes it easier to perform analogical and associational reasoning, but performing multi-step deductive reasoning typically requires a discrete knowledge base. In this paper, we show a method by which deductive inference can be performed directly on high-dimensional semantic vectors, and characterize some limitations and advantages of this approach. We provide a method for taking a set of semantic vectors representing propositions and encoding a knowledge base telling how those propositions are logically related.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
How to automatically choose vectors that sum up to \(\alpha \) is discussed in Sect. 2.
- 2.
Notice that switching signs, like logical negation, is an involution, so that \(--a = a\), and that addition, like disjunction, is commutative, so that order doesn’t matter. However, \(\lnot (a \vee b)\) cannot just be encoded as \(-(a+b)\) and then simplified to \(-a-b\): the distributive property does not hold. Simplification must take place before encoding as vectors.
- 3.
Converting new assertions to CNF before adding them to the knowledge base is a technique commonly used in large knowledge bases such as Cyc: see http://www.cyc.com/subl-information/cyc-canonicalizer/.
- 4.
In our Matlab implementation, we use non-negative Lasso with a range of values for the sparsity parameter \(\lambda \), which can be calculated efficiently using the DPP package [23]. The active-set method in Matlab would use lsqnonneg. Other solvers, such as fast-NNLS and L-BFGS, are optimized for problems with a larger number of dimensions compared to the number of samples and either take longer or require unworkable amounts of memory for this problem.
- 5.
the fact that analogies can be rearranged in the same way as equal fractions is the origin for the connection between “rational numbers” and “rational thinking”.
References
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)
Breiman, L.: Better subset regression using the nonnegative garrote. Technometrics 37(4), 373–384 (1995)
Caulfield, H.J., Qian, L., Vikram, C.S., Zavalin, A., Chouffani, K., Hardy, J., Mccurdy, W.J., Westphal, J.: Conservative optical logic devices: cold. Adv. Imaging Electron Phys. 142, 1–52 (2006)
Martin Gardner (1959). Logic machines and diagrams
Gordon, A.S., USC EDU: Solving interpretation problems with etcetera abduction. In: Proceedings of the Fifth Annual Conference on Advances in Cognitive Systems (2017)
Harley, R.: The stanhope demonstrator. Mind 4(14), 192–210 (1879)
Kanerva, P.: Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors. Cogn. Comput. 1(2), 139–159 (2009)
Kiros, R., Zhu, Y., Salakhutdinov, R.R., Zemel, R., Urtasun, R., Torralba, A., Fidler, S.: Skip-thought vectors. In: Advances in Neural Information Processing Systems, pp. 3294–3302 (2015)
Lawson, C.L., Hanson, R.J.: Solving least squares problems, vol. 15. Siam (1995)
Lee, M., He, X., Yih, W., Gao, J., Deng, L., Smolensky, P.: Reasoning in vector space: an exploratory study of question answering. arXiv preprint arXiv:1511.06426 (2015)
Levy, S.D., Gayler, R.: Vector symbolic architectures: a new building material for artificial general intelligence. In: Proceedings of the 2008 Conference on Artificial General Intelligence 2008: Proceedings of the First AGI Conference, pp. 414–418. IOS Press (2008)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)
Neelakantan, A., Roth, B., Mc-Callum, A.: Compositional vector space models for knowledge base inference. In: 2015 AAAI Spring Symposium Series (2015)
Rocktäschel, T., Riedel, S.: Learning knowledge base inference with neural theorem provers. In: Proceedings of the 5th Workshop on Automated Knowledge Base Construction, pp. 45–50 (2016)
Rudinger, R., Duh, K., Van Durme, B.: Skip-prop: Representing sentences with one vector per proposition. In: IWCS 2017 12th International Conference on Computational Semantics - Short papers (2017)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science (1985)
Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach. Pearson Education Limited, Malaysia (2016)
Summers-Stay, D.: Deductive and analogical reasoning on a semantically embedded knowledge graph. In: International Conference on Artificial General Intelligence, pp. 112–122 (2017)
Summers-Stay, D.: Semantic vector spaces for broadening consideration of consequences. In: Autonomy and Artificial Intelligence: A Threat or Savior? pp. 227–243 (2017)
Summers-Stay, D., Voss, C., Cassidy, T.: Using a distributional semantic vector space with a knowledge base for reasoning in uncertain conditions. Biol. Inspired Cogn. Archit. 16, 34–44 (2016)
Turney, P.D.: Measuring semantic similarity by latent relational analysis. arXiv preprint cs/0508053 (2005)
Wang, H., Onishi, T., Gimpel, K., McAllester, D.: Emergent predication structure in vector representations of neural readers (2016)
Wang, J.: Dpc (dpp) screening methods for nonnegative lasso. http://dpc-screening.github.io/nnlasso.html. Accessed 14 Nov 2018
Widdows, D., Cohen, T.: Reasoning with vectors: a continuous model for fast robust inference. Logic J. IGPL 23(2), 141–173 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Summers-Stay, D. (2020). Propositional Deductive Inference by Semantic Vectors. In: Bi, Y., Bhatia, R., Kapoor, S. (eds) Intelligent Systems and Applications. IntelliSys 2019. Advances in Intelligent Systems and Computing, vol 1037. Springer, Cham. https://doi.org/10.1007/978-3-030-29516-5_61
Download citation
DOI: https://doi.org/10.1007/978-3-030-29516-5_61
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29515-8
Online ISBN: 978-3-030-29516-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)