Elsevier

Knowledge-Based Systems

Volume 238, 28 February 2022, 107910
Knowledge-Based Systems

Knowledge structure driven prototype learning and verification for fact checking

https://doi.org/10.1016/j.knosys.2021.107910Get rights and content

Abstract

To inhibit the spread of rumorous information, fact checking aims at retrieving relevant evidence to verify the veracity of a given claim. Previous work on fact checking typically uses knowledge graphs (KGs) as external repositories and develop reasoning methods to retrieve evidence from KGs. Domain knowledge structure, including category hierarchy and attribute relationships, can be utilized as discriminative information to facilitate KG based learning and verification. However, in previous fact checking research, category hierarchy and attribute information was often scattered in a KG and treated as the ordinary triple facts in the learning process like other types of information, or was utilized in a limited way without the consideration of category hierarchy or the combination of category hierarchy with the learning process. Thus to better utilize category hierarchy and attribute relationships, in this paper, we propose an end-to-end knowledge structure driven prototype learning and verification method for fact checking. For improving intra-category compactness and inter-category separation, we develop a hierarchical prototype learning technique that jointly learns a prototype for each sub-category to enhance entity embeddings and optimize embedding representations using high-level category. For further enhancing embedding learning, we propose a graph attention network to aggregate information from neighboring attribute nodes. We construct a real-world dataset on food domain, and experimental results on the benchmark datasets and our domain dataset show the effectiveness of our method compared to both previous fact checking methods and representative KG reasoning methods.

Introduction

Internet and social media enable every individual to be a publisher, communicating true or false information instantly and globally. The spread of knowledge-based rumorous information causes severe consequences to individuals and society. False knowledge affects science progress and societal development, undermining trust in science and the capacity of individuals to make evidence-informed choices, including on life-or-death issues [1]. Among the false information on the Web, knowledge-based misinformation accounts for a large portion, based on the statistics reported by an authority website.1

To inhibit the spread of knowledge-based rumors, considerable research efforts have been devoted to fact checking, which aims at retrieving relevant evidence to verify the truthfulness of a given claim. Previous methods on fact checking typically use KGs as external repositories and develop reasoning methods to retrieve evidence from KGs. These methods can be classified into two classes: path-based methods [2], [3], [4], [5] and embedding-based methods [6], [7]. Path-based methods extract evidence from the paths between head and tail entity pairs, but they cannot always find effective paths to support verification due to the incompleteness problem in real-world KGs [8]. To induce the inner connections in KGs, embedding-based methods map the KG components (i.e. entities and relations) into a vector space for effective verification based on the semantic information, which can alleviate the incompleteness issue. However, due to the long-tailed distribution in real-world KGs [9], these methods often suffer from the overfitting problem with insufficient training triples.

In order to reduce overfitting and enhance KG based learning and verification for fact checking, knowledge structure plays an important role. Fig. 1 illustrates an example of domain knowledge structure, which includes category hierarchy and attribute relationships. The upper part in Fig. 1 shows an example of category hierarchy, where grape and apple are entity nodes belonging to sub-categories berry and pome respectively, which belong to the high-level category fruit. This hierarchical structure of categories can be utilized as additional information to improve intra-category compactness and inter-category discrimination in KG based learning. In fact, when human fact-checkers examine an assertion, they would attempt to understand the generalized notion of the assertion by taking advantage of entity category [4]. The lower part in Fig. 1 gives an example of attribute relationships, where entity nodes have attributes such as ingredients fiber and folate and effects anti-cancer and cough-relief. Leveraging the neighboring attribute relationships can enrich the semantic information of entities. The above knowledge structure can provide important semantic and discriminative information to facilitate KG based learning and verification.

However, in previous fact checking research, this information was often scattered in KGs and treated as the ordinary triple facts in the learning process, just the same as other types of information. Although several works on fact checking [4], [5] and KG reasoning [10] has incorporated categories into their methods, they either only utilized the limited category information (i.e. one level of categories) without the consideration of category hierarchy [4], [5], or they did not combine hierarchical category information with the learning process [10].

To make better use of category hierarchy, prototype learning (PL) is an excellent fit for acquiring discriminative category representation. Previously, PL was typically used to find representatives (i.e. prototypes) in the input space and then predict the class label based on its distance to the prototypes [11], [12]. Recent research combines PL with deep learning to develop deep prototype learning models that learn discriminative class prototypes for robust pattern classification [13], [14], [15]. Different from previous research that uses PL only for classification, our focus is on using category hierarchy as discriminative information for developing prototype based embedding learning. Thus we learn the prototypes for each category and take them as representatives to improve the semantic representation of entity embedding learning.

In this paper, we propose an end-to-end Knowledge Structure driven Proto-type Learning and Verification method (KS-PLV) for fact checking. To achieve intra-category compactness and inter-category discrimination in KG based learning, we propose a hierarchical prototype learning method, which learns prototypes for each sub-category and pulls entity embeddings closer to its corresponding prototypes, and by designing the loss function, further pulls closer the embedding clusters of sub-categories belonging to the same high-level category. We then propose a graph attention network, which aggregates the neighboring attribute nodes to enhance the semantic representations of entities. Finally, we design multiple loss functions to conduct KG based embedding learning and verification jointly for fact checking.

The main contributions of our work are as follows:

We propose an end-to-end knowledge structure driven method for fact checking, which can effectively utilize category hierarchy and attribute relationships in KG based learning and verification.

We develop the first hierarchical prototype learning method to improve the robustness of learning entity embeddings at both entity level and category level.

We construct a real-world dataset on food domain, and experimental results on two benchmark datasets and our domain dataset show the effectiveness of our method compared to previous fact checking methods and representative KG reasoning methods.

The rest of the paper is organized as follows. Section 2 introduces the related work on KG reasoning and fact checking. Section 3 describes our proposed method in detail. In Section 4, we conduct intensive empirical studies to evaluate our work and analyze the experimental results. Section 5 concludes the paper and raises some future work.

Section snippets

Related work

Fact checking can be broadly viewed as a reasoning task, typically using KGs as external repositories. In this section, we first review the related research on KG reasoning, and then review the related work on fact checking, focusing on KG based methods.

Proposed method

Given an unverified claim triple (h,r,t) composed of a head entity h, a relation r and a tail entity t, the goal of a fact checker f() is to compute a truth value f(h,r,t) for the triple, with the help of a corresponding knowledge graph (KG) G that contains a large number of triple facts. To verify the truthfulness of claim triples, we expect that the fact checking model f() can produce a higher score for a true claim than that for a false one.

Fig. 2 gives an overview of our method KS-PLV,

Experiments

In this section, we validate our KS-PLV method by comparing it with the previous fact checking methods and representative KG reasoning methods in the related work. We also analyze the experimental results and discuss on the related issues.

Conclusions and future work

This paper proposes an end-to-end knowledge structure driven method KS-PLV for fact checking, which aims to utilize category hierarchy and attribute relationship to facilitate KG based learning and verification. We develop the first hierarchical prototype learning method to jointly learn a prototype for each sub-category and improve entity embeddings using high-level category information. We then propose a relation enhanced graph attention network, which can effectively induce neighboring

CRediT authorship contribution statement

Shuai Wang: Conceptualization, Methodology, Software, Validation, Visualization, Investigation, Data curation, Writing – original draft. Wenji Mao: Conceptualization, Writing – original draft, Writing – review & editing, Supervision. Penghui Wei: Investigation, Writing – original draft (partial). Daniel D. Zeng: Funding acquisition, Project administration.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work is supported in part by the Ministry of Science and Technology of China under Grant #2020AAA0108405, and National Natural Science Foundation of China under Grants #1183 2001 and #71621002.

References (46)

  • ShiB. et al.

    Discriminative predicate path mining for fact checking in knowledge graphs

    Knowl.-Based Syst.

    (2016)
  • HopfH. et al.

    Fake science and the knowledge crisis: ignorance can be fatal

    R. Soc. Open Sci.

    (2019)
  • CiampagliaG.L. et al.

    Computational fact checking from knowledge networks

    PLoS One

    (2015)
  • P. Shiralkar, A. Flammini, F. Menczer, G.L. Ciampaglia, Finding streams in knowledge graphs to support fact checking,...
  • V. Fionda, G. Pirrò, Fact checking via evidence patterns, in: Proceedings of IJCAI, 2018, pp....
  • T. Dong, Z. Wang, J. Li, C. Bauckhage, A.B. Cremers, Triple classification using regions and fine-grained entity...
  • J.Z. Pan, S. Pavlova, C. Li, N. Li, Y. Li, J. Liu, Content based fake news detection using knowledge graphs, in:...
  • A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, O. Yakhnenko, Translating embeddings for modeling multi-relational...
  • X. Lv, Y. Gu, X. Han, L. Hou, J. Li, Z. Liu, Adapting meta knowledge graph information for multi-hop reasoning over...
  • R. Xie, Z. Liu, M. Sun, Representation learning of knowledge graphs with hierarchical types, in: Proceedings of IJCAI,...
  • KohonenT.

    The self-organizing map

    Proc. IEEE

    (1990)
  • GevaS. et al.

    Adaptive nearest neighbor pattern classification

    IEEE Trans. Neural Netw. Learn. Syst

    (1991)
  • J. Snell, K. Swersky, R. Zemel, Prototypical networks for few-shot learning, in: Proceedings of NIPS, 2017, pp....
  • H. Yang, X. Zhang, F. Yin, C. Liu, Robust classification with convolutional prototype learning, in: Proceedings of...
  • K. Allen, E. Shelhamer, H. Shin, J. Tenenbaum, Infinite mixture prototypes for few-shot learning, in: Proceedings of...
  • P.-N. Tan, V. Kumar, J. Srivastava, Selecting the right interestingness measure for association patterns, in:...
  • L.A. Galárraga, C. Teflioudi, K. Hose, F. Suchanek, AMIE: Association rule mining under incomplete evidence in...
  • GalárragaL. et al.

    Fast rule mining in ontological knowledge bases with AMIE+

    VLDB J.

    (2015)
  • F. Yang, Z. Yang, W.W. Cohen, Differentiable learning of logical rules for knowledge base reasoning, in: Proceedings of...
  • N. Lao, T. Mitchell, W.W. Cohen, Random walk inference and learning in a large scale knowledge base, in: Proceedings of...
  • LaoN. et al.

    Relational retrieval using a combination of path-constrained random walks

    Mach. Learn.

    (2010)
  • R. Das, A. Neelakantan, D. Belanger, A. McCallum, Chains of reasoning over entities, relations, and text using...
  • W. Xiong, T. Hoang, W.Y. Wang, DeepPath: A reinforcement learning method for knowledge graph reasoning, in: Proceedings...
  • Cited by (6)

    View full text