Abstract
Commonsense knowledge (CSK) is the information that people use in daily life but do not often mention. It summarizes the practical knowledge about how the world works. Existing machines have knowledge but lack commonsense because they do not understand and master commonsense knowledge in the same way that humans do. In the latest works, crowdsourcing-based method is costly and has low coverage, knowledge base completion method can highly fit samples, and methods extracted from unstructured data have the defects of low quality. CG &BF is commonsense knowledge construction with a concept-based generator and a BERT-based filter. We utilize semantic search for node matching and entropy encoder for filtering triples with high abstraction. Two algorithms based on concept aggregation and path credibility are proposed to obtain high-quality CSK triples. We subsequently finetuning a BERT to filter incorrect triples. We obtain 500,000 CSK triples based on ConceptNet, which is superior to other construction methods in novelty and quality. In the reading comprehension task, the three-way attention network is selected as the basic model and the knowledge we generate enables the base model to perform better, which proves that the output of CG &BF has higher quality and ease of use.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bhargava, P., Ng, V.: Commonsense knowledge reasoning and generation with pre-trained language models: a survey. In: Proceedings of the AAAI Conference on Artificial Intelligence (2022). Arxiv:2201.12438
Malaviya, C., Bhagavatula, C., Bosselut, A., Choi, Y.: Commonsense knowledge base completion with structural and semantic context. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2925–2933 (2020)
Zhang, H., Khashabi, D., Song, Y., Roth, D.: Transomcs: from linguistic graphs to commonsense knowledge. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pp. 4004–4010 (2021)
Sap, M., et al.: Atomic: an atlas of machine commonsense for if-then reasoning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3027–3035 (2019)
Zhang, H., Liu, X., Pan, H., Song, Y., Leung, C.W.K.: Aser: a large-scale eventuality knowledge graph. In: Proceedings of The Web Conference 2020, pp. 201–211 (2020)
Nguyen, T.P., Razniewski, S., Weikum, G.: Advanced semantics for commonsense knowledge extraction. In: Proceedings of the Web Conference 2021, pp. 2636–2647 (2021)
Xu, F.F., Lin, B.Y., Zhu, K.: Automatic extraction of commonsense locatednear knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 96–101 (2018)
Liu, J., Xiao, Y., Wang, A., He, L., Shao, B.: Capableof reasoning: a step towards commonsense oracle. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1797–1800. ACM (2020)
Li, X., Taheri, A., Tu, L., Gimpel, K.: Commonsense knowledge base completion. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 1445–1455 (2016)
Saito, I., Nishida, K., Asano, H., Tomita, J.: Commonsense knowledge base completion and generation. In: Proceedings of the 22nd Conference on Computational Natural Language Learning, pp. 141–150 (2016)
Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., Choi, Y.: Comet: commonsense transformers for knowledge graph construction. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 4762–4779 (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the Conference on Neural Information Processing Systems, pp. 6000–6010 (2017)
Hwang, J.D., et al.: Comet-atomic 2020: on symbolic and neural commonsense knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6384–6392 (2021)
Brown, T., et al.: Language models are few-shot learners. In: Proceedings of the Conference on Neural Information Processing Systems, pp. 1877–1901 (2020)
Lewis, M., et al.: Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871–7880 (2020)
Davison, J., Feldman, J., Rush, A.M.: Commonsense knowledge mining from pretrained models. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 1173–1178 (2019)
Jiang, S., Nie, T., Shen, D., Kou, Y., Yu, G.: Entity alignment of knowledge graph by joint graph attention and translation representation. Web Information Systems and Applications, pp. 347–358 (2021)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Proceedings of the Conference on Neural Information Processing Systems, pp. 2787–2795 (2013)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations (2017)
Shang, C., Tang, Y., Huang, J., Bi, J., He, X., Zhou, B.: End-to-end structure-aware convolutional networks for knowledge base completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3060–3067 (2019)
Liu, J., et al.: Mining verb-oriented commonsense knowledge. In: Proceedings of the 36th IEEE International Conference on Data Engineering, pp. 1830–1833 (2020)
Petroni, F., et al.: Language models as knowledge bases? In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 2463–2473 (2019)
Ostermann, S., Roth, M., Modi, A., Thater, S., Pinkal, M.: Semeval-2018 task 11: machine comprehension using commonsense knowledge. In: Proceedings of the 12th International Workshop on Semantic Evaluation, pp. 747–757 (2018)
Wang, L., Sun, M., Zhao, W., Shen, K., Liu, J.: Yuanfudao at semeval-2018 task 11: three-way attention and relational knowledge for commonsense machine comprehension. In: Proceedings of The 12th International Workshop on Semantic Evaluation, pp. 758–762 (2018)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cai, H., Zhao, F., Jin, H. (2022). Commonsense Knowledge Construction with Concept and Pretrained Model. In: Zhao, X., Yang, S., Wang, X., Li, J. (eds) Web Information Systems and Applications. WISA 2022. Lecture Notes in Computer Science, vol 13579. Springer, Cham. https://doi.org/10.1007/978-3-031-20309-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-20309-1_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20308-4
Online ISBN: 978-3-031-20309-1
eBook Packages: Computer ScienceComputer Science (R0)