Abstract
Sociolinguistics believes that common sense is the key factor for people to ensure the robustness of language understanding and text classification in complex social environments. However, commonsense facts are often vague and implicit, and hard to be formalized or combined with current text classification techniques. As an opportunity for us, the Q &A community is a socialized platform for people to communicate knowledge, and implicit common sense can be more easily reflected by the distribution of keywords contained in Q &A data. To this end, our key insight in this paper is to obtain implicit common sense by innovatively exploiting statistical keyword distributions from annotated Q &A corpora to present a Statistics-based Label Interactive Model (SLIM) as an enhancement framework for current deep learning text classification models. Specifically, we first draw inspirations from the sociality shared by Q &A community and commonsense knowledge, and propose a light pre-classification strategy. Then, to exploit the valuable semantics of Q &A domain labels, we devise a commonsense attention module to enhance the textual representation. Afterwards, we design a label interactive attention module to make our methods transplantable to any other deep learning models and text classification datasets. Finally, extensive experiments with several deep learning text classification models on three datasets exactly verify the rationality and effectiveness of our methods.
H. Tao and G. Zhu—Equal contribution.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bao, Z., Han, Z.: What drives users’ participation in online social q &a communities? an empirical study based on social cognitive theory. Aslib J. Inf. Manag. 71, 637–656 (2019)
Cunha, W., et al.: On the cost-effectiveness of neural and non-neural approaches and representations for text classification: a comprehensive comparative study. Inf. Process. Manage. 58(3), 102481 (2021)
Davison, J., Feldman, J., Rush, A.M.: Commonsense knowledge mining from pretrained models. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 1173–1178 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186 (2019)
Weiner, J.A., Simpson, E.S.C.: Oxford English Dictionary (1989)
Fischbein, E.: Intuition and proof. Learn. Math. 3(2), 9–24 (1982)
Hammer, D.: Epistemological beliefs in introductory physics. Cogn. Instr. 12(2), 151–183 (1994)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hotho, A., Nürnberger, A., Paaß, G.: A brief survey of text mining. In: LDV Forum. vol. 20, pp. 19–62. Citeseer (2005)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751. Association for Computational Linguistics (2014)
Linardatos, P., Papastefanopoulos, V., Kotsiantis, S.: Explainable AI: a review of machine learning interpretability methods. Entropy 23(1), 18 (2021)
McCarthy, J.: Circumscription-a form of non-monotonic reasoning. Artif. Intell. 13(1–2), 27–39 (1980)
Peters, M., et al.: Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 2227–2237 (2018)
Rajagopal, D., Olsher, D., Cambria, E., Kwok, K.: Commonsense-based topic modeling. In: Proceedings of the Second International Workshop on Issues of Sentiment Discovery and Opinion Mining, pp. 1–8 (2013)
Sap, M., Horvitz, E., Choi, Y., Smith, N.A., Pennebaker, J.W.: Recollection versus imagination: exploring human memory and cognition via neural language models. In: Association for Computational Linguistics (2020)
Singh, P., et al.: The public acquisition of commonsense knowledge. In: Proceedings of AAAI Spring Symposium: Acquiring (and Using) Linguistic (and World) Knowledge for Information Access (2002)
Taatgen, N.A., Van Rijn, H., Anderson, J.: An integrated theory of prospective time interval estimation: the role of cognition, attention, and learning. Psychol. Rev. 114(3), 577 (2007)
Tenenbaum, J.B., Kemp, C., Griffiths, T.L., Goodman, N.D.: How to grow a mind: statistics, structure, and abstraction. Science 331(6022), 1279–1285 (2011)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Xin, Y., Lieberman, H., Chin, P.: Patchcomm: using commonsense knowledge to guide syntactic parsers. In: Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning, vol. 18, pp. 712–716 (2021)
Yatskar, M., Ordonez, V., Farhadi, A.: Stating the obvious: extracting visual common sense knowledge. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 193–198 (2016)
Yuan, Q., Cong, G., Thalmann, N.M.: Enhancing Naive Bayes with various smoothing methods for short text classification. In: Proceedings of the 21st International Conference on World Wide Web, pp. 645–646 (2012)
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., Liu, Q.: Ernie: enhanced language representation with informative entities. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1441–1451 (2019)
Zhou, Y., Xu, B., Xu, J., Yang, L., Li, C.: Compositional recurrent neural networks for Chinese short text classification. In: 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI), pp. 137–144. IEEE (2016)
Acknowledgments
This research was partially supported by grants from the National Key Research and Development Program of China (Grant No. 2021YFF0901005), the National Natural Science Foundation of China (Grant No. U20A20229) and the USTC Research Funds of the Double First-Class Initiative (No. YD2150002009).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tao, H., Zhu, G., Xu, T., Liu, Q., Chen, E. (2022). Teaching Text Classification Models Some Common Sense via Q &A Statistics: A Light and Transplantable Approach. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13551. Springer, Cham. https://doi.org/10.1007/978-3-031-17120-8_46
Download citation
DOI: https://doi.org/10.1007/978-3-031-17120-8_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17119-2
Online ISBN: 978-3-031-17120-8
eBook Packages: Computer ScienceComputer Science (R0)