Abstract
Recent progress in the area of modern technologies confirms that information is not only a commodity but can also become a tool for competition and rivalry among governments and corporations, or can be applied by ill-willed people to use it in their hate speech practices. The impact of information is overpowering and can lead to many socially undesirable phenomena, such as panic or political instability. To eliminate the threats of fake news publishing, modern computer security systems need flexible and intelligent tools. The design of models meeting the above-mentioned criteria is enabled by artificial intelligence, and above all by the state-of-the-art neural network architectures, applied in NLP tasks. The BERT neural network belongs to this type of architectures. This paper presents a hybrid architecture connecting BERT with RNN; the architecture was used to create models for detecting fake news.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Pretrained models. https://huggingface.co/transformers/v2.3.0/pretrained_models.html. Accessed 04 May 2020
Ahmed, H., Traore, I., Saad, S.: Detecting opinion spams and fake news using text classification. Secur. Privacy 1(1), e9 (2018)
Akbik, A., Bergmann, T., Blythe, D., Rasul, K., Schweter, S., Vollgraf, R.: FLAIR: an easy-to-use framework for state-of-the-art NLP. In: Ammar, W., Louis, A., Mostafazadeh, N. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, 2–7 June 2019, Demonstrations, pp. 54–59. Association for Computational Linguistics (2019)
Choraś, M., Pawlicki, M., Kozik, R., Demestichas, K.P., Kosmides, P., Gupta, M.: Socialtruth project approach to online disinformation (fake news) detection and mitigation. In: Proceedings of the 14th International Conference on Availability, Reliability and Security, ARES 2019, Canterbury, UK, 26–29 August 2019, pp. 68:1–68:10. ACM (2019)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, 2–7 June 2019, vol. 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (2019)
Giełczyk, A., Wawrzyniak, R., Choraś, M.: Evaluation of the existing tools for fake news detection. In: Saeed, K., Chaki, R., Janev, V. (eds.) Computer Information Systems and Industrial Management - 18th International Conference, CISIM 2019, Belgrade, Serbia, September 19–21, 2019, Proceedings, Lecture Notes in Computer Science, vol. 11703, pp. 144–151. Springer (2019)
Jwa, H., Dongsuk, O., Park, K., Kang, J., Lim, H.: exBAKE: automatic fake news detection model based on bidirectional encoder representations from transformers (bert). Appl. Sci. 9(19), 4062 (2019)
Ksieniewicz, P., Choraś, M., Kozik, R., Wozniak, M.: Machine learning methods for fake news classification. In: Yin, H., Camacho, D., Tiño, P., Tallón-Ballesteros, A.J., Menezes, R., Allmendinger, R. (eds.), Intelligent Data Engineering and Automated Learning - IDEAL 2019 - 20th International Conference, Manchester, UK, 14–16 November, 2019, Proceedings, Part II, Lecture Notes in Computer Science, vol. 11872, pp. 332–339. Springer (2019)
Kula, S., Choraś, M., Kozik, R., Ksieniewicz, P., Woźniak, M.: Sentiment analysis for fake news detection by means of neural networks. In: Krzhizhanovskaya, V.V., Závodszky, G., Lees, M.H., Dongarra, J.J., Sloot, Sérgio Brissos, P.M.A., Teixeira, J. (eds.) Computational Science – ICCS 2020, pp. 653–666. Springer, Cham (2020)
Pierre, S.: Fake News Classification with BERT. https://towardsdatascience.com/fake-news-classification-with-bert-afbeee601f41. Accessed 02 May 2020
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Guyon, S., von Luxburg, U., Bengio, A., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 5998–6008 (2017)
Vlad, G.-A., Tanase, M.-A., Onose, C., Cercel, D.-C.: Sentence-level propaganda detection in news articles with transfer learning and BERT-BiLSTM-capsule model. In: Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, Hong Kong, China, pp. 148–154 (019)
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., Choi, Y.: Defending against neural fake news. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8–14 December 2019 Canada, Vancouver, BC, pp. 9051–9062 (2019)
Acknowledgement
This work is supported by SocialTruth project (http://socialtruth.eu), which has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 825477.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kula, S., Choraś, M., Kozik, R. (2021). Application of the BERT-Based Architecture in Fake News Detection. In: Herrero, Á., Cambra, C., Urda, D., Sedano, J., Quintián, H., Corchado, E. (eds) 13th International Conference on Computational Intelligence in Security for Information Systems (CISIS 2020). CISIS 2019. Advances in Intelligent Systems and Computing, vol 1267. Springer, Cham. https://doi.org/10.1007/978-3-030-57805-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-57805-3_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-57804-6
Online ISBN: 978-3-030-57805-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)