skip to main content
10.1145/3478586.3478629acmotherconferencesArticle/Chapter ViewAbstractPublication PagesairConference Proceedingsconference-collections
research-article

Contextual BI-Directional Attention Flow With Embeddings From Language Models: A Generative Approach to Emotion Detection

Published: 28 December 2021 Publication History

Abstract

Detection of Emotions from the text is a tedious task. Presently, existing models failed to detect the emotion in absence of the emotional word in the text. The cause phrase selection which gives a deep insight into emotions is considered to be a tough task. The proposed model for detecting emotions is developed through seven layers. Initially, the dataset is represented in the Topical documents using Adversarial Topic Modelling (ATM). Convolutional Neural Network (CNN) maps each phrase in the topical document to Higher-dimensional vectors, followed by the ELMo Model to obtain the fixed word Embeddings vectors. LSTM is responsible for making the interaction between the words in word embeddings and produces the context and query vectors. The bi-directional Attention flow layer determines the most relevant similarity between Context and Query. Finally, Robustly Optimized BERT (RoBERT) architecture is used to detect the Emotion. It is noted that the proposed multi-stage model detects better emotions than all the existing state-of-art models for detecting emotions.

References

[1]
Aishwarya Agrawal, Jiasen Lu, Stanislaw Antol, Margaret Mitchell, C. Lawrence Zitnick, Dhruv Batra, and Devi Parikh. [n.d.]. VQA: Visual Question Answering. arxiv:1505.00468 [cs] http://arxiv.org/abs/1505.00468Comment: The first three authors contributed equally. International Conference on Computer Vision (ICCV) 2015.
[2]
Ying Chen, Wenjun Hou, Xiyao Cheng, and Shoushan Li. [n.d.]. Joint Learning for Emotion Classification and Emotion Cause Detection. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (Brussels, Belgium, 2018-10). Association for Computational Linguistics, 646–651. https://doi.org/10.18653/v1/D18-1066
[3]
Denise Bandeira daSilva, André Schmidt, Diogo Cristiano, Rodrigo Rosa Righi, and Björn Eskofier. [n.d.]. DeepSigns: A Predictive Model Based on Deep Learning for the Early Detection of Patient Health Deterioration. 165 ([n. d.]), 113905. https://doi.org/10.1016/j.eswa.2020.113905
[4]
Lin Gui, Jiannan Hu, Yulan He, Ruifeng Xu, Qin Lu, and Jiachen Du. [n.d.]. A Question Answering Approach to Emotion Cause Extraction. arxiv:1708.05482 [cs] http://arxiv.org/abs/1708.05482Comment: Accepted by EMNLP 2017.
[5]
Maryam Hasan, Elke Rundensteiner, Xiangnan Kong, and Emmanuel Agu. [n.d.]. Discover Trends in Public Emotion Using Social Sensing. ([n. d.]), 1–5. Issue Spring. https://doi.org/10.1145/3065953.3065955
[6]
Maruf Hassan, Md. Sakib Bin Alam, and Tanveer Ahsan. [n.d.]. Emotion Detection from Text Using Skip-Thought Vectors. In 2018 International Conference on Innovations in Science, Engineering and Technology (ICISET) (Chittagong, Bangladesh, 2018-10). IEEE, 501–506. https://doi.org/10.1109/ICISET.2018.8745615
[7]
Sepp Hochreiter and Jürgen Schmidhuber. [n.d.]. Long Short-Term Memory. 9, 8 ([n. d.]), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
[8]
Vinay Kumar Jain, Shishir Kumar, and Steven Lawrence Fernandes. [n.d.]. Extraction of Emotions from Multilingual Text Using Intelligent Text Processing and Computational Linguistics. 21 ([n. d.]), 316–326. https://doi.org/10.1016/j.jocs.2017.01.010
[9]
Zilong Jiang, Shu Gao, and Liangchen Chen. [n.d.]. Study on Text Representation Method Based on Deep Learning and Topic Information. 102, 3([n. d.]), 623–642. https://doi.org/10.1007/s00607-019-00755-y
[10]
Yoon Kim. [n.d.]. Convolutional Neural Networks for Sentence Classification. arxiv:1408.5882 [cs] http://arxiv.org/abs/1408.5882Comment: To appear in EMNLP 2014.
[11]
Bernhard Kratzwald, Suzana Ilic, Mathias Kraus, Stefan Feuerriegel, and Helmut Prendinger. [n.d.]. Deep Learning for Affective Computing: Text-Based Emotion Recognition in Decision Support. 115 ([n. d.]), 24–35. https://doi.org/10.1016/j.dss.2018.09.002 arxiv:1803.06397Comment: Accepted by Decision Support Systems (DSS).
[12]
Gang Liu and Jiabao Guo. [n.d.]. Bidirectional LSTM with Attention Mechanism and Convolutional Layer for Text Classification. 337 ([n. d.]), 325–338. https://doi.org/10.1016/j.neucom.2019.01.078
[13]
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. [n.d.]. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arxiv:1907.11692 [cs] http://arxiv.org/abs/1907.11692
[14]
Saif M. Mohammad and Peter D. Turney. [n.d.]. Crowdsourcing a Word-Emotion Association Lexicon. arxiv:1308.6297 [cs] http://arxiv.org/abs/1308.6297
[15]
Elham Mohammadi, Hessam Amini, and Leila Kosseim. [n.d.]. Neural Feature Extraction for Contextual Emotion Detection. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019) (Varna, Bulgaria, 2019-09). INCOMA Ltd., 785–794. https://doi.org/10.26615/978-954-452-056-4_091
[16]
H. Palangi, L. Deng, Y. Shen, J. Gao, X. He, J. Chen, X. Song, and R. Ward. [n.d.]. Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval. 24, 4([n. d.]), 694–707. https://doi.org/10.1109/TASLP.2016.2520371
[17]
Peyman Passban, Qun Liu, and Andy Way. [n.d.]. Enriching Phrase Tables for Statistical Machine Translation Using Mixed Embeddings. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (Osaka, Japan, 2016-12). The COLING 2016 Organizing Committee, 2582–2591. https://www.aclweb.org/anthology/C16-1243
[18]
Stephen K. Reed and Adam Pease. [n.d.]. A Framework for Constructing Cognition Ontologies Using WordNet, FrameNet, and SUMO. 33 ([n. d.]), 122–144. https://doi.org/10.1016/j.cogsys.2014.06.001
[19]
Nooshin Riahi and Pegah Safari. [n.d.]. Implicit Emotion Detection from Text with Information Fusion. 7, 2 ([n. d.]), 85–99. http://jacr.iausari.ac.ir/article_648851.html
[20]
Matthew Richardson, Christopher J C Burges, and Erin Renshaw. [n.d.]. MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text. ([n. d.]), 11.
[21]
Russ R Salakhutdinov. [n.d.]. Learning in Markov Random Fields Using Tempered Transitions. ([n. d.]), 9.
[22]
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. [n.d.]. Bidirectional Attention Flow for Machine Comprehension. arxiv:1611.01603 [cs] http://arxiv.org/abs/1611.01603Comment: Published as a conference paper at ICLR 2017.
[23]
Jingbo Shang, Jialu Liu, Meng Jiang, Xiang Ren, Clare Voss, and Jiawei Han. [n.d.]. Automated Phrase Mining from Massive Text Corpora. PP ([n. d.]). https://doi.org/10.1109/TKDE.2018.2812203
[24]
Kush Shrivastava, Shishir Kumar, and Deepak Kumar Jain. [n.d.]. An Effective Approach for Emotion Detection in Multimedia Text Data Using Sequence Based Convolutional Neural Network. 78, 20([n. d.]), 29607–29639. https://doi.org/10.1007/s11042-019-07813-9
[25]
Y. Sun, G. G. Yen, and Z. Yi. [n.d.]. Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations. 23, 1([n. d.]), 89–103. https://doi.org/10.1109/TEVC.2018.2808689
[26]
Hao Tang, Donghong Ji, and Qiji Zhou. [n.d.]. Joint Multi-Level Attentional Model for Emotion Detection and Emotion-Cause Pair Extraction. ([n. d.]), S092523122030566X. https://doi.org/10.1016/j.neucom.2020.03.105
[27]
Rui Wang, Deyu Zhou, and Yulan He. [n.d.]. ATM: Adversarial-Neural Topic Model. 56, 6([n. d.]), 102098. https://doi.org/10.1016/j.ipm.2019.102098
[28]
Peng Wu, Xiaotong Li, Si Shen, and Daqing He. [n.d.]. Social Media Opinion Summarization Using Emotion Cognition and Convolutional Neural Networks. 51 ([n. d.]), 101978. https://doi.org/10.1016/j.ijinfomgt.2019.07.004
[29]
Yongliang Wu, Shuliang Zhao, and Wenbin Li. [n.d.]. Phrase2Vec: Phrase Embedding Based on Parsing. 517 ([n. d.]), 100–127. https://doi.org/10.1016/j.ins.2019.12.031
[30]
Rui Xia and Zixiang Ding. [n.d.]. Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts. arxiv:1906.01267 [cs] http://arxiv.org/abs/1906.01267Comment: Accepted by ACL 2019.
[31]
Rui Xia, Mengran Zhang, and Zixiang Ding. [n.d.]. RTHN: A RNN-Transformer Hierarchical Network for Emotion Cause Extraction. ([n. d.]), 5285–5291. https://www.ijcai.org/Proceedings/2019/734
[32]
Di Xue, Lifa Wu, Zheng Hong, Shize Guo, Liang Gao, Zhiyong Wu, Xiaofeng Zhong, and Jianshan Sun. [n.d.]. Deep Learning-Based Personality Recognition from Text Posts of Online Social Networks. 48, 11([n. d.]), 4232–4246. https://doi.org/10.1007/s10489-018-1212-4
[33]
Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan. [n.d.]. A Multimedia Retrieval Framework Based on Semi-Supervised Ranking and Relevance Feedback. 34, 4([n. d.]), 723–742. https://doi.org/10.1109/TPAMI.2011.170
[34]
R. Zhao and K. Mao. [n.d.]. Fuzzy Bag-of-Words Model for Document Representation. 26, 2([n. d.]), 794–804. https://doi.org/10.1109/TFUZZ.2017.2690222

Cited By

View all
  • (2023)Emotional Intelligence Through Artificial Intelligence: NLP and Deep Learning in the Analysis of Healthcare Texts2023 International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI)10.1109/ICAIIHI57871.2023.10489117(1-7)Online publication date: 29-Dec-2023
  • (2023)Text-based emotion recognition using contextual phrase embedding modelMultimedia Tools and Applications10.1007/s11042-023-14524-982:23(35329-35355)Online publication date: 16-Mar-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
AIR '21: Proceedings of the 2021 5th International Conference on Advances in Robotics
June 2021
348 pages
ISBN:9781450389716
DOI:10.1145/3478586
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 December 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. BiDAF
  2. Deep Learning
  3. ELMo
  4. Phrase Mining
  5. RoBERT
  6. Word Embeddings

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

AIR2021

Acceptance Rates

Overall Acceptance Rate 69 of 140 submissions, 49%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)19
  • Downloads (Last 6 weeks)1
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Emotional Intelligence Through Artificial Intelligence: NLP and Deep Learning in the Analysis of Healthcare Texts2023 International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI)10.1109/ICAIIHI57871.2023.10489117(1-7)Online publication date: 29-Dec-2023
  • (2023)Text-based emotion recognition using contextual phrase embedding modelMultimedia Tools and Applications10.1007/s11042-023-14524-982:23(35329-35355)Online publication date: 16-Mar-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media