research-article

A Supplementary Feature Set for Sentiment Analysis in Japanese Dialogues

Authors:

Peter Lajos Ihasz,

Victor V. KryssanovAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), Volume 18, Issue 4

Article No.: 39, Pages 1 - 21

https://doi.org/10.1145/3310283

Published: 07 May 2019 Publication History

Abstract

Recently, real-time affect-awareness has been applied in several commercial systems, such as dialogue systems and computer games. Real-time recognition of affective states, however, requires the application of costly feature extraction methods and/or labor-intensive annotation of large datasets, especially in the case of Asian languages where large annotated datasets are seldom available. To improve recognition accuracy, we propose the use of cognitive context in the form of “emotion-sensitive” intentions. Intentions are often represented through dialogue acts and, as an emotion-sensitive model of dialogue acts, a tagset of interpersonal-relations-directing interpersonal acts (the IA model) is proposed. The model's adequacy is assessed using a sentiment classification task in comparison with two well-known dialogue act models, the SWBD-DAMSL and the DIT++. For the assessment, five Japanese in-game dialogues were annotated with labels of sentiments and the tags of all three dialogue act models which were used to enhance a baseline sentiment classifier system. The adequacy of the IA tagset is demonstrated by a 9% improvement to the baseline sentiment classifier's recognition accuracy, outperforming the other two models by more than 5%.

References

[1]

O. Abdel-Hamid, L. Deng, and D. Yu. 2013. Exploring convolutional neural network structures and optimization techniques for speech recognition. In Proceedings of 14th Annual Conference of the International Speech Communication Association (INTERSPEECH'13).

[2]

J. Ang, R. Dhillon, A. Krupski, E. Shriberg, and A. Stolcke. 2002. Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In Proceedings of the 7th International Conference on Spoken Language Processing (INTERSPEECH'02).

[3]

Y. Arimoto and H. Mori. 2017. Emotion category mapping to emotional space by cross-corpus emotion labeling. In Proceedings of the International Conference on Situated Interaction (INTERSPEECH'17).

[4]

A. Batliner, K. Fischer, R. Huber, J. Spilker, and E. Noth. 2003. How to find trouble in communication. Speech Communication 40, 1 117--143.

Digital Library

[5]

P. Brown and S. C. Levinson. 1987. Politeness: Some Universals in Language Usage (Vol. 4), Cambridge University Press, Cambridge.

[6]

H. Bunt. 2009. The DIT++ taxonomy for functional dialogue markup. In Proceedings of AAMAS 2009 Work. 13--24.

[7]

H. Bunt. 2011. Multifunctionality in dialogue. Comput. Speech Lang. 25 222--245.

Digital Library

[8]

H. Bunt, J. Alexandersson, J. Choe, A. C. Fang, K. Hasida, V. Petukhova, A. Popescu-Belis, and D. Traum. 2012. ISO 24617-2: A semantically-based standard for dialogue annotation. In Proceedings of LREC 2012. 430--437.

[9]

H. Bunt, V. Petukhova, D. Traum, and J. Alexandersson. 2017. Dialogue Act Annotation with the ISO 24617-2 Standard. Multimodal Interaction with W3C Standards: Towards Natural User Interfaces to Everything, Deborah Dahl (Ed.). Springer, Berlin, 109--135.

[10]

J. Chung, C. Gulcehre, K. Cho, and Y. Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. ArXiv1--9.

[11]

D. Duncan, G. Shine, and C. English. 2016. Facial Emotion Recognition in Real Time. Report, Stanford.

[12]

F. Eyben, M. Wolmiller, and B. Schuller. 2009. OpenEAR—Introducing the Munich open-source emotion and affect recognition toolkit. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction. IEEE, 1--6.

[13]

H. M. Fayek, M. Lech, and L. Cavedon. 2015. Towards real-time speech emotion recognition using deep neural networks. In Proceedings of the Conference on Signal Processing and Communication Systems (ICSPCS). IEEE, 1--5.

[14]

N. H. Frijda. 1987. Emotion, cognitive structure, and action tendency. Cogn. Emot. 1 115--143.

[15]

P. L. Ihasz, T. H. Van, and V. V. Kryssanov. 2015. A computational model for conversational Japanese. In Proceedings of 2015 International Conference on Culture and Computing. 64--71.

Digital Library

[16]

D. Jurafsky, E. Shriberg, and D. Biasca. 1997. Switchboard SWBD-DAMSL Shallow Discourse-Function Annotation (Coders Manual, Draft 13). Technical Report 97-02. University of Colorado, Institute of Cognitive Science, Colorado.

[17]

D. P. Kingma and J. Ba. 2014. Adam: A Method for Stochastic Optimization. ArXiv Preprint ArXiv:1412.6980.

[18]

C. M. Lee and S. S. Narayanan. 2005. Toward detecting emotions in spoken dialogs. IEEE Transactions on Speech and Audio Processing 13, 2, 293--303.

[19]

J. Liscombe, G. Riccardi, and D. Hakkani-Tur. 2005. Using context to improve emotion detection in spoken dialog systems. In Proceedings of the 9th European Conference on Speech Communication and Technology (INTERSPEECH'05).

[20]

M. Mateas and A. Stern. 2005. Structuring content in the façade interactive drama architecture. In Proceedings of the 1st Artificial Intelligence and Interactive Digital Entertainment Conference. 93--98.

Digital Library

[21]

Y. Matsumoto. 1988. Reexamination of the universality of face: Politeness phenomena in Japanese. Journal of Pragmatics 12, 4 (1988), 403--426.

[22]

M. Obaid, C. Han, and M. Billinghurst. 2008. Feed the fish: An affect-aware game. In Proceedings of the 5th Australasian Conference on Interactive Entertainment. ACM.

Digital Library

[23]

D. W. Opitz and R. Maclin. 1999. Popular ensemble methods: An empirical study. Journal of Artificial Intelligence Resources 11, 169--198.

Digital Library

[24]

J. Pennington, R. Socher, and C. D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods of Natural Language Processes. 532--543.

[25]

S. Planet and I. Iriondo. 2012. Comparison between decision-level and feature-level fusion of acoustic and linguistic features for spontaneous emotion recognition. In Proceedings of the 7th Iberian Conference on Information Systems and Technologies (CISTI). IEEE, 1--6.

[26]

R. Plutchik. 2001. The nature of emotions. American Scientist 89, 4, 344--350.

[27]

A. Popescu-Belis. 2008. Dimensionality of dialogue act tagsets: An empirical analysis of large corpora. Language Res. Eval. 42, 99--107.

[28]

J. A. Russel. 1980. A circumplex model of affect. Journal of Personality and Social Psychology 39, 6, 1161.

[29]

M. Szwoch and W. Szwoch. 2014. Emotion recognition for affect aware video games. Image Processing 8 Communications Challenges 6, 313, 227.

[30]

L. Tian, J. D. Moore, and C. Lai. 2015. Emotion recognition in spontaneous and acted dialogues. In Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction. IEEE, 698--704.

Digital Library

[31]

T. Vogt, E. Andre, and N. Bee. 2008. Emovoice—A framework for online recognition of emotions from voice. In Proceedings of the 4th Tutorial and Research Workshop on Perception in Multimodal Dialogue Systems. IEEE, 188--199.

Digital Library

[32]

Wikimedia Project Editors. 2017. Wikimedia database dump of the Japanese Wikipedia on July 20, 2016, https://archive.org/details/jawiki-20160720, Last accessed: 2017/09/04.

[33]

H. Yoon, S. Park, Y. K. Lee, and J. H. Jang. 2013. Emotion recognition of serious game players using a simple brain computer interface. In Proceedings of ICT Convergence (ICTC). IEEE, 783--786.

Cited By

Jiang XDeng NFan XJia H(2022)Examining the role of perceived value and consumer innovativeness on consumers’ intention to watch intellectual property filmsEntertainment Computing10.1016/j.entcom.2021.10045340(100453)Online publication date: Jan-2022
https://doi.org/10.1016/j.entcom.2021.100453
Saha TGupta DSaha SBhattacharyya P(2021)A Unified Dialogue Management Strategy for Multi-intent Dialogue Conversations in Multiple LanguagesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/346176320:6(1-22)Online publication date: 20-Sep-2021
https://dl.acm.org/doi/10.1145/3461763

Index Terms

A Supplementary Feature Set for Sentiment Analysis in Japanese Dialogues
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Information extraction
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Sentiment analysis

Recommendations

Distant collocations between suppositional adverbs and clause-final modality forms in Japanese language corpora
LKR'08: Proceedings of the 3rd international conference on Large-scale knowledge resources: construction and application

Co-occurring of modal adverbs and clause-final modality forms in the Japanese language exhibits a strong agreement-like behaviour. We refer to such co-occurrences as distant collocations - a notion that warrants further consideration within the fields of ...
Feature-guided Multimodal Sentiment Analysis towards Industry 4.0
Highlights
- Advanced and efficient image-text multimodal fusion approach.
- Clever use of matrix transformation to achieve alignment of different modal features.
- Using attention mechanism to ensure the parallelism of the model and improve the ...
Abstarct
Combining Artificial Intelligence (AI) to process rich media information has become an important part of Industry 4.0. Sentiment recognition in AI aims to analyze user emotions contained in rich media to facilitate service enhancement. Previous ...
Graphical abstract

Display Omitted
A multimodal sentiment recognition method based on attention mechanism
AICCC '22: Proceedings of the 2022 5th Artificial Intelligence and Cloud Computing Conference

Effective sentiment analysis on social media data can help to better understand the public's sentiment and opinion tendencies. Combining multimodal content for sentiment classification uses the correlation information of data between modalities, thereby ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 18, Issue 4

December 2019

305 pages

ISSN:2375-4699

EISSN:2375-4702

DOI:10.1145/3327969

Editor:
Nianwen Xue
Brandeis University, Waltham, USA

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2019

Accepted: 01 January 2019

Revised: 01 January 2019

Received: 01 February 2018

Published in TALLIP Volume 18, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
162
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang XDeng NFan XJia H(2022)Examining the role of perceived value and consumer innovativeness on consumers’ intention to watch intellectual property filmsEntertainment Computing10.1016/j.entcom.2021.10045340(100453)Online publication date: Jan-2022
https://doi.org/10.1016/j.entcom.2021.100453
Saha TGupta DSaha SBhattacharyya P(2021)A Unified Dialogue Management Strategy for Multi-intent Dialogue Conversations in Multiple LanguagesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/346176320:6(1-22)Online publication date: 20-Sep-2021
https://dl.acm.org/doi/10.1145/3461763

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents