research-article

A Comparative Study on Detection Accuracy of Cloud-Based Emotion Recognition Services

Authors:

Osamah M. Al-Omair,

Shihong HuangAuthors Info & Claims

SPML '18: Proceedings of the 2018 International Conference on Signal Processing and Machine Learning

Pages 142 - 148

https://doi.org/10.1145/3297067.3297079

Published: 28 November 2018 Publication History

Abstract

The ability of software systems adapting to human's input is a key element in the symbiosis of human-system co-adaptation, where human and software-based systems work together in a close partnership to achieve synergetic goals. This seamless integration will eliminate the barriers between human and machine. A critical requirement for co-adaptive systems is software system's ability to recognize human emotion, in which the system can detect and interpret users' emotions and adapt accordingly. There are numerous solutions that provide the technologies for emotion recognition. However, selecting an appropriate solution for a given task within a specific application domain can be challenging. The vast variation between these solutions makes the selecting task even more difficult. This paper compares cloud-based emotion recognition services offered by Amazon, Google, and Microsoft. These services detect human emotion through facial expression recognition with the utilization of computer vision. The focus of this paper is to measure the detection accuracy of these services. Accuracy is calculated based on the highest confidence rating returned by each service. All three services have been tested with the same dataset. This paper concludes with findings and recommendations based on our comparative analysis among these services.

References

[1]

T. T. Hewett et al., ACM SIGCHI Curricula for Human-Computer Interaction, vol. 2008, no. February 17. Association for Computing Machinery, 1992.

[2]

T. B. Sheridan, Humans and automation: System design and research issues. Santa Monica, CA, US: Human Factors and Ergonomics Society, 2002.

Digital Library

[3]

R. W. Picard, "Toward Machines With Emotional Intelligence," Sci. Emot. Intell. Knowns Unknowns, pp. 1--22, 2012.

[4]

R. W. Picard, "Affective computing," Pattern Anal. Appl., vol. 1, no. 1, pp. 71--73, 1997.

Digital Library

[5]

E. Lloyd, S. Huang, and E. Tognoli, "Improving Human-in-the-Loop Adaptive Systems Using Brain-Computer Interaction," Proc. - 2017 IEEE/ACM 12th Int. Symp. Softw. Eng. Adapt. Self-Managing Syst. SEAMS 2017, pp. 163--174, 2017.

Digital Library

[6]

Oxford Dictionaries, "Definition of Adaptability in English by Oxford Dictionaries," 2016. {Online}. Available: https://en.oxforddictionaries.com/definition/adaptability. {Accessed: 15-Mar-2018}.

[7]

R. de Lemos et al., "Software Engineering for Self-Adaptive Systems: A Second Research Roadmap," Springer, Berlin, Heidelberg, 2013, pp. 1--32.

[8]

T. Sawaragi, "Dynamical and complex behaviors in human-machine co-adaptive systems," in IFAC Proceedings Volumes (IFAC-PapersOnline), 2005, vol. 16, no. 1, pp. 94--99.

[9]

Amazon.com, "Amazon - Press Room - Press Release," Amazon.com, 1998. {Online}. Available: http://phx.corporate-ir.net/phoenix.zhtml?c=176060&p=irol-newsArticle&ID=502989&highlight. {Accessed: 08-Apr-2018}.

[10]

"Amazon Rekognition - Video and Image - AWS." {Online}. Available: https://aws.amazon.com/rekognition/. {Accessed: 15-Apr-2018}.

[11]

"Vision API - Image Content Analysis | Google Cloud." {Online}. Available: https://cloud.google.com/vision/. {Accessed: 15-Apr-2018}.

[12]

"Face API - Facial Recognition Software | Microsoft Azure." {Online}. Available: https://azure.microsoft.com/en-us/services/cognitive-services/face/. {Accessed: 15-Apr-2018}.

[13]

D. Lundqvist, A. Flykt, and A. Öhman, "The Karolinska directed emotional faces (KDEF)," CD ROM from Dep. Clin. Neurosci. Psychol. Sect. Karolinska Institutet, no. 1998, 1998.

[14]

"Python Data Analysis Library - pandas: Python Data Analysis Library." {Online}. Available: https://pandas.pydata.org/. {Accessed: 02-Jan-2018}.

[15]

P. Ekman and W. V Friesen, Unmasking the face: A guide to recognizing emotions from facial clues. Ishk, 2003.

[16]

C. Vinola and K. Vimaladevi, "A survey on human emotion recognition approaches, databases and applications," Electron. Lett. Comput. Vis. Image Anal., vol. 14, no. 2, pp. 24--44, 2015.

[17]

S. Veni and S. Thushara, "Multimodal Approach to Emotion Recognition for Enhancing Human Machine Interaction - A Survey," Int. J. Adv. Sci. Eng. Inf. Technol., vol. 7, no. 4, pp. 1428--1433, 2017.

[18]

Z. Li, J. Imai, and M. Kaneko, "Facial-component-based bag of words and PHOG descriptor for facial expression recognition," in 2009 IEEE International Conference on Systems, Man and Cybernetics, 2009, pp. 1353--1358.

Digital Library

[19]

T. Kanade, J. F. Cohn, and Yingli Tian, "Comprehensive database for facial expression analysis," in Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580), pp. 46--53.

Digital Library

[20]

I. a Essa and A. P. Pentland, "Facial expression recognition using a dynamic model and motion energy," in Proceedings of IEEE International Conference on Computer Vision, 1995, no. 307, pp. 360--367.

Digital Library

[21]

M. El Ayadi, M. S. Kamel, and F. Karray, "Survey on speech emotion recognition: Features, classification schemes, and databases," Pattern Recognit., vol. 44, no. 3, pp. 572--587, 2011.

Digital Library

[22]

S. Huang and E. Tognoli, "Brainware: synergizing software systems and neural inputs," Companion Proc. 36th Int. Conf. Softw. Eng. - ICSE Companion 2014, pp. 444--447, 2014.

Digital Library

[23]

V. Petrushin, "How well can people and computers recognize emotions in speech?," Proc. AAAI Fall Symp., pp. 141--145, 1998.

[24]

B. Schuller, R. J. Villar, G. Rigoll, and M. Lang, "Meta-Classifiers in Acoustic and Linguistic Feature Fusion-Based Affect Recognition," in Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005., vol. 1, pp. 325--328.

[25]

V. Rozgic, S. N. Vitaladevuni, and R. Prasad, "Robust EEG emotion classification using segment level decision fusion," in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2013, pp. 1286--1290.

[26]

Wei-Long Zheng, Bo-Nan Dong, and Bao-Liang Lu, "Multimodal emotion recognition using EEG and eye tracking data," 2014 36th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 5040--5043, Aug. 2014.

[27]

"IBM Cloud products." {Online}. Available: https://www.ibm.com/cloud/products. {Accessed: 04-Jul-2018}.

Cited By

Wyman AZhang Z(2025)A Tutorial on the Use of Artificial Intelligence Tools for Facial Emotion Recognition in RMultivariate Behavioral Research10.1080/00273171.2025.2455497(1-15)Online publication date: 14-Feb-2025
https://doi.org/10.1080/00273171.2025.2455497
Kumar DBhawna Narzary D(2024)Exploring the Utility of Emotion Recognition Systems in HealthcareUsing Machine Learning to Detect Emotions and Predict Human Psychology10.4018/979-8-3693-1910-9.ch011(245-271)Online publication date: 12-Apr-2024
https://doi.org/10.4018/979-8-3693-1910-9.ch011
Obwegeser DTimofte RMayer CBornstein MSchätzle MPatcas R(2024)Scoring facial attractiveness with deep convolutional neural networks: How training on standardized images reduces the bias of facial expressionsOrthodontics & Craniofacial Research10.1111/ocr.1282027:S2(25-32)Online publication date: 2-Jun-2024
https://doi.org/10.1111/ocr.12820
Show More Cited By

Index Terms

A Comparative Study on Detection Accuracy of Cloud-Based Emotion Recognition Services
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Biometrics

Recommendations

Gaussian mixture model based estimation of the neutral face shape for emotion recognition

When the goal is to recognize the facial expression of a person given an expressive image, there are mainly two types of information encoded in the image that we have to deal with: identity-related information and expression related information. ...
Anthropometric Facial Emotion Recognition
Proceedings of the 13th International Conference on Human-Computer Interaction. Part II: Novel Interaction Methods and Techniques

The aim of this project is detection, analysis and recognition of facial features. The system operates on grayscale images. For the analysis Haar-like face detector was used along with anthropometric face model and a hybrid feature detection approach. ...
Emotion Recognition Using Physiological Signals
MIDI '15: Proceedings of the Mulitimedia, Interaction, Design and Innnovation

In this paper the problem of emotion recognition using physiological signals is presented. Firstly the problems with acquisition of physiological signals related to specific human emotions are described. It is not a trivial problem to elicit real ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SPML '18: Proceedings of the 2018 International Conference on Signal Processing and Machine Learning

November 2018

177 pages

ISBN:9781450366052

DOI:10.1145/3297067

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SPML '18

SPML '18: 2018 International Conference on Signal Processing and Machine Learning

November 28 - 30, 2018

Shanghai, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
434
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)4

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wyman AZhang Z(2025)A Tutorial on the Use of Artificial Intelligence Tools for Facial Emotion Recognition in RMultivariate Behavioral Research10.1080/00273171.2025.2455497(1-15)Online publication date: 14-Feb-2025
https://doi.org/10.1080/00273171.2025.2455497
Kumar DBhawna Narzary D(2024)Exploring the Utility of Emotion Recognition Systems in HealthcareUsing Machine Learning to Detect Emotions and Predict Human Psychology10.4018/979-8-3693-1910-9.ch011(245-271)Online publication date: 12-Apr-2024
https://doi.org/10.4018/979-8-3693-1910-9.ch011
Obwegeser DTimofte RMayer CBornstein MSchätzle MPatcas R(2024)Scoring facial attractiveness with deep convolutional neural networks: How training on standardized images reduces the bias of facial expressionsOrthodontics & Craniofacial Research10.1111/ocr.1282027:S2(25-32)Online publication date: 2-Jun-2024
https://doi.org/10.1111/ocr.12820
Ho Ro SLi YGong J(2024)A Machine learning approach for Post-Disaster data curationAdvanced Engineering Informatics10.1016/j.aei.2024.10242760(102427)Online publication date: Apr-2024
https://doi.org/10.1016/j.aei.2024.102427
Al-Omair O(2023)Incorporating Eye-Tracking and Facial Expression Recognition for Preference Prediction in Recommender Systems2023 3rd International Conference on Computing and Information Technology (ICCIT)10.1109/ICCIT58132.2023.10273933(274-279)Online publication date: 13-Sep-2023
https://doi.org/10.1109/ICCIT58132.2023.10273933
Çini AJärvelä SDindar MMalmberg J(2023)How multiple levels of metacognitive awareness operate in collaborative problem solvingMetacognition and Learning10.1007/s11409-023-09358-718:3(891-922)Online publication date: 26-Sep-2023
https://doi.org/10.1007/s11409-023-09358-7
Al-Omair OHuang S(2022)An Emotional Support Robot Framework Using Emotion Recognition as Nonverbal Communication for Human-Robot Co-adaptationProceedings of the Future Technologies Conference (FTC) 2022, Volume 310.1007/978-3-031-18344-7_30(451-462)Online publication date: 14-Oct-2022
https://doi.org/10.1007/978-3-031-18344-7_30
Ahmad KWang SVogel CJain PO’Neill OSufi B(2021)Comparing the Performance of Facial Emotion Recognition Systems on Real-Life Videos: Gender, Ethnicity and AgeProceedings of the Future Technologies Conference (FTC) 2021, Volume 110.1007/978-3-030-89906-6_14(193-210)Online publication date: 24-Oct-2021
https://doi.org/10.1007/978-3-030-89906-6_14
Santamaria-Granados LMendoza-Moreno JRamirez-Gonzalez G(2020)Tourist Recommender Systems Based on Emotion Recognition—A Scientometric ReviewFuture Internet10.3390/fi1301000213:1(2)Online publication date: 24-Dec-2020
https://doi.org/10.3390/fi13010002
Al-Omair OHuang S(2019)A Comparative Study of Algorithms and Methods for Facial Expression Recognition2019 IEEE International Systems Conference (SysCon)10.1109/SYSCON.2019.8836770(1-6)Online publication date: Apr-2019
https://doi.org/10.1109/SYSCON.2019.8836770
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten