Aggeliki Vlachostergiou
ABSTRACT
Recent advances in Affective Computing (AC) include research towards automatic analysis of human emotionally enhanced behavior during multiparty interactions within different contextual settings. Current paper delves on how is context incorporated into multiparty and multimodal interaction within the AC framework. Aspects of context incorporation such as importance and motivation for context incorporation, appropriate emotional models, resources of multiparty interactions useful for context analysis, context as another modality in multimodal AC and context-aware AC systems are addressed as research questions reviewing the current state-of-the-art in the research field. Challenges that arise from the incorporation of context are identified and discussed in order to foresee future research directions in the domain. Finally, we propose a context incorporation architecture into affect-aware systems with multiparty interaction including detection and extraction of semantic context concepts, enhancing emotional models with context information and context concept representation in appraisal estimation.
- G. D. Abowd, A. K. Dey, P. J. Brown, N. Davies, M. Smith, and P. Steggles. Towards a better understanding of context and context-awareness. In Handheld and Ubiquitous Computing, pages 304--307. Springer, 1999. Google Scholar
- S. Al Moubayed, J. Edlund, and J. Gustafson. Analysis of gaze and speech patterns in three-party quiz game interaction. In INTERSPEECH, pages 1126--1130, 2013.Google Scholar
- S. O. Ba and J.-M. Odobez. Multi-party focus of attention recognition in meetings from head pose and multimodal contextual cues. In International Conference on Acoustics, Speech and Signal Processing,ICASSP, pages 2221--2224. IEEE, 2008.Google ScholarCross Ref
- S. O. Ba and J.-M. Odobez. Recognizing visual focus of attention from head pose in natural meetings. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 39(1):16--33, 2009. Google ScholarDigital Library
- R. Bock, A. Wendemuth, S. Gluge, and I. Siegert. Annotation and classification of changes of involvement in group conversation. In Proceedings of the Humaine Association Conference on Affective Computing and Intelligent Interaction, pages 803--808. IEEE, 2013. Google ScholarDigital Library
- F. Bonin, R. Bock, and N. Campbell. How do we react to context? annotation of individual and group engagement in a video corpus. In Privacy, Security, Risk and Trust (PASSAT), International Conference on Social Computing (SocialCom), pages 899--903. IEEE, 2012. Google ScholarDigital Library
- D. Borth, R. Ji, T. Chen, T. Breuel, and S.-F. Chang. Large-scale visual sentiment ontology and detectors using adjective noun pairs. In Proceedings of the 21st ACM International Conference on Multimedia, pages 223--232. ACM, 2013. Google ScholarDigital Library
- P. J. Brown, J. D. Bovey, and X. Chen. Context-aware applications: from the laboratory to the marketplace. Personal Communications, 4(5):58--64, 1997.Google ScholarCross Ref
- R. A. Calvo and S. D'Mello. Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1):18--37, 2010. Google ScholarDigital Library
- J. M. Carroll and J. A. Russell. Do facial expressions signal specific emotions? judging emotion from the face in context. Journal of Personality and Social Psychology, 70(2):205, 1996.Google ScholarCross Ref
- T. Choudhury and A. Pentland. Modeling face-to-face communication using the sociometer. 5:3--8, 2003.Google Scholar
- J. R. Curhan and A. Pentland. Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. Journal of Applied Psychology, 92(3):802, 2007.Google ScholarCross Ref
- A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. Collecting large, richly annotated facial-expression databases from movies. IEEE MultiMedia, 19(3):0034, 2012. Google ScholarDigital Library
- Z. Duric, W. D. Gray, R. Heishman, F. Li,A. Rosenfeld, M. J. Schoelles, C. Schunn, and H. Wechsler. Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction. Proceedings of the IEEE, 90(7):1272--1289, 2002.Google ScholarCross Ref
- {15} S. Favre, H. Salamin, J. Dines, and A. Vinciarelli. Role recognition in multiparty recordings using social affiliation networks and discrete distributions. In Proceedings of the 10th international conference on Multimodal interfaces, pages 29--36. ACM, 2008. Google ScholarDigital Library
- D. Gatica-Perez. Analyzing group interactions in conversations: a review. In International Conference on Multisensor Fusion and Integration for Intelligent Systems, pages 41--46. IEEE, 2006.Google ScholarCross Ref
- D. Gatica-Perez. Automatic nonverbal analysis of social interaction in small groups: A review. Image and Vision Computing, 27(12):1775--1787, 2009. Google ScholarDigital Library
- H. Gunes and B. Schuller. Categorical and dimensional affect analysis in continuous input: Current trends and future directions. Image and Vision Computing, 31(2):120--136, 2013. Google ScholarDigital Library
- N. Jovanovic and R. op den Akker. Towards automatic addressee identification in multi-party dialogues. Association for Computational Linguistics, 2004.Google Scholar
- M. Koutsombogera, S. A. Moubayed, B. Bollepalli, A. H. Abdelaziz, M. Johansson, J. D. Á. Lopes, J. Novikova, C. Oertel, K. Stefanov, and G. Varol. The tutorbot corpus; a corpus for studying tutoring behaviour in multiparty face-to-face spoken dialogue. In LREC, pages 4196--4201, 2014.Google Scholar
- D. Lenat. The dimensions of context-space. Cycorp Technical Report, 1998.Google Scholar
- H. Liu and P. Singh. Conceptnet-a practical commonsense reasoning toolkit. BT technology journal, 22(4):211--226, 2004. Google ScholarDigital Library
- T. Masuda, P. C. Ellsworth, B. Mesquita, J. Leu, S. Tanida, and E. Van de Veerdonk. Placing the face in context: cultural differences in the perception of facial emotion. Journal of personality and social psychology, 94(3):365, 2008.Google Scholar
- G. A. Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39--41, 1995. Google ScholarDigital Library
- M. Mortillaro, B. Meuleman, and K. R. Scherer. Advocating a componential appraisal model to guide emotion recognition. International Journal of Synthetic Emotions (IJSE), 3(1):18--32, 2012. Google ScholarDigital Library
- M. Pantic, A. Nijholt, A. Pentland, and T. S. Huanag. Human-centred intelligent human-computer interaction (hci2): how far are we from attaining it? International Journal of Autonomous and Adaptive Communications Systems, 1(2):168--187, 2008. Google ScholarDigital Library
- F. Pianesi, N. Mana, A. Cappelletti, B. Lepri, and M. Zancanaro. Multimodal recognition of personality traits in social interactions. In Proceedings of the 10th international conference on Multimodal interfaces, pages 53--60. ACM, 2008. Google ScholarDigital Library
- N. S. Ryan, J. Pascoe, and D. R. Morse. Enhanced reality fieldwork: the context-aware archaeological assistant. In V. Gaffney, M. van Leusen, and S. Exxon, editors, Computer applications in archaeology, British Archaeological Reports, pages 182--196. Tempus Reparatum, 1998.Google Scholar
- H. Salamin and A. Vinciarelli. Automatic role recognition in multiparty conversations: An approach based on turn organization, prosody, and conditional random fields. IEEE Transactions on Multimedia, 14(2):338--345, 2012.Google ScholarDigital Library
- B. Schilit, N. Adams, and R. Want. Context-aware computing applications. In Proceedings of the First Workshop on Mobile Computing Systems and Applications, pages 85--90. IEEE, 1994. Google ScholarDigital Library
- M. Soleymani, M. Larson, T. Pun, and A. Hanjalic. Corpus development for affective video indexing. IEEE Transactions on Multimedia, 16(4), 2014.Google ScholarDigital Library
- M. Soleymani and M. Pantic. Human-centered implicit tagging: Overview and perspectives. In International Conference on Systems, Man, and Cybernetics (SMC), pages 3304--3309. IEEE, 2012.Google ScholarCross Ref
- P. J. Stone, D. C. Dunphy, M. S. Smith, and D. M. Ogilvie. The General Inquirer: A Computer Approach to Content Analysis. MIT Press, Cambridge, MA, 1966.Google Scholar
- F. Talantzis, A. Pnevmatikakis, and A. G. Constantinides. Audio-visual active speaker tracking in cluttered indoors environments. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 38(3):799--807, 2008. Google ScholarDigital Library
- A. Vinciarelli. Speakers role recognition in multiparty audio recordings using social network analysis and duration distribution modeling. IEEE Transactions on Multimedia, 9(6):1215--1226, 2007. Google ScholarDigital Library
- A. Vinciarelli, A. Dielmann, S. Favre, and H. Salamin. Canal9: A database of political debates for analysis of social interactions. In Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction,(ACII), pages 1--4. IEEE, 2009.Google ScholarCross Ref
- A. Vinciarelli, H. Salamin, and M. Pantic. Social signal processing: Understanding social interactions through nonverbal behavior analysis. In Proceedings of International Workshop on Computer Vision and Pattern Recognition for Human Behavior Workshops, pages 42--49. IEEE, 2009.Google ScholarCross Ref
- M. Wöllmer, F. Eyben, B. W. Schuller, and G. Rigoll. Temporal and situational context modeling for improved dominance recognition in meetings. In Proc. of Interspeech 2012, Portland, Oregon, USA, pages 350--353. ISCA, 2012.Google Scholar
- B. Wrede and E. Shriberg. The relationship between dialogue acts and hot spots in meetings. In Proc. IEEE Automatic Speech Recognition and Understanding Workshop, (ASRU'03), pages 180--185, 2003.Google ScholarCross Ref
- Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1):39--58, 2009. Google ScholarDigital Library
- A. Zimmermann, A. Lorenz, and R. Oppermann. An operational definition of context. In Modeling an Using Context, pages 558--571. Springer Berlin Heidelberg, 2007. Google ScholarDigital Library
Index Terms
- Context in Affective Multiparty and Multimodal Interaction: Why, Which, How and Where?
Recommendations
Automatic understanding of affective and social signals by multimodal mimicry recognition
ACII'11: Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part IIHuman mimicry is one of the important behavioral cues displayed during social interaction that inform us about the interlocutors' interpersonal states and attitudes. For example, the absence of mimicry is usually associated with negative attitudes. A ...
Multimodal embodied mimicry in interaction
COST'10: Proceedings of the 2010 international conference on Analysis of Verbal and Nonverbal Communication and EnactmentNonverbal behavior plays an important role in human-human interaction. One particular kind of nonverbal behavior is mimicry. Behavioral mimicry supports harmonious relationships in social interaction through creating affiliation, rapport, and liking ...
Toward multimodal fusion of affective cues
HCM '06: Proceedings of the 1st ACM international workshop on Human-centered multimediaDuring face to face communication, it has been suggested that as much as 70% of what people communicate when talking directly with others is through paralanguage involving multiple modalities combined together (e.g. voice tone and volume, body language)...
Comments