Francesca Bonin
ABSTRACT
In this paper we discuss three topics central to discussion of the future of multimodal research -- genre differentiation, stardardization of annotation, and integration of social and verbal context.
- J. F. Allen, D. K. Byron, M. Dzikovska, G. Ferguson, L. Galescu, and A. Stent. Toward conversational human-computer interaction. AI magazine, 22(4):27, 2001. Google Scholar
Digital Library
- J. Allwood, M. Björnberg, L. Grönqvist, E. Ahlsén, and C. Ottesjö. The spoken language corpus at the department of linguistics, göteborg university. In FQS - Forum Qualitative Social Research, volume 1, 2000.Google Scholar
- J. Allwood, L. Cerrato, K. Jokinen, C. Navarretta, and P. Paggio. A coding scheme for the annotation of feedback, turn management and sequencing phenomena. In Proc. of the LREC Workshop on Multi-modal Corpora. From Multi-modal Behaviour to Usable Models. Genova, Italy, pages 38--42, 2006.Google Scholar
- A. Anderson, M. Bader, E. Bard, E. Boyle, G. Doherty, S. Garrod, S. Isard, J. Kowtko, J. McAllister, J. Miller, et al. The HCRC map task corpus. Language and speech, 34(4):351--366, 1991.Google ScholarCross Ref
- R. Baker and V. Hazan. LUCID: a corpus of spontaneous and read clear speech in british english. In Proceedings of the DiSS-LPSS Joint Workshop 2010, 2010.Google Scholar
- M. M. Bakhtine. Speech genres and other late essays. Number 8. University of Texas Press, 1986.Google Scholar
- BNC-Consortium. British national corpus. URL http://www. hcu. ox. ac. uk/BNC, 2000.Google Scholar
- H. C. Bunt. Dialogue control functions and interaction design. NATO ASI SERIES F COMPUTER AND SYSTEMS SCIENCES, 142:197--197, 1995.Google Scholar
- N. Campbell. Speech & expression; the value of a longitudinal corpus. In LREC, 2004.Google Scholar
- N. Campbell. Approaches to conversational speech rhythm: Speech activity in two-person telephone dialogues. In Proc XVIth International Congress of the Phonetic Sciences, Saarbrucken, Germany, pages 343--348, 2007.Google Scholar
- M. Core and J. Allen. Coding dialogs with the DAMSL annotation scheme. In AAAI fall symposium on communicative action in humans and machines, pages 28--35, 1997.Google Scholar
- J. W. DuBois, W. L. Chafe, C. Meyer, and S. A. Thompson. Santa Barbara Corpus of Spoken American English. CD-ROM. Philadelphia: Linguistic Data Consortium. 2000.Google Scholar
- R. I. M. Dunbar. The social brain: Mind, language, and society in evolutionary perspective. Annual Review of Anthropology, pages 163--181, 2003.Google ScholarCross Ref
- J. Edlund, J. Beskow, K. Elenius, K. Hellmer, S. Strömbergsson, and D. House. Spontal: A swedish spontaneous dialogue corpus of audio, video and motion capture. In LREC, 2010.Google Scholar
- E. Gilmartin and N. Campbell. More than just words: Building a chatty robot. In Natural Interaction with Robots, Knowbots and Smartphones, pages 179--185. Springer, 2014.Google Scholar
- J. J. Godfrey, E. C. Holliman, and J. McDaniel. SWITCHBOARD: Telephone speech corpus for research and development. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on, volume 1, pages 517--520, 1992. Google ScholarDigital Library
- S. Greenbaum. ICE: The international corpus of english. English Today, 28(7.4):3--7, 1991.Google Scholar
- ISO. 24617--2 (2010) language resource management: Semantic annotation framework (SemAF), part 2: Dialogue acts. ISO, Geneva, January, 2010.Google Scholar
- A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gelbart, N. Morgan, B. Peskin, T. Pfau, E. Shriberg, and A. Stolcke. The ICSI meeting corpus. In Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP'03). 2003 IEEE International Conference on, volume 1, pages 1--364, 2003.Google Scholar
- S. Kim, M. Filippone, F. Valente, and A. Vinciarelli. Predicting the conflict level in television political debates: an approach based on crowdsourcing, nonverbal communication and gaussian processes. In Proceedings of the 20th ACM international conference on Multimedia, pages 793--796. ACM, 2012. Google ScholarDigital Library
- S. Larsson and D. R. Traum. Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural language engineering, 6(3&4):323--340, 2000. Google ScholarDigital Library
- J. Laver. Communicative functions of phatic communion. Organization of behavior in face-to-face interaction, page 215--238, 1975.Google Scholar
- J. L. Lemke. Analyzing verbal data: Principles, methods, and problems. In Second International Handbook of Science Education, pages 1471--1484. Springer, 2012.Google ScholarCross Ref
- B. Malinowski. The problem of meaning in primitive languages. Supplementary in the Meaning of Meaning, pages 1--84, 1923.Google Scholar
- I. McCowan, J. Carletta, W. Kraaij, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, and V. Karaiskos. The AMI meeting corpus. In Proceedings of the 5th International Conference on Methods and Techniques in Behavioral Research, volume 88, 2005.Google Scholar
- P. Paggio, J. Allwood, E. Ahlsén, and K. Jokinen. The NOMCO multimodal nordic resource-goals and characteristics. 2010.Google Scholar
- M. Pantic, R. Cowie, F. D'Errico, D. Heylen, M. Mehu, C. Pelachaud, I. Poggi, M. Schroeder, and A. Vinciarelli. Social signal processing: The research agenda. In T. B. Moeslund, A. Hilton, V. Krüger, and L. Sigal, editors, Visual Analysis of Humans, pages 511--538. Springer Verlag, London, 2011.Google ScholarCross Ref
- H. Sacks, E. Schegloff, and G. Jefferson. A simplest systematics for the organization of turn-taking for conversation. Language, pages 696--735, 1974.Google Scholar
- H. Salamin, A. Polychroniou, and A. Yinciarelli. Automatic recognition of personality and conflict handling style in mobile phone conversations. In Image Analysis for Multimedia Interactive Services (WIAMIS), 2013 14th International Workshop on, pages 1--4, July 2013.Google ScholarCross Ref
- B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, F. Weninger, F. Eyben, E. Marchi, et al. The interspeech 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism. 2013.Google Scholar
- K. J. Van Engen, M. Baese-Berk, R. E. Baker, A. Choi, M. Kim, and A. R. Bradlow. The wildcat corpus of native-and foreign-accented english: Communicative efficiency across conversational dyads with varying language alignment profiles. Language and Speech, 53(4):510--540, 2010.Google ScholarCross Ref
- A. Vinciarelli, M. Pantic, and H. Bourlard. Social signal processing: Survey of an emerging domain. Image and Vision Computing, 27(12):1743--1759, 2009. Visual and multimodal analysis of human spontaneous behaviour:. Google ScholarDigital Library
- A. Vinciarelli, H. Salamin, A. Polychroniou, G. Mohammadi, and A. Origlia. From nonverbal cues to perception: personality and social attractiveness. In Cognitive Behavioural Systems, pages 60--72. Springer, 2012. Google ScholarDigital Library
Index Terms
- Topics for the Future: Genre Differentiation, Annotation, and Linguistic Content Integration in Interaction Analysis
Recommendations
Automatic understanding of affective and social signals by multimodal mimicry recognition
ACII'11: Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part IIHuman mimicry is one of the important behavioral cues displayed during social interaction that inform us about the interlocutors' interpersonal states and attitudes. For example, the absence of mimicry is usually associated with negative attitudes. A ...
Multi-party interaction with a virtual character and a human-like robot
VRST '13: Proceedings of the 19th ACM Symposium on Virtual Reality Software and TechnologyResearch on interactive virtual characters and social robots focuses mainly on one-to-one interactions and multi-party interactions concept are rather less explored. As we are developing these characters to be helpful to us in our daily lives as guides, ...
Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour
IUI '18: Proceedings of the 23rd International Conference on Intelligent User InterfacesRapport, the close and harmonious relationship in which interaction partners are "in sync" with each other, was shown to result in smoother social interactions, improved collaboration, and improved interpersonal outcomes. In this work, we are first to ...
Comments