Skip to main content
SpringerLink
Log in

Social signals: a framework in terms of goals and beliefs

  • Research Report
  • Published:
Cognitive Processing Aims and scope Submit manuscript

Abstract

The paper defines the notion of social signal, in terms of a cognitive model of mind and social interaction, as a communicative or informative signal or a cue that directly or indirectly provides information about “social facts”: social interactions, social emotions, social attitudes, evaluations and stances, social relations, and social identities. This notion is compared with other connected notions in Semiotics, Ethology, and Psychology, several types of informative and communicative signals and cues are exemplified, and their process of production and interpretation is overviewed, while considering the role of context and previous knowledge in it. The implications of this model are outlined for the construction of systems for the analysis of social signals and their simulation in Virtual Agents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

References

  • Allwood J (1978) On the analysis of communicative action. In: Brenner J (ed) The structure of action. Basil Blackwell, Oxford

    Google Scholar 

  • Allwood J (2002) Bodily communication—dimensions of expression and content. In: Granström B, House D, Karlsson J (eds) Multimodality in language and speech systems. Kluwer Academic Publishers, Dordrecht, pp 7–26

    Google Scholar 

  • Allwood J (2008) Dimensions of embodied communication—towards a typology of embodied communication. In: Wachsmuth I, Lenzen M, Knoblich G (eds) Embodied communication in humans and machines. Oxford University Press, Oxford, pp 257–284

    Chapter  Google Scholar 

  • Augustine (389 a.C.). De Magistro. P. Domenico Bassi, Fiorentina 1930

  • Austin JL (1962) How to do things with words. Oxford University Press, Oxford

    Google Scholar 

  • Bargh JA, Chen M, Burrows L (1996) Automaticity of social behavior: direct effects of trait construct and stereotype activation on action. J Pers Soc Psychol 71:230–244

    Article  PubMed  CAS  Google Scholar 

  • Barsalou LW (1999) Perceptual symbol systems. Behav Brain Sci 22:577–609

    PubMed  CAS  Google Scholar 

  • Bergmann K, Kopp S (2010) Systematicity and idiosyncrasy in iconic gesture use: empirical analysis and computational modeling. In: Kopp S, Wachsmuth I (eds) Gesture in embodied communication and human-computer interaction (LNAI 5934. Springer, Berlin, pp 182–194

    Chapter  Google Scholar 

  • Bevacqua E (2009) Computational model of listener behavior for embodied conversational agents. Ph.D. dissertation, University of Paris 8

  • Bousmalis K, Mehu M, Pantic M (2009) Spotting agreement and disagreement: a survey of nonverbal audiovisual cues and tools. In: Proceedings of the international conference on affective computing and intelligent interaction

  • Brunswik E (1956) Perception and the representative design of psychological experiments. University of California Press, Berkeley

    Google Scholar 

  • Campanella S, Belin P (2007) Integrating face and voice in person perception. Trends Cogn Sci 11(12):535–543

    Article  PubMed  Google Scholar 

  • Castelfranchi C (1981) Scopi esterni. Rassegna Italiana di Sociologia XXII(1):329–381

  • Castelfranchi C (2003) The micro-macro constitution of powers. In understanding the social II. In: Tuomela R, Preyer G, Peter G, Philosophy of Sociality (eds) special issue of ProtoSociology. An International Journal of Interdisciplinary Research, pp 208–265

  • Castelfranchi C (2005) ToM and BIC: intentional behavioral communication as based on the theory of mind. In: Proceedings of the AISB symposium on social virtual agents, vol 37. pp 155–162

  • Castelfranchi C (2012). Ascribing minds. Cogn Process (this issue). doi:10.1007/s10339-011-0423-y

  • Castelfranchi C, Parisi D (1980) Linguaggio, Conoscenze e Scopi. Il Mulino, Bologna

  • Castelfranchi C, Poggi I (1987) Communication: beyond the cognitive approach and speech act theory. In: Verschueren JJ, Bertuccelli M, Papi A (eds) The pragmatic perspective. John Benjamins, Amsterdam, pp 239–254

    Google Scholar 

  • Castelfranchi C, Poggi I (1990) Blushing as a discourse: was Darwin wrong? In: Crozier R (ed) Shyness and embarrassment. Perspectives from social Psychology. Cambridge University Press, New York

    Google Scholar 

  • Castelfranchi C, Poggi I (1998) Bugie, finzioni, sotterfugi. Per una scienza dell’inganno, Carocci

    Google Scholar 

  • Chaiken S (1987) The heuristic model of persuasion. In: Zanna MP, Olson JM, Herman CP (eds) Social influence: The Ontario Symposium, vol 5. Erlbaum, Hillsdale, pp 3–39

    Google Scholar 

  • Chartrand T, Bargh J (1999) The chameleon effect: the perception-behavior link and social interaction. J Personality Soc Psychol 76(6):893–910

    Article  CAS  Google Scholar 

  • Cohen PR, Levesque HJ (1990) Rational interaction as the basis for communication. In: Cohen PR, Morgan J, Pollack ME (eds) Intentions in communication. The MIT Press, Cambridge, pp 221–256

    Google Scholar 

  • Cohn JF (2007) Foundations of human computing: facial expression and emotion. In: Huang TS, Nijholt A, Pantic M, Pentland A (eds) Artifical intelligence for human computing, LNAI 4451. Springer, Berlin, pp 1–16

    Chapter  Google Scholar 

  • Cohn JF (2010) Social signal processing in depression. In Proceedings of the 2nd international workshop on social signal processing, SSPW’10, New York, NY, USA. ACM. pp 1–2

  • Conte R, Castelfranchi C (1995) Cognitive and social action. University College, London

    Google Scholar 

  • D’Errico F, Leone G, Poggi I (2010) Types of help in the teacher’s multimodal behavior. In: Salah AA et al (eds) Human behavior understanding 2010, LNCS 6219. Springer, Heidelberg

    Google Scholar 

  • Darwin C (1872) The expression of the emotions in man and animals. Appleton and Company, New York

    Book  Google Scholar 

  • de Saussure F (1922) Cours de Linguistique générale. Payot, Paris

    Google Scholar 

  • Dunbar NE, Burgoon JK (2005) Perceptions of power and interactional dominance in interpersonal relationships. J Soc Pers Relatsh 22(2):207–233

    Article  Google Scholar 

  • Ekman P (1985) Telling lies. Norton, New York

    Google Scholar 

  • Fazio RH (1995) Attitudes as object–evaluation associations: Determinants, consequences, and correlates of attitude accessibility. In: Petty RE, Krosnick JA (eds) Attitude strength: antecedents and consequences. Lawrence Erlbaum Associates, Mahwah, pp 247–282

    Google Scholar 

  • Frank MG, Ekman P (1993) Not all smiles are created equal the differences between enjoyment and nonenjoyment smiles. Humor 6:9–26

    Article  Google Scholar 

  • Gensini S (2004) Manuale di semiotica. Carocci, Roma

    Google Scholar 

  • Goel V, Bruchel C, Frith C, Dolan R (2000) Dissociation of mechanisms underlying syllogistic reasoning”. Neuroimage 12(5):504–514

    Article  PubMed  CAS  Google Scholar 

  • Grice HP (1957) Meaning. The philosophical review 66:377–388

    Article  Google Scholar 

  • Grice HP (1975) Logic and conversation. In: Cole P, Morgan JL (eds) Syntax and semantics, speech acts, vol III. Academic Press, New York

    Google Scholar 

  • Hasson O (1994) Cheating signals. J Theor Biol 167:223–238

    Article  Google Scholar 

  • Kendon A (2004) Gesture. Visible action as utterance. Cambridge University Press, Cambridge

    Google Scholar 

  • Kita S, Ozyürek A (2003) What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. J Mem Lang 48:16–32

    Article  Google Scholar 

  • Kopp S, Tepper P, Cassell J (2004) Towards integrated microplanning of language and iconic gesture for multimodal output. In: Proceedings of the international conference on multimodal interfaces (ICMI’04). ACM Press, pp 97–104

  • Kopp S, Bergmann K, Wachsmuth I (2008) Multimodal communication from multimodal thinking—towards an integrated model of speech and gesture production. Int J Semant Comput 2(1):115–136

    Article  Google Scholar 

  • Krebs JR, Dawkins R (1984) Animal signals: mind-reading and manipulation. In: Krebs JR, Davies NB (eds) Behavioural Ecology: an evolutionary approach, vol 2. Blackwell Scientific Publications, Oxford, pp 380–402

    Google Scholar 

  • Kröger B, Kopp S, Lowit A (2010) A model for production, perception, and acquisition of actions in face-to-face communication. Cogn Process 11(3):187–205

    Article  PubMed  Google Scholar 

  • Lakin JL (2006) Automatic cognitive processes and nonverbal communication. In: Manusov V, Patterson ML (eds), The sage handbook of nonverbal communication, pp 59–77

  • Leary MR, Britt TW, Cutlip WD II, Templeton JL (1992) Social blushing. Psychol Bull 112:446–460. doi:10.1037/0033-2909.112.3.446

    Article  PubMed  CAS  Google Scholar 

  • Levelt WJM (1999) Models of word production. Trends Cogn Sci 3(6):223–232

    Article  PubMed  Google Scholar 

  • Mancini M, Pelachaud C (2010) Generating distinctive behavior for Embodied Conversational Agents. J Multimodal User Interfaces pp 1–13. doi:10.1007/s12193-010-0048-y. ISSN 1783-7677

  • Matsumoto D (2006) Culture and nonverbal behavior. In: Manusov V, Patterson M (eds) The Handbook of nonverbal communication. Sage, Thousand Oaks, pp 219–235

    Chapter  Google Scholar 

  • Maynard-Smith J, Harper DG (1995) Animal signals: models and terminology. J Theor Biol 177:305–311

    Article  Google Scholar 

  • Maynard-Smith J, Harper DG (2003) Animal signals. Oxford University Press, Oxford

    Google Scholar 

  • Mehu M, Scherer KR (2012) A psycho-ethological approach to social signal processing. Cogn Process. doi:10.1007/s10339-012-0435-2

  • Meredith MA (2002) On the neuronal basis for multisensory convergence: a brief overview. Cogn Brain Res 14:31–40

    Article  Google Scholar 

  • Miceli M, Castelfranchi C (1995) Le difese della mente. La Nuova Italia Scientifica, Roma

    Google Scholar 

  • Morton J, Patterson KE (1980) A new attempt at an interpretation, or, an attempt at a new interpretation. In: Coltheart M, Patterson KE, Marshall JC (eds) Deep dyslexia. Routledge & Kegan Paul, London, pp 91–118

    Google Scholar 

  • Oppenheim AV, Schafer RW (1975) Digital signal processing. Prentice Hall, New Jersey

    Google Scholar 

  • Paivio A (2007) Mind and its evolution: a dual coding theoretical approach. Lawrence Erlbaum Associates, Mahwah

    Google Scholar 

  • Patterson ML (1991) A functional approach to nonverbal exchange. In: Feldman R, Rimé B (eds) Fundamentals of nonverbal behavior. Cambridge University Press, Cambridge

    Google Scholar 

  • Patterson ML (2001) Toward a comprehensive model of nonverbal communication. In: Robinson WP, Giles H (eds) The new handbook of language and social psychology. Wiley & Sons, Chichester, pp 159–176

    Google Scholar 

  • Peirce C (1977) Semiotics and Significs. Charles Hardwick (ed) Indiana University Press, Bloomington

  • Pelachaud C (2009) Modelling multimodal expression of emotion in a virtual agent. Philos Trans Royal Soc 364:3539–3548

    Article  Google Scholar 

  • Pentland A (2008) Honest signals. How do they shape our world. MIT Press, Cambridge

    Google Scholar 

  • Poggi I (2007a) Mind, hands, face and body. A goal and belief view of multimodal communication. Weidler, Berlin

    Google Scholar 

  • Poggi I (2007b) Enthusiasm and its contagion. Nature and function. In: Paiva A, Prada R, Picard RW (eds) Affective computing and intelligent interaction. Springer, Berlin, pp 410–421

    Chapter  Google Scholar 

  • Poggi I, D’Errico F (2010a) Cognitive modelling of human social signals. In: Proceedings of SSPW 2010—social signal processing workshop. ACM, Sheridan Press, New Jersey

  • Poggi I, D’Errico F (2011) Social signals: A psychological perspective. In: Salah AA, Gevers T (eds) Computer analysis of human behavior. Springer Verlag’s Advances in Pattern Recognition series, pp 185–225

  • Poggi I, Pelachaud C (2008) Persuasive gestures and the expressivity of ECAs. In: Wachsmuth I, Lenzen M, Knoblich G (eds) Embodied communication in humans and machines. Oxford University Press, Oxford, pp 391–424

    Chapter  Google Scholar 

  • Poggi I, D’Errico F, Vincze L (2010) Types of nods. The polysemy of a social signal. In: Calzolari N et al. (ed) Proceedings of the seventh conference on international language resources and evaluation (LREC’10). Malta 19-21 May. European Language Resources Association (ELRA), pp 2570–2576. ISBN:2-9517408-6-7

  • Poggi I, D’Errico F (2010b) Dominance in political debates. In: Salah AA et al (eds) HBU 2010, LNCS 6219. Springer, Heidelberg, pp 163–174

    Google Scholar 

  • Posner R (1993) Believing, causing, intending: the basis for a hierarchy of sign concepts in the reconstruction of communication. In: Jorna RJ, van Heusden J, Posner R (eds) Signs, search, and communication: semiotic aspects of artificial intelligence. De Gruyter, Berlin, pp 215–270

    Google Scholar 

  • Posner MI, Snyder CRR (1975) Attention and cognitive control. In: Solso RL (ed) Information processing and cognition: the loyola symposium. Erlbaum, Hillsdale, pp 55–85

    Google Scholar 

  • Rendall D, Owren R, Ryan MJ (2009) What do animal signals mean? Anim Behav 78(2):233–240

    Article  Google Scholar 

  • Scheflen AE (1974) How behavior means. Aronson

  • Scherer KR (1978) Personality inference from voice quality: the loud voice of extroversion. Eur J Soc Psychol 8(4):467–487

    Article  Google Scholar 

  • Scherer KR (2003) Vocal communication of emotion: a review of research paradigms. Speech Commun 40:227–256

    Article  Google Scholar 

  • Searle JR (1969) Speech Acts: an essay in the philosophy of language. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Shannon CE, Weaver W (1949) The mathematical theory of communication. University of Illinois, Urbana

    Google Scholar 

  • Sperber D, Wilson D (1986/1995a). Relevance: communication and cognition, 2nd edn. Blackwell, Oxford

  • Stanovich W, West R (2000) Individual difference in reasoning: implications for the rationality debate? Behav Brain Sci 23:645–726

    Article  PubMed  CAS  Google Scholar 

  • Strawson PF (1964) Intention and convention in speech acts. Philos Rev 73:439–460

    Article  Google Scholar 

  • Tiedens LA, Fragale AR (2003) Power moves: complementarity in dominant and submissive nonverbal behavior. J Personality Soc Psychol 84:558–568

    Article  Google Scholar 

  • Tinbergen N (1951) The study of instinct. Clarendon Press, Oxford

    Google Scholar 

  • Tomasello M (2008) Origins of human communication. The MIT press, Cambridge

    Google Scholar 

  • Vinciarelli A, Pantic M, D’Errico F, Heylen D, Pantic M, Pelachaud C, Poggi I, Schroeder M (2012) Bridging the gap between social animal and unsocial machine: a survey of social signal processing. IEEE Trans Affect Comput 3(1):69–87

    Article  Google Scholar 

  • Watzlawick P, Beavin Bavelas J, Jackson DD (1967) Pragmatics of human communication. W.W. Norton and Co, NY

  • Wittgenstein L (1953) Philosophical investigations. In: Anscombe GEM, Rhees ER (eds) Basil. Blackwell, Oxford

    Google Scholar 

  • Zahavi A, Zahavi A (1997) The handicap principle: a missing part of Darwin’s puzzle. Oxford University Press, Oxford

    Google Scholar 

Download references

Acknowledgments

This research is supported by the Seventh Framework Program, European Network of Excellence SSPNet (Social Signal Processing Network), Grant Agreement N. 231287. We are indebted to anonymous reviewers who in their reviews of the paper provided important feedback and suggestion.

Author information

Authors and Affiliations

  1. Department of Education, Roma Tre University, Rome, Italy

    Isabella Poggi & Francesca D’Errico

Authors
  1. Isabella Poggi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesca D’Errico
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesca D’Errico.

Additional information

This article is part of the Supplement Issue on 'Social Signals. From Theory to Applications', guest-edited by Isabella Poggi, Francesca D'Errico, and Alessandro Vinciarelli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Poggi, I., D’Errico, F. Social signals: a framework in terms of goals and beliefs. Cogn Process 13 (Suppl 2), 427–445 (2012). https://doi.org/10.1007/s10339-012-0512-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10339-012-0512-6

Keywords

Search

Navigation