Skip to main content
SpringerLink
Log in

Modeling Multimodal Communication as a Complex System

  • Conference paper
Modeling Communication with Robots and Virtual Humans

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4930))

Abstract

The overall behavior and nature of complex natural systems is in large part determined by the number and variety of the mechanisms involved – and the complexity of their interactions. Embodied natural communication belongs to this class of systems, encompassing many cognitive mechanisms that interact in highly complex ways, both within and between communicating individuals, constituting a heterogeneous, large, densely-coupled system (HeLD). HeLDs call for finer model granularity than other types of systems, lest we risk them to be not only incomplete but likelyincorrect. Consequently, models of communication must encompass a large subset of the functions and couplings that make up the real system, calling for a powerful methodology for integrating information from multiple fields and for producing runnable models. In this paper I propose such an approach, abstract module hierarchies, that leverages the benefits of modular construction without forcing modularity on the phenomena being modeled.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Gerver, D.: Simultaneous listening and speaking and retention of prose. Quart. J. Exp. Psych. 26(3), 337–341 (1974)

    Article  Google Scholar 

  2. Lee, T.: Simultaneous Listening and Speaking in English into Korean Simultaneous Interpretation. Meta 44(4) (1999), http://www.erudit.org/revue/meta/1999/v44/n4

  3. Simon, H.A.: Can there be a science of complex systems? In: Bar-Yam, Y. (ed.) Unifying themes in complex systems: Proceedings from the International Conference on Complex Systems, 1997, Cambridge, MA, vol. 1997, pp. 4–14. Perseus Press (1999)

    Google Scholar 

  4. Magnusson, M.S.: Understanding Social Interaction: Discovering Hidden Structure with Model and Algorithms. In: Anolli, L., Duncan Jr, S., Magnusson, M.S., Riva, G. (eds.) The Hidden Structure of Interaction: From Neurons to Culture Patterns, IOS Press, Amsterdam (2005)

    Google Scholar 

  5. Card, S.K., Moran, T.P., Newell, A.: The Model Human Processor: An Engineering Model of Human Performance. In: Boff, K.R., Kaufman, L., Thomas, J.P. (eds.) Handbook of Human Perception Vol. II, John Wiley and Sons, New York (1986)

    Google Scholar 

  6. Chandrasekaran, B., Josephson, S.G.: Architecture of Intelligence: The Problems and Current Approaches to Solutions. In: Honavar, V., Uhr, L. (eds.) Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, Academic Press, San Diego (1994)

    Google Scholar 

  7. Lieberman, M.D.: Reflective and Reflexive Judgment Processes: A Social Cognitive Neuroscience Approach. In: Forgas, J.P., Williams, K.R., von Hippel, W. (eds.) Social judgments: Implicit and explicit processes, pp. 44–67. Cambridge University Press, New York (2003)

    Google Scholar 

  8. Nanayakkara, T., Shadmehr, R.: Saccade Adaptation in Response to Altered Arm Dynamics. J. Neurophysiol 90, 4016–4021 (2003)

    Article  Google Scholar 

  9. Spivey, M.J., Tannenhaus, M.K., Eberhard, M.K., Sedivy, J.K.: Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution. Cogn. Psych. 45, 447–481 (2002)

    Article  Google Scholar 

  10. Thórisson, K.R.: Computational Characteristics of Multimodal Dialogue. In: AAAI Fall Symposium on Embodied Language and Action, Massachusetts Institute of Technology, Cambridge, MA, November 10-12, pp. 102–108 (1995)

    Google Scholar 

  11. O’Connell, D.C., Kowal, S., Kaltenbacher, E.: Turn-Taking: A Critical Analysis of the Research Tradition. Journal of Psycholinguistic Research 19(6), 345–373 (1990)

    Article  Google Scholar 

  12. Sacks, H., Schegloff, E.A., Jefferson, G.A.: A Simplest Systematics for the Organization of Turn-Taking in Conversation. Language 50, 696–735 (1974)

    Article  Google Scholar 

  13. Wison, M., Wilson, T.P.: An oscillator model of the timing of turn-taking. Psychonomic Bulletin and Review 12(6), 957–968 (2005)

    Google Scholar 

  14. Goodwin, C.: Conversational Organization: Interaction Between Speakers and Hearers. Academic Press, New York (1981)

    Google Scholar 

  15. Duncan Jr, S.: Some Signals and Rules for Taking Speaking Turns in Conversations. J. of Personality and Soc. Psych. 23(2), 283–292 (1972)

    Article  Google Scholar 

  16. Wolfram, S.: A New Kind of Science. Wolfram Media (2002)

    Google Scholar 

  17. Levi-Strauss, C.: The family. In: Shapiro, H. (ed.) Man, culture and society, Oxford University Press, Oxford (1956)

    Google Scholar 

  18. Popper, C.: Conjectures and Refutations. In: The Growth of Scientific Knowledge, Routledge, London (1963)

    Google Scholar 

  19. Newell, A.: You can’t play 20 questions with nature and win. In: Chase, W.G. (ed.) Visual information processing, Academic Press, New York (1973)

    Google Scholar 

  20. Kosslyn, S.M.: You can play 20 questions with nature and win: Categorical versus coordinate spatial relations as a case study. Neuropsychologia 44(9), 1519–1523 (2006)

    Article  Google Scholar 

  21. Turing, A.M.: Computing machinery and intelligence. Mind 59, 433–460 (1950)

    Article  MathSciNet  Google Scholar 

  22. Turing, A.M.: On Computable Numbers, With an Application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, Series 2 42 (1936)

    Google Scholar 

  23. Laughlin, S.B.: The Implications of Metabolic Energy Requirements for the Representation of Information in Neurons. In: Gazzaniga, M.S. (ed.) The Cognitive Neurosciences III, pp. 187–196. M.I.T. Press, Massachusetts (2004)

    Google Scholar 

  24. Thagard, P.: Mind: Introduction to Cognitive Science, 2nd edn. MIT Press, Cambridge (1996)

    Google Scholar 

  25. Chalmers, D.: Does a Rock Implement a Finite State Automaton? Synthese 108, 310–333 (1996)

    Article  MathSciNet  Google Scholar 

  26. Chalmers, D.J.: A Computational Foundation for the Study of Cognition. Philosophy-Neuroscience-Psychology Technical Report 94–03, Washington University (1994)

    Google Scholar 

  27. Calabretta, R., Parisi, D.: Evolutionary Connectionism and Mind/Brain Modularity. In: Callebaut, W., Rasskin-Gutman, D. (eds.) Modularity: Understanding the Development and Evolution of Natural Complex Systems, pp. 309–330. MIT Press, Cambridge (2005)

    Google Scholar 

  28. Takahashi, K.: Development of Holistic Climate Simulation Codes for a non-Hydrostatic Atmosphere-Ocean Coupled Systems. In: Annual Report of the Earth Simulator Cente, Japan, April 2004 – March 2005, pp. 52–67 (2005)

    Google Scholar 

  29. Vlachos, D.G.: A review of multiscale analysis: Examples from systems biology, materials engineering, and other fluid-surface interacting systems. Adv. Chem. Eng. 30, 1–61 (2005)

    Article  Google Scholar 

  30. Abadi, M.G., Navarro, J.F., Steinmetz, M., Eke, V.R.: Simluations of Galaxy Formation in a Lambda CDM Universe II: The Fine Structure of Simulated Galactic Disks. Astrophys. J 597, 21–34 (2003)

    Article  Google Scholar 

  31. Wildberger, A.M.: A.I. and Simluation. Simulation, 1–2 (March 1999)

    Google Scholar 

  32. Simon, H.A.: Complex systems: The interplay of organizations and markets in contemporary society. Computational & Mathematical Organization Theory 7(2), 79–85 (2001)

    Article  Google Scholar 

  33. Scwabacher, M., Gelsey, A.: Multi-Level Simulation and Numerical Optimization of Complex Engineering Designs. In: 6th AIAA/NASA/USAF Multidisciplinary Analysis & Optmization Symposium, Bellevue, WA. AIAA-1996-4021 (1996)

    Google Scholar 

  34. Schaffner, K.F.: Reduction: the Cheshire cat problem and a return to roots. Synthese 151(3), 377–402 (2006)

    Article  Google Scholar 

  35. Gaud, N., Gechter, F., Galland, S., Koukam, A.: Holonic Multiagent Multilevel Simulation Application to Real-time Pedestrians Simulation in Urban Environment. In: Proceedings of IJCAI-2007, pp. 1275–1280 (2007)

    Google Scholar 

  36. Dayan, P.: Levels of Analysis in Neural Modeling. In: Encyclopedia of Cognitive Science, MacMillan Press, London, England (2000)

    Google Scholar 

  37. Arbib, M.A.: Levels of Modeling of Visually Guided Behavior (with peer commentary and author’s response). Behavioral and Brain Sciences 10, 407–465 (1987)

    Article  Google Scholar 

  38. Bakker, B., den Dulk, P.: Causal Relationships and Relationships between Levels: The Modes of Description Perspective. In: Hahn, M., Stoness, S.C. (eds.) Proceedings of the Twenty-First Annual Conference of the Cognitive Science Society, pp. 43–48 (1999)

    Google Scholar 

  39. Saemundsson, R., Thórisson, K.R., Jonsdottir, G.R., Arinbjarnar, M., Finnsson, H., Gudnason, H., Hafsteinsson, V., Hannesson, G., Ísleifsdóttir, J., Jóhannsson, Th., Kristjánsson, G., Sigmundarson, S.: Modular Simulation of Knowledge Development in Industry: A Multi-Level Framework. In: WEHIA – 1st International Conference on Economic Sciences with Heterogeneous Interacting Agents, University of Bologna, Italy, 15-17 June (2006)

    Google Scholar 

  40. Thórisson, K.R., Benko, H., Arnold, A., Abramov, D., Maskey, S., Vaseekaran, A.: Constructionist Design Methodology for Interactive Intelligences. A.I. Magazine 25(4), 77–90 (2004); Menlo Park, CA: American Association for Artificial Intelligence.

    Google Scholar 

  41. Fink, G.A., Jungclaus, N., Kummer, F., Ritter, H., Sagerer, G.: A Distributed System for Integrated Speech and Image Understanding. In: International Symposium on Artificial Intelligence, Cancun, Mexico, pp. 117–126 (1996)

    Google Scholar 

  42. Fink, G.A., Jungclaus, N., Ritter, H., Saegerer, G.: A Communication Framework for Heterogeneous Distributed Pattern Analysis. In: International Conference on Algorithms and Architectures for Parallel Processing, Brisbane, Australia, pp. 881–890 (1995)

    Google Scholar 

  43. Martinho, C., Paiva, A., Gomes, M.R.: Emotions for a Motion: Rapid Development of Believable Pathematic Agents in Intelligent Virtual Environments. Applied Artificial Intelligence 14(1), 33–68 (2000)

    Article  Google Scholar 

  44. Bischoff, R.: Towards the Development of ‘Plug-and-Play’ Personal Robots. In: 1st IEEE-RAS International Conference on Humanoid Robots, September 7–8, 2000, vol. 8, MIT, Cambridge (2000)

    Google Scholar 

  45. Simmons, R., Goldberg, D., Goode, A., Montemerlo, M., Roy, N., Sellner, B., Urmson, C., Schultz, A., Abramson, M., Adams, W., Atrash, A., Bugajska, M., Coblenz, M., MacMahon, M., Perzanowski, D., Horswill, I., Zubek, R., Kortenkamp, D., Wolfe, B., Milam, T., Maxwell, B.: GRACE: An Autonomous Robot for the AAAI Robot Challenge. A.I. Magazine 24(2), 51–72 (2003)

    Google Scholar 

  46. Simon, H.A.: Near decomposability and the speed of evolution. Industrial and Corporate Change 11(3), 587–599 (2002)

    Article  Google Scholar 

  47. Simon, H.A., Ando, A.: Aggregation of Variables in Dynamic Systems. Econometrica 29, 111–138 (1961)

    Article  MATH  Google Scholar 

  48. Fodor, J.: The Modularity of Mind. Bradford Books / MIT Press, Cambridge (1983)

    Google Scholar 

  49. Thórisson, K.R.: Integrated A.I. Systems. Minds & Machines 17, 11–25 (2007); Invited paper at The Dartmouth Artificial Intelligence Conference: The Next 50 Years — Commemorating the 1956 Founding of AI as a Research Discipline, July 13-15, 2006, Dartmouth, New Hampshire, U.S.A. (2006)

    Google Scholar 

  50. Marr, D.: Vision. W.H. Freeman, New York (1982)

    Google Scholar 

  51. Minsky, M.: The Society of Mind. Simon & Schuster, New York (1986)

    Google Scholar 

  52. Scheutz, M.: When physical systems realize functions... Minds and Machines 9, 161–196 (1999)

    Article  Google Scholar 

  53. Ng-Thow-Hing, V., List, T., Thórisson, K.R., Lim, J., Wormer, J.: Design and Evaluation of Communication Middleware in a Distributed Humanoid Robot Architecture. In: Accepted to IROS (2008)

    Google Scholar 

  54. Thórisson, K.R., List, T., Pennock, C., DiPirro, J., Magnusson, F.: Scheduling Blackboards for Interactive Robots. Reykjavik University Department of Computer Science Technical Report, RUTR-CS05002 (2005)

    Google Scholar 

  55. Bonaiuto, J., Thórisson, K.R.: Towards a Neurocogntive Model of Multimodal Turntaking. In: Wachsmuth, I., Knoblich, G., Lenzen, M. (eds.) Embodied Communication in Humans and Machines, forthcoming, Oxford University Press, London (2007)

    Google Scholar 

  56. Thórisson, K.R.: Natural Turn-Taking Needs No Manual: A Computational Model, From Perception to Action. In: Granström, B., House, D., Karlsson, I. (eds.) Multimodality in Language and Speech Systems, pp. 173–207. Kluwer Academic Publishers, Dordrecht, the Netherlands (2002)

    Google Scholar 

  57. Bonaiuto, J., Arbib, M.A.: What Did I Just Do? A New Role for Mirror Neurons (in preparation)

    Google Scholar 

  58. Fagg, A., Arbib, M.A.: Modeling parietal-premotor interactions in primate control of grasping. Neural Netw. 7–8, 1277–1303 (1998)

    Article  Google Scholar 

  59. Alstermark, B., Lundberg, A., Norrsell, U., Sybirska, E.: Integration in descending motor pathways controlling the forelimb in the cat: 9. Differential behavioural defects after spinal cord lesions interrupting defined pathways from higher centres to motorneurones. Experimental Brain Research 42(3), 299–318 (1981)

    Article  Google Scholar 

  60. Garel, S., Rubenstein, J.L.R.: Patterning of the Cerebral Cortex. In: Gazzaniga, M.S. (ed.) The Cognitive Neurosciences III, pp. 69–84. M.I.T. Press, Massachusetts (2004)

    Google Scholar 

  61. Swanson, L.W.: Interactive Brain Maps and Atlases. In: Arbib, M.A., Grethe, J.S. (eds.) Computing the Brain, pp. 167–177. Academic Press, San Diego (2001)

    Chapter  Google Scholar 

  62. Bryson, J.: Modular Representations of Cognitive Phenomena in AI, Psychology and Neuroscience. In: Davis, D. (ed.) Visions of Mind, pp. 66–89. Idea Group, London (2005)

    Google Scholar 

  63. Bryson, J., Stein, L.A.: Modularity and Specialized Learning: Mapping Between Agent Architectures and Brain Organization. In: Wermter, S., Austin, J., Willshaw, D. (eds.) Emergent Neural Computational Architectures based on Neuroscience, Springer, Heidelberg, Germany (2001)

    Google Scholar 

  64. Koechlin, E., Ody, C., Kouneiher, F.: The Architecture of Cognitive Control in Human Prefrontal Cortex. Science 302, 1181–1185 (2003)

    Article  Google Scholar 

  65. Miller, E.K., Cohen, J.D.: An Integrative Theory of Prefrontal Cortex Function. Annu. Rev. Neurosci. 24, 167–202 (2001)

    Article  Google Scholar 

  66. van ’t Wouta, M., Kahn, R.S., Sanfeyd, A.G., Alemanc, A.: Repetitive transcranial magnetic stimulation over the right dorsolateral prefrontal cortex affects strategic decision-making. Cognitive Neuroscience and Neuropsychology 16(16), 1849–1952 (2005)

    Google Scholar 

  67. Oshio, K., Morita, S., Osana, Y., Oka, K.: C. elegans synaptic connectivity data. Technical Report of CCeP, Keio Future, No.1, Keio University (1998)

    Google Scholar 

  68. Zheng, Y., Brockie, P.J., Mellem, J.E., Madsen, D.M., Maricq, A.V.: Neuronal Control of Locomotion in C. elegans Is Modified by a Dominant Mutation in the GLR-1 Ionotropic Glutamate Receptor. Neuron 24, 347–361 (1999)

    Article  Google Scholar 

  69. Bryson, J.: Evidence of Modularity From Primate Errors During Task Learning. In: Cangelosi, A., Bugmann, G., Borisyuk, R. (eds.) Proceedings of The Ninth Neural Computation and Psychology Workshop (NCPW 9), World Scientific, Singapore (2005)

    Google Scholar 

  70. Carruthers, P.: The case for massively modular models of mind. In: Stainton, R. (ed.) Contemporary Debates in Cognitive Science, pp. 205–225. Blackwell, Oxford, England (2005)

    Google Scholar 

  71. List, T., Bins, J., Fisher, R.B., Tweed, D., Thórisson, K.R.: Two Approaches to a Plug-and-Play Vision Architecture - CAVIAR and Psyclone. In: Thórisson, K.R., Vilhjalmsson, H., Marsella, S. (eds.) AAAI-2005 Workshop on Modular Construction of Human-Like Intelligence, Pittsburgh, Pennsylvania, Menlo Park, CA, American Association for Artificial Intelligence, pp. 16–23 (July 10, 2005)

    Google Scholar 

  72. Barrett, H.C., Kurzban, R.: Modularity in Cognition: Framing the Debate. Psych. Rev. 113(3), 628–647 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Analysis and Design of Intelligent Agents and Department of Computer Science, Reykjavik University, Kringlunni 1, 103, Reykjavik, Iceland

    Kristinn R. Thórisson

Authors
  1. Kristinn R. Thórisson
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Ipke Wachsmuth Günther Knoblich

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Thórisson, K.R. (2008). Modeling Multimodal Communication as a Complex System. In: Wachsmuth, I., Knoblich, G. (eds) Modeling Communication with Robots and Virtual Humans. Lecture Notes in Computer Science(), vol 4930. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-79037-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-79037-2_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-79036-5

  • Online ISBN: 978-3-540-79037-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation