Skip to main content
SpringerLink
Log in

Analysis of Articulated Motion for Social Signal Processing

  • Chapter
  • First Online:
Companion Technology

Part of the book series: Cognitive Technologies ((COGTECH))

  • 787 Accesses

Abstract

Companion technologies aim at developing sustained long-term relationships by employing non-verbal communication (NVC) skills. Visual NVC signals can be conveyed over a variety of non-verbal channels, such as facial expressions, gestures, or spatio-temporal behavior. It remains a challenge to equip technical systems with human-like abilities to reliably and effortlessly detect and analyze such social signals. In this proposal, we focus our investigation on the modeling of visual mechanisms for the processing and analysis of human-articulated motion and posture information from spatially intermediate to remote distances. From a modeling perspective, we investigate how visual features and their integration over several stages in a processing hierarchy take part in the establishment of articulated motion representations. We build upon known structures and mechanisms in cortical networks of primates and emphasize how generic processing principles might realize the building blocks for such network-based distributed processing through learning. We demonstrate how feature representations in segregated pathways and their convergence lead to integrated form and motion representations using artificially generated articulated motion sequences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We use the terms articulated motion and biological motion in a somewhat loose sense. In order to be more precise, articulated motion refers to the movement of parts, or limbs, which are connected by joints. These are themselves composed of elementary movements and concerted into a sequence of actions. The term biological motion is used in the social and cognitive neuroscience community to refer to moving animate objects, which can be attributed as being locomotive. Efforts have been devoted to impoverishing the stimuli depicting such animate movements in order to reveal the key features underlying the perception of such locomotions, e.g., the point-light motion sequences proposed by Johansson [27].

  2. 2.

    The superior temporal sulcus (STS) is anatomically not an area, but a region that contains several areas and subcomponents thereof. We use the term “complex” for the model in order to highlight its specific functionality within the model as a convergent zone of information fusion.

References

  1. Argyle, M.: Bodily Communication. Methuen & Co Ltd, London (1988)

    Google Scholar 

  2. Baker, C., Keysers, C., Jellema, T., Wicker, B., Perrett, D.: Neuronal representation of disappearing and hidden objects in temporal cortex of the macaque. Exp. Brain Res. 140(3), 375–381 (2001)

    Article  Google Scholar 

  3. Barraclough, N.E., Xiao, D., Oram, M.W., Perrett, D.: The sensitivity of primate STS neurons to walking sequences and to the degree of articulation in static images. Prog. Brain Res. 154, 135–148 (2006)

    Article  Google Scholar 

  4. Bayerl, P., Neumann, H.: Disambiguating visual motion through contextual feedback modulation. Neural Comput. 16(10), 2041–2066 (2004)

    Article  MATH  Google Scholar 

  5. Beauchamp, M.S., Lee, K.E., Haxby, J.V., Martin, A.: FMRI responses to video and point-light displays of moving humans and manipulable objects. J. Cogn. Neurosci. 15(7), 991–1001 (2003)

    Article  Google Scholar 

  6. Benyon, D., Mival, O.: Landscaping personification technologies: from interactions to relationships. In: Proceedings of the CHI ’08, Extended Abstracts on Human Factors in Computing Systems, CHI EA ’08, pp. 3657–3662. ACM, New York (2008)

    Google Scholar 

  7. Benyon, D., Mival, O.: Scenarios for companions. In: Your Virtual Butler. Lecture Notes in Computer Science, vol. 7407, pp. 79–96. Springer, Berlin (2013)

    Google Scholar 

  8. Bickmore, T.W., Picard, R.W.: Establishing and maintaining long-term human-computer relationships. ACM Trans. Comput.-Hum. Interaction 12, 293–327 (2005)

    Article  Google Scholar 

  9. Blakemore, S.J., Decety, J.: From the perception of action to the understanding of intention. Nat. Rev. Neurosci. 2(8), 561–567 (2001)

    Google Scholar 

  10. Bobick, A.F., Davis, J.W.: The recognition of human movement using temporal templates. IEEE Trans. Pattern Anal. Mach. Intell. 23(3), 257–267 (2001)

    Article  Google Scholar 

  11. Bouecke, J.D., Tlapale, E., Kornprobst, P., Neumann, H.: Neural mechanisms of motion detection, integration, and segregation: from biology to artificial image processing systems. EURASIP J. Adv. Signal Process. 2011(1), 781561 (2010)

    Article  Google Scholar 

  12. Carandini, M., Heeger, D.J., Movshon, J.A.: Linearity and gain control in V1 simple cells. Cereb. Cortex (13), 401–444 (1999)

    Article  Google Scholar 

  13. Carpenter, G.A.: Neural network models for pattern recognition and associative memory. Neural Netw. 2(4), 243–257 (1989)

    Article  Google Scholar 

  14. Casile, A., Giese, M.A.: Critical features for the recognition of biological motion. J. Vis. 5(4), 6 (2005)

    Article  Google Scholar 

  15. Castellano, G., McOwan, P.W.: Towards affect sensitive and socially perceptive companions. In: Your Virtual Butler. Lecture Notes in Computer Science, vol. 7407, pp. 42–53. Springer, Berlin (2013)

    Google Scholar 

  16. Dollár, P., Rabaud, V., Cottrell, G., Belongie, S.: Behavior recognition via sparse spatio-temporal features. In: 2nd Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005, pp. 65–72. IEEE, New York (2005)

    Google Scholar 

  17. Escobar, M.J., Kornprobst, P.: Action recognition via bio-inspired features: the richness of center–surround interaction. Comput. Vis. Image Underst. 116(5), 593–605 (2012)

    Article  Google Scholar 

  18. Escobar, M.J., Masson, G.S., Vieville, T., Kornprobst, P.: Action recognition using a bio-inspired feedforward spiking network. Int. J. Comput. Vis. 82(3), 284–301 (2009)

    Article  Google Scholar 

  19. Frith, C.D., Wolpert, D.M.: The Neuroscience of Social Interaction: Decoding, Imitating, and Influencing the Actions of Others. Oxford University Press, Oxford (2004)

    Google Scholar 

  20. Giese, M.A., Poggio, T.: Neural mechanisms for the recognition of biological movements. Nat. Rev. Neurosci. 4(3), 179–192 (2003)

    Article  Google Scholar 

  21. Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2247–2253 (2007)

    Article  Google Scholar 

  22. Grüsser, O.J.: Grundlagen der neuronalen Informationsverarbeitung in den Sinnesorganen und im Gehirn. In: GI - 8. Jahrestagung, pp. 234–273. Springer, Berlin (1978)

    Google Scholar 

  23. Hansen, T., Neumann, H.: A recurrent model of contour integration in primary visual cortex. J. Vis. 8(8), 1–25 (2008)

    Article  Google Scholar 

  24. Jellema, T., Perrett, D.I.: Cells in monkey STS responsive to articulated body motions and consequent static posture: a case of implied motion? Neuropsychologia 41(13), 1728–1737 (2003)

    Article  Google Scholar 

  25. Jellema, T., Maassen, G., Perrett, D.I.: Single cell integration of animate form, motion and location in the superior temporal cortex of the macaque monkey. Cereb. Cortex 14(7), 781–790 (2004)

    Article  Google Scholar 

  26. Jhuang, H., Serre, T., Wolf, L., Poggio, T.: A biologically inspired system for action recognition. In: Proceedings of the 11th IEEE International Conference on Computer Vision, pp. 1–8 (2007)

    Google Scholar 

  27. Johansson, G.: Visual perception of biological motion and a model for its analysis. Percept. Psychophys. 14(2), 201–211 (1973)

    Article  Google Scholar 

  28. Kourtzi, Z., Kanwisher, N.: Activation in human MT/MST by static images with implied motion. J. Cogn. Neurosci. 12(1), 48–55 (2000)

    Article  Google Scholar 

  29. Lange, J., Lappe, M.: A model of biological motion perception from configural form cues. J. Neurosci. 26(11), 2894–2906 (2006)

    Article  Google Scholar 

  30. Lappe, M.: Perception of biological motion as motion-from-form. e-Neuroforum 3(3), 67–73 (2012)

    Google Scholar 

  31. Laptev, I.: On space-time interest points. Int. J. Comput. Vis. 64(2-3), 107–123 (2005)

    Article  Google Scholar 

  32. Laptev, I., Caputo, B., Schüldt, C., Lindeberg, T.: Local velocity-adapted motion events for spatio-temporal recognition. Comput. Vis. Image Underst. 108(3), 207–229 (2007)

    Article  Google Scholar 

  33. Layher, G., Giese, M.A., Neumann, H.: Learning representations of animated motion sequences - a neural model. Top. Cogn. Sci. 6(1), 170–182 (2014)

    Article  Google Scholar 

  34. Oja, E.: Simplified neuron model as a principal component analyzer. J. Math. Biol. 15(3), 267–273 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  35. Pentland, A.: Social Signal Processing. IEEE Signal Process. Mag. 24(4), 108–111 (2007)

    Article  Google Scholar 

  36. Raudies, F., Mingolla, E., Neumann, H.: A model of motion transparency processing with local center-surround interactions and feedback. Neural Comput. 1–45 (2011)

    Google Scholar 

  37. Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025 (1999)

    Article  Google Scholar 

  38. Rittscher, J., Blake, A., Hoogs, A., Stein, G.: Mathematical modelling of animate and intentional motion. Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci. 358(1431), 475–490 (2003)

    Article  Google Scholar 

  39. Schindler, K., Van Gool, L.: Action snippets: how many frames does human action recognition require? In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE Computer Society, New York (2008)

    Google Scholar 

  40. Senior, C., Barnes, J., Giampietroc, V., Simmons, A., Bullmore, E.T., Brammer, M., David, A.S.: The functional neuroanatomy of implicit-motion perception or ‘representational momentum’. Curr. Biol. 10(1), 16–22 (2000)

    Article  Google Scholar 

  41. Thirkettle, M., Benton, C.P., Scott-Samuel, N.E.: Contributions of form, motion and task to biological motion perception. J. Vis. 9(3), 28 (2009)

    Article  Google Scholar 

  42. Thompson, J.C., Clarke, M., Stewart, T., Puce, A.: Configural processing of biological motion in human superior temporal sulcus. J. Neurosci. 25(39), 9059–9066 (2005)

    Article  Google Scholar 

  43. Turaga, P., Chellappa, R., Subrahmanian, V.S., Udrea, O.: Machine recognition of human activities: a survey. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1473–1488 (2008)

    Article  Google Scholar 

  44. Ungerleider, L.G., Pasternak, T.: Ventral and dorsal cortical processing streams. Vis. Neurosci. 1(34), 541–562 (2004)

    Google Scholar 

  45. Wallis, G., Rolls, E.: Invariant face and object recognition in the visual system. Prog. Neurobiol. 51(2), 167–194 (1997)

    Article  Google Scholar 

  46. Weidenbacher, U., Neumann, H.: Extraction of surface-related features in a recurrent model of V1-V2 interactions. PloS ONE 4(6), e5909 (2009)

    Google Scholar 

Download references

Acknowledgements

This work was done within the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG).

Author information

Authors and Affiliations

  1. Institute of Neural Information Processing, Ulm University, James-Franck-Ring, 89069 Ulm, Germany

    Georg Layher, Michael Glodek & Heiko Neumann

Authors
  1. Georg Layher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Glodek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heiko Neumann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Georg Layher .

Editor information

Editors and Affiliations

  1. Institute of Artificial Intelligence, Universität Ulm, Ulm, Germany

    Susanne Biundo

  2. Cognitive Systems Group, Institute for Information Technology and Communications (IIKT) and Center for Behavioral Brain Sciences (CBBS), Otto-von-Guericke Universität Magdeburg, Magdeburg, Germany

    Andreas Wendemuth

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Layher, G., Glodek, M., Neumann, H. (2017). Analysis of Articulated Motion for Social Signal Processing. In: Biundo, S., Wendemuth, A. (eds) Companion Technology. Cognitive Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-43665-4_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-43665-4_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-43664-7

  • Online ISBN: 978-3-319-43665-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation