Abstract
Perceptual functions are central to many applications in robotics and for the construction of efficient human–robot interfaces. The study of perception in biological systems has revealed important information-processing principles that have been converted to powerful applications in robotics and computer vision. The chapter first discusses two central theories of object recognition: model- and exemplar-based theories. A review of experimental results from the study of object recognition in biological systems suggests that exemplar-based approaches capture important properties of object recognition in the brain. We then discuss how very similar principles have been realized in highly efficient technical systems for object recognition and detection, including realizations that are based on biologically inspired neural architectures. Principles for the efficient processing of complex shapes can be extended to the representation of complex movements and actions. We illustrate this by first reviewing some properties of the cortical mechanisms of the recognition of complex movements and actions, focusing on principles that are useful for robotics applications. Again, exemplar-based approaches seem to capture important properties of motion recognition in the brain, and at the same time provide a powerful approach for building technical movement recognition systems. Finally, it is shown that the example-based framework is not only useful for recognition, but also provides the basis for powerful synthesis methods. As one example we discuss the synthesis of photorealistic three-dimensional (3-D) models of faces, exploiting correspondencebetween training examples. Related approaches have been developed for spatiotemporal patterns. We review a class of algorithms that permit the accurate modeling of movements and movement styles by interpolation between example trajectories with high relevance for the synthesis of movements, e.g., in humanoid robotics.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Abbreviations
- AIP:
-
anterior interparietal area
- AIT:
-
anterior inferotemporal cortex
- EBA:
-
extrastriate body part area
- GSD:
-
geon structural description
- HMM:
-
hidden Markov model
- IT:
-
inferotemporal
- IT:
-
intrinsic tactile
- LGN:
-
lateral geniculate nucleus
- MT:
-
medial temporal
- MT:
-
multitask
- NAP:
-
nonaccidental properties
- PCA:
-
principle components analysis
- PFC:
-
prefrontal cortex
- PIT:
-
posterior inferotemporal cortex
- RBC:
-
recognition-by-components
- RBF:
-
radial basis function
- RT:
-
reaction time
- RT:
-
room-temperature
- STS:
-
superior temporal sulcus
- ZMP:
-
zero-moment point
- fMRI:
-
functional magnetic resonance imaging
References
D. Marr: Vision (Freeman, San Francisco 1982)
R.A. Brooks: A robust layered control system for a mobile robot, IEEE J. Robot. Automat. 2(1), 14–23 (1986)
J.A.S. Kelso: Dynamic Patterns: The Self-Organization of Brain and Behaviour (MIT Press, Cambridge 1995)
G. Schöner, M. Dose, C. Engels: Dynamics of behavior: theory and applications for autonomous robot architectures, Robot. Auton. Syst. 16, 213–245 (1997)
J. Tani, M. Ito: Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment, IEEE Trans. Syst. Man Cybernet. Part A: Syst Humans 33(4), 481–488 (2003)
W.H. Warren: The dynamics of perception and action, Psychol. Rev. 113, 358–389 (2006)
D. Marr, H. Nishihara: Representation and recognition of the spatial organization of three-dimensional shapes, Proc. R. Soc. London B 200, 269–294 (1978)
I. Biederman: Recognition-by-components: A theory of human image understanding, Psycholog. Rev. 94, 115–147 (1987)
D. Lowe: Perceptual Organization and Visual Recognition (Kluwer, Boston 1985)
S. Ullman: High-level vision. Object Recognition and Visual Cognition (MIT Press, Cambridge 1996)
M.A. Kurbat: Structural description theories: is RBC/JIM a general-purpose theory of human entry-level object recognition?, Perception 23, 1339–1368 (1994)
S. Edelman: Representation and Recognition in Vision (MIT Press, Cambridge 1999)
M.J. Tarr, H.H. Bülthoff: Image-based object recognition in man, monkey and machine, Cognition 67(1–2), 1–20 (1998)
M. Graf, W. Schneider: Structural descriptions in HIT - a problematic commitment, Behav. Brain Sci. 24, 483–484 (2001)
H.H. Bülthoff, S. Edelman: Psychophysical support for a two-dimensional view interpolation theory of object recognition, Proc. Nat. Acad. Sci. USA 89, 60–64 (1992)
M.J. Tarr, S. Pinker: Mental orientation and orientation-dependence in shape recognition, Cognit. Psychol. 21, 233–282 (1989)
W.G. Hayward, M.J. Tarr: Testing conditions for viewpoint invariance in object recognition, J. Exp. Psychol. Human Percept. Perform. 23, 1511–1521 (1997)
S.E. Palmer, E. Rosch, P. Chase: Canonical perspective and the perception of objects. In: Attention and Performance IX, ed. by J. Long, A. Baddeley (Erlbaum, Hillsdale 1981) pp. 135–151
H. Hill, P.G. Schyns, S. Akamatsu: Information and viewpoint dependence in face recognition, Cognition 62, 201–222 (1997)
C. Wallraven, A. Schwaninger, S. Schuhmacher, H.H. Bülthoff: View-Based Recognition of Faces in Man and Machine: Re-visiting Inter-Extra-Ortho, Lect. Notes Comput. Sci. 2525, 651–660 (2002)
M.J. Tarr: Rotating objects to recognize them: a case study on the role of viewpoint dependency in the recognition of three-dimensional objects, Psychonom. Bull. Rev. 2, 55–82 (1995)
R. Lawson, G.W. Humphreys: View-specific effects of depth rotation and foreshortening on the initial recognition and priming of familiar objects, Percept. Psychophys. 60, 1052–1066 (1998)
E. Ashbridge, D.I. Perrett: Generalizing across object orientation and size. In: Perceptual Constancy. Why Things look as they do, ed. by V. Walsh, J. Kulikowski (Cambridge University Press, Cambridge 1998) pp. 192–209
M. Dill, S. Edelman: Imperfect invariance to object translation in the discrimination of complex shapes, Perception 30, 707–724 (2001)
K.R. Cave, S. Pinker, L. Giorgi, C.E. Thomas, L.M. Heller, J.M. Wolfe, H. Lin: The representation of location in visual images, Cognit. Psychol. 26, 1–32 (1994)
S. Ullman: Aligning pictorial descriptions: an approach to object recognition, Cognition 32, 193–254 (1989)
S. Ullman, R. Basri: Recognition by linear combinations of models, IEEE Trans. Pattern Anal. Mach. Intell. 13, 992–1006 (1991)
T. Poggio, S. Edelman: A network that learns to recognize three-dimensional objects, Nature 343, 263–266 (1990)
D. Perrett, W.M. Oram: Visual recognition based on temporal cortex cells: viewer-centred processing of pattern configurations, Zeitschrift Naturforschung C 53, 518–541 (1998)
M. Riesenhuber, T. Poggio: Hierarchical models of object recognition in cortex, Nature Neurosci. 2, 1019–1025 (1999)
E.T. Rolls, T. Milward: A model of invariant object recognition in the visual system: learning rules, activation functions, lateral inhibition, and information-based performance measures, Neural Comput. 2(11), 2547–2572 (2000)
G. Wallis, H.H. Bülthoff: Learning to recognize objects, Trends Cognit. Sci. 3, 22–31 (1999)
D. Perrett, W.M. Oram, E. Ashbridge: Evidence accumulation in cell populations responsive to faces: an account of generalization of recognition without mental transformations, Cognition 67, 111–145 (1998)
H. Sakata: The role of the parietal cortex in grasping, Adv. Neurol. 93, 121–139 (2003)
D.H. Hubel, T.N. Wiesel: Receptive fields, binocular interaction and functional architecture in the catʼs visual cortex, J. Physiol. (London) 160, 106–154 (1962)
G. Wang, M. Tanifuji, K. Tanaka: Functional architecture in monkey inferotemporal cortex revealed by in vivo optical imaging, Neurosci. Res. 32, 33–46 (1998)
K. Tanaka: Representation of visual feature objects in the inferotemporal cortex, Neural Netw. 9(8), 1459–1475 (1996)
K. Grill-Spector, R. Malach: The human visual cortex, Annu. Rev. Neurosci. 27, 649–677 (2004)
N.K. Logothetis, J. Pauls, H.H. Bülthoff, T. Poggio: View-dependent object recognition by monkeys, Curr. Biol. 4, 401–414 (1994)
L. Roberts: Machine perception of three-dimensional solids. In: Optical and Electro-optical Information Processing, ed. by J.T. Tippet (MIT Press, Cambridge 1965) pp. 159–197
M. Swain, D. Ballard: Color indexing, Int. J. Comput. Vis. 7, 11–32 (1991)
C. Schmid, R. Mohr: Local greyvalue invariants for image retrieval, IEEE Trans. Pattern Mach. Intell. 19, 530–535 (1997)
D. Lowe: Distinctive image features from scale invariant keypoints, Int. J. Comput. Vis. 60(2), 91–110 (2004)
M. Kirby, L. Sirovich: Applications of the Karhunen-Loeve procedure for the characterization of human faces, IEEE Trans. Pattern Mach. Intell. 12, 103–108 (1990)
A. Delorme, S. Thorpe: SpikeNET: An event-driven simulation package for modeling large networks of spiking neurons, Netw. Comput. Neural Syst. 14, 613–627 (2003)
Y. LeCun, F. Huang, L. Bottou: Learning Methods for Generic Object Recognition with Invariance to Pose and Lighting in Proceedings of 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2004)
T. Serre, L. Wolf, T. Poggio: Object recognition with features inspired by visual cortex. In: Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005)
S. Thorpe, D. Fize, C. Marlot: Speed of processing in the human visual system, Nature 381(6582), 520–522 (1996)
G. Wallis, H.H. Bülthoff: Effects of temporal association on recognition memory, Proc. Nat. Acad. Sci. USA 98, 4800–4804 (2001)
J.V. Stone: Object recognition using spatio-temporal signatures, Vis. Res. 38, 947–951 (1998)
J.V. Stone: Object recognition: View-specificity and motion-specificity, Vis. Res. 39, 4032–4044 (1999)
Q.C. Voung, M.J. Tarr: Rotation direction affects object recognition, Vis. Res. 44(14), 1717–1730 (2004)
C. Wallraven, H.H. Bülthoff: Automatic acquisition of exemplar-based representations for recognition from image sequences. CVPR 2001 - Workshop on Models versus Exemplars (2001)
C. Wallraven, B. Caputo, A.B.A. Graf: Recognition with local features: The kernel recipe, Proc. Int. Conf. Comput. Vis. 2, 257–264 (2003), IEEE Press
F.N. Newell, M.O. Ernst, B.S. Tjan, H.H. Bülthoff: Viewpoint dependence in visual and haptic object recognition, Psychol. Sci. 12, 37–42 (2001)
M.A. Giese, T. Poggio: Neural mechanisms for the recognition of biological movements, Nat. Rev. Neurosci. 4, 179–192 (2003)
C. Wallraven, H.H. Bülthoff: Object Recognition in Man and Machine. Object Recognition, Attention and Action (Springer, Tokyo) (in press)
J.-P. Ewert: Neural mechanisms of prey-catching and avoidance behavior in the toad (bufo bufo), Brain Behav. Evol. 3, 36–56 (1970)
G. Johansson: Visual perception of biological motion and a model for its analysis, Percept. Psychophys. 14, 201–211 (1973)
K. Verfaillie: Perceiving human locomotion: priming effects in direction discrimination, Brain Cogn. 44, 192–213 (2000)
A.J. OʼToole, D.A. Roark, H. Abdi: Recognizing moving faces: A psychological and neural synthesis, Trends Cognit. Sci. 6, 261–266 (2002)
D. Perrett, A. Puce: Electrophysiology and brain imaging of biological motion, Phil. Trans. R. Soc. Lond. B 358, 435–445 (2003)
D.D. Hoffman, B.E. Flinchbaugh: The interpretation of biological motion, Biol. Cybernet. 42, 195–204 (1982)
J.A. Webb, J.K. Aggarwal: Structure from motion of rigid and jointed objects, Artif. Intell. 19, 107–130 (1982)
M. Peelen, A. Wiggett, P. Downing: Patterns of fMRI activity dissociate overlapping functional brain areas that respond to biological motion, Neuron 49, 815–822 (2006)
G. Rizzolatti, L. Craighero: The mirror-neuron system, Ann. Rev. Neurosci. 27, 169–192 (2004)
E.D. Grossman, R. Blake, C.Y. Kim: Learning to see biological motion: brain activity parallels behavior, J. Cogn. Neurosci. 16, 1669–1679 (2004)
J. Jastorff, Z. Kourtzi, M.A. Giese: Learning to discriminate complex movements: Biological versus artificial trajectories, J. Vis. 6, 791–804 (2006)
H. Hill, F.E. Pollick: Exaggerating temporal differences enhances recognition of individuals from point light displays, Psychol. Sci. 11, 223–228 (2000)
B. Knappmeyer, I.M. Thornton, H.H. Bülthoff: The use of facial motion and facial form during the processing of identity, Vision Res. 43, 1921–1936 (2003)
J. Lee, W. Wong: A stochastic model of coherent motion detection, Biol. Cybernet. 91, 306–314 (2004)
P. Kornprobst, T. Vieille, I.K. Dimo: Could early visual processes be sufficient to label motions? Proceedings of the IEEE International Joint Conference of Neural Networks (IJNNʼ05) (2005) pp. 1687–1692
G. Metta, G. Sandini, L. Natale, L. Craighero, L. Fadiga: Understanding mirror neurons: a bio-robotic approach, Interaction Studies 7, in press
D.M. Wolpert, K. Doya, M. Kawato: A unifying computational framework for motor control and social interaction, Philos. Trans. R. Soc. London B Biol. Sci. 358, 593–602 (2003)
A. Bobick: Movement, activity, and action: The role of knowledge in the perception of motion, Philos. Trans. R. Soc. London B 352, 1257–1265 (1997)
I. Essa, A. Pentland: Coding, analysis, interpretation, and recognition of facial expressions, IEEE Trans. Pattern Anal. Mach. Intell. 19, 757–763 (1997)
J.K. Aggarwal, Q. Cai: Human motion analysis: A reviewm, Comput. Vision Image Understand. 73, 428–440 (1999)
D.M. Gavrila: The visual analysis of human movement: A survey, Comp. Vis. Image Underst. 73, 82–98 (1999)
T.B. Moeslund, G. Granum: A survey of computer vision-based human motion capture, Comp. Vis. Image Underst. 81, 231–268 (2001)
D. Marr, L.M. Vaina: Representation and recognition of the movements of shapes, Proc. R. Soc. London B 214, 501–524 (1982)
S. Wachter, H.-H. Nagel: Tracking of persons in monocular image sequences, Comput. Vis. Image Underst. 74, 174–192 (1999)
J.J. Wang, S. Singh: Video analysis of human dynamics - a survey, Real-Time Imaging 9, 321–346 (2003)
R. Rosales, S. Sclaroff: Inferring body pose without tracking body parts, Proc. IEEE Conf. Comput. Vis. Pattern Recognition (CVPR) II, 721–727 (2000)
A. Agarwal, B. Triggs: Recovering 3D human pose from monocular images, IEEE Trans. Pattern Anal. Machine Intell. (PAMI) 28, 44–58 (2006)
C. Curio, M.A. Giese: Combining View-Based and Model-Based Tracking of Articulated Human Movements. In: IEEE Workshop on Applications of Computer Vision/IEEE Workshop on Motion and Video Computing (WACV/MOTION 2005) (IEEE Computer Society, Los Alamitos 2005) pp. 261–268
T. Poggio, E. Bizzi: Generalization in vision and motor control, Nature 431, 768–774 (2004)
P. Niyogi, F. Girosi, T. Poggio: Incorporating prior information in machine learnig by creating virtual examples, Proc. IEEE 86, 2196–2209 (1998)
T. Valentine: A unified account of the effects of distinctiveness, inversion, and race in face recognition, Quart. J. Exp. Psychol. A: Human Exp. Psychol. 43, 161–204 (1991)
T. Vetter, T. Poggio: Linear object classes and image synthesis from a single example image, IEEE Trans. Pattern Anal. Mach. Intell. 19, 733–742 (1997)
A. Lanitis, C.J. Taylor, T.F. Cootes: Automatic Interpretation and Coding of Face Images Using Flexible Models, IEEE Trans. Pattern Mach. Intell. 19, 743–756 (1997)
V. Blanz, T. Vetter: A morphable model for the synthesis of 3d faces. In: Proc. ACM SIGGRAPH (1999) pp. 187–194
I. Bülthoff, F.N. Newell: Categorical perception of sex occurs in familiar but not unfamiliar faces, Vis. Cognit. 11, 823–855 (2004)
A.J. OʼToole, T. Vetter, V. Blanz: Three-dimensional shape and two-dimensional surface reflectance contributions to face recognition: An application of three-dimensional morphing, Vis. Res. 39, 3145–3155 (1999)
D.A. Leopold, A.J. OʼToole, T. Vetter, V. Blanz: Prototype-referenced shape encoding revealed by high-level after effects, Nat. Neurosci. 4, 89–94 (2001)
D.A. Leopold, I.V. Bondar, M.A. Giese: Norm-based face encoding by single neurons in the monkey inferotemporal cortex, Nature 442, 572–575 (2006)
M. Breidt, C. Wallraven, D.W. Cunningham, H.H. Bülthoff: Facial Animation Based on 3D Scans and Motion Capture. SIGGRAPH ʼ03 Sketches & Applications (ACM, New York 2003)
C. Wallraven, M. Breidt, D.W. Cunningham, H.H. Bülthoff: Psychophysical evaluation of animated facial expressions, Proc. 2nd Symp. Appl. Percept. Graphics Visual., 17–24 (2005)
M. Unuma, K. Anjyo, R. Takeuchi: Fourier principles for emotion-based human figure animation, ACM Trans. Comput. Graphics 29, 91–99 (1995)
A. Bruderlin, L. Williams: Motion signal processing, ACM Trans. Comput. Graphics 29, 97–104 (1995)
D.J. Wiley, J.K. Hahn: Interpolation synthesis of articulated figure motion, IEEE Comp. Graphics Appl. 17, 39–45 (1997)
Y. Yacoob, M.J. Black: Parameterized modeling and recognition of activities, Comput. Vis. Image Understand. 73, 232–247 (1999)
M.A. Giese, T. Poggio: Morphable models for the analysis and synthesis of complex motion patterns, Int. J. Comput. Vis. 38, 59–73 (2000)
M. Brand, A. Hertzmann: Style Machines. Proc. ACM SIGGRAPH 2000 (ACM, New York 2000) pp. 183–192
T. Flash, B. Hochner: Motor primitives in vertebrates and invertebrates, Curr. Opin. Neurobiol. 15(6), 660–666 (2005)
O.C. Jenkins, M.J. Mataric: Performance-derived behavior vocabularies: Data-driven acqusition of skills from motion, Int. J. Human Robot. 1, 237–288 (2004)
A. Safonova, J.K. Hodgins, N.S. Pollard: Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces, ACM Trans. Comput. Graphics 23, 514–521 (2004)
W. Ilg, G.H. Bakir, J. Mezger, M.A. Giese: On the representation, learning and transfer of spatio-temporal movement characteristics, Int. J. Human Robot. 1, 613–636 (2004)
O. Arikan, D.A. Forsyth, J.F. OʼBrien: Motion synthesis from annotations, ACM Trans. Graphics 22, 402–408 (2003)
L. Kovar, M. Gleicher, F. Pighin: Motion Graphs. Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (ACM, New York 2002) pp. 473–482
L. Ren, G. Shakhnarovich, J.K. Hodgins, H. Pfister, P. Viola: Learning silhouette features for control of human motion, ACM Trans. Graphics 24, 1303–1331 (2005)
S. Schaal, A. Ijspeert, A. Billard: Computational approaches to motor learning by imitation, Phil. Trans. R. Soc. London B 358, 537–547 (2003)
M. Gleicher, P. Litwinowicz: Constraint-based motion adaptation, J. Vis. Comput. Anim. 9, 65–94 (1998)
S. Tak, H.S. Ko: A physically based motion retargeting filter, ACM Trans. Graphics 24, 98–117 (2005)
C.L. Colby: Action-oriented spatial reference frames in cortex, Neuron 20, 15–24 (1998)
A.R. Kilgour, R. Kitada, P. Servos, T.W. James, S.J. Lederman: Haptic face identification activates ventral occipital and temporal areas: an fMRI study, Brain Cogn. 59, 246–257 (2005)
D.H. Foster, S.J. Gilson: Recognizing novel three-dimensional objects by summing signals from parts and views, Proc. R. Soc. London B 269, 1939–1947 (2002)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag
About this entry
Cite this entry
Bülthoff, H.H., Wallraven, C., Giese, M.A. (2008). Perceptual Robotics. In: Siciliano, B., Khatib, O. (eds) Springer Handbook of Robotics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30301-5_64
Download citation
DOI: https://doi.org/10.1007/978-3-540-30301-5_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23957-4
Online ISBN: 978-3-540-30301-5
eBook Packages: EngineeringEngineering (R0)