Abstract
We have implemented a neurobiologically plausible system on a robot that integrates visual attention, object recognition, language and action processing using a coherent cortex-like architecture based on neural associative memories. This system enables the robot to respond to spoken commands like ”bot show plum” or ”bot put apple to yellow cup”. The scenario for this is a robot close to one or two tables carrying certain kinds of fruit and other simple objects. Tasks such as finding and pointing to certain fruits in a complex visual scene according to spoken or typed commands can be demonstrated. This involves parsing and understanding of simple sentences, relating the nouns to concrete objects sensed by the camera, and coordinating motor output with planning and sensory processing.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Knoblauch, A., Fay, R., Kaufmann, U., Markert, H., Palm, G.: Associating words to visually recognized objects. In: Coradeschi, S., Saffiotti, A. (eds.) Anchoring symbols to sensor data. Papers from the AAAI Workshop. Technical Report WS-04-03, pp. 10–16. AAAI Press, Menlo Park (2004)
Hebb, D.: The organization of behavior. A neuropsychological theory. Wiley, New York (1949)
Palm, G.: Cell assemblies as a guideline for brain research. Concepts in Neuroscience 1, 133–148 (1990)
Willshaw, D., Buneman, O., Longuet-Higgins, H.: Non-holographic associative memory. Nature 222, 960–962 (1969)
Palm, G.: On associative memories. Biological Cybernetics 36, 19–31 (1980)
Palm, G.: Neural Assemblies. An Alternative Approach to Artificial Intelligence. Springer, Berlin (1982)
Palm, G.: Memory capacities of local rules for synaptic modification. A comparative review. Concepts in Neuroscience 2, 97–128 (1991)
Schwenker, F., Sommer, F., Palm, G.: Iterative retrieval of sparsely coded associative memory patterns. Neural Networks 9, 445–455 (1996)
Sommer, F., Palm, G.: Improved bidirectional retrieval of sparse patterns stored by hebbian learning. Neural Networks 12, 281–297 (1999)
Hopfield, J.: Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Science, USA 79, 2554–2558 (1982)
Knoblauch, A., Palm, G.: Pattern separation and synchronization in spiking associative memories and visual areas. Neural Networks 14, 763–780 (2001)
Knoblauch, A.: Synchronization and pattern separation in spiking associative memory and visual cortical areas. PhD thesis, Department of Neural Information Processing, University of Ulm, Germany (2003)
Utz, H., Sablatnög, S., Enderle, S., Kraetzschmar, G.K.: Miro – middleware for mobile robot applications. IEEE Transaction on Robotics and Automation, Special Issue on Object-Oriented Distributed Control Architectures 18, 493–497 (2002)
Fay, R., Kaufmann, U., Schwenker, F., Palm, G.: Learning object recognition in an neurobotic system. In: 3rd IWK Workshop SOAVE 2004 - SelfOrganization of AdaptiVE behavior, Illmenau, Germany (2004) (in press)
Jähne, B.: Digitale Bildverarbeitung. Springer, Heidelberg (1997)
Foley, J., van Dam, A., Feiner, S., Hughes, J.: Computer Graphics, 2nd edn. Addison-Wesley, Reading (2003)
Freeman, W.T., Roth, M.: Orientation histograms for hand gesture recognition. In: International Workshop on Automatic Face- and Gesture-Recognition, Zürich (1995)
Smith, A.R.: Color gamut transform pairs. Computer Graphics 12, 12–19 (1978)
Gonzales, R.C., Woods, R.E.: Digital Image Processing, 2nd edn. Addison-Wesley, Reading (1992)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (2000)
Schürmann, J.: Pattern Classification (1996)
Browne, A., Sun, R.: Connectionist inference models. Neural Networks 14, 1331–1355 (2001)
Palm, G., Kraetzschmar, G.K.: Sfb 527: Integration symbolischer und subsymbolischer informationsverarbeitng in adaptiven sensomotorischen systemen. In: Jarke, M., Pasedach, K., Pohl, K. (eds.) Informatik 1997 - Informatik als Innovationsmotor, 27. Jahrestagung der Gesellschaft für Informatik. Springer, Aachen (1997)
McGarry, K., Wermter, S., MacIntyre, J.: Hybrid neural systems: From simple coupling to fully integrated neural networks. Neural Computing Surveys 2, 62–93 (1999)
Knoblauch, A., Fay, R., Kaufmann, U., Markert, H., Palm, G.: Associating words to visually recognized objects. In: AAAI Workshop on Anchoring Symbols to Sensor Data, San Jose, California (2004)
Tou, J.T., Gonzales, R.C.: Pattern Recognition Principles. Addison-Wesley, Reading (1979)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn., New York (2001)
Broomhead, D., Lowe, D.: Multivariable functional interpolation and adaptive networks. Complex Systems 2, 321–355 (1988)
Schwenker, F., Kestler, H.A., Palm, G.: Three learning phases for radial-basis-function netwoks. Neural Networks 14, 439–458 (2001)
Knoblauch, A., Palm, G.: Cortical assemblies of language areas: Development of cell assembly model for Broca/Wernicke areas. Technical Report 5 (WP 5.1), MirrorBot project of the European Union IST-2001-35282, Department of Neural Information Processing, University of Ulm (2003)
Pulvermüller, F.: The neuroscience of language: on brain circuits of words and serial order. Cambridge University Press, Cambridge (2003)
Fay, R., Kaufmann, U., Knoblauch, A., Markert, H., Palm, G.: Video of mirrorbot demonstration (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Fay, R., Kaufmann, U., Knoblauch, A., Markert, H., Palm, G. (2005). Combining Visual Attention, Object Recognition and Associative Information Processing in a NeuroBotic System. In: Wermter, S., Palm, G., Elshaw, M. (eds) Biomimetic Neural Learning for Intelligent Robots. Lecture Notes in Computer Science(), vol 3575. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11521082_8
Download citation
DOI: https://doi.org/10.1007/11521082_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27440-7
Online ISBN: 978-3-540-31896-5
eBook Packages: Computer ScienceComputer Science (R0)