Abstract
To make progess in understanding human brain functionality, we will need to understand its basic functions at an abstract level. One way of accomplishing such an integration is to create a model of a human that has a useful amount of complexity. Essentially, one is faced with proposing an embodied “operating system” model that can be tested against human performance. Recently technological advances have been made that allow progress to be made in this direction. Graphics models that simulate extensive human capabilities can be used as platforms from which to develop synthetic models of visuo-motor behavior. Currently such models can capture only a small portion of a full behavioral repertoire, but for the behaviors that they do model, they can describe complete visuo-motor subsystems at a level of detail that can be tested against human performance in realistic environments. This paper outlines one such model and shows both that it can produce interesting new hypotheses as to the role of vision and also that it can enhance our understanding of visual attention.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Clark, A.: Being There: Putting Brain, Body, and World Together Again. MIT Press, Cambridge (1997)
Merleau-Ponty, M.: Phenomenology of Perception. Routledge & Kegan Paul, London (1962)
Terzopoulos, D., Rabie, T.F.: Animat vision: Active vision in artificial animals. Videre: Journal of Computer Vision Research 1, 2–19 (1997)
Pelz, J., Hayhoe, M., Loeber, R.: The coordination of eye, head, and hand movements in a natural task. Experimental Brain Research 139, 166–177 (2001)
Newell, A.: Unified Theories of Cognition. Harvard University Press, Cambridge (1990)
Nilsson, N.: Shakey the robot. Technical Report 223, SRI International (1984)
Marr, D.: Vision. W.H. Freeman and Co., Oxford (1982)
Brooks, R.A.: A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation RA-2, 14–23 (1986)
Hartley, R., Pipitone, F.: Experiments with the subsumption architecture. In: Proceedings of the International Converence on Robotics and Automation (1991)
Firby, R.J., Kahn, R.E., Prokopowicz, P.N., Swain, M.J.: An architecture for vision and action, pp. 72–79 (1995)
Bryson, J.J., Stein, L.A.: Modularity and design in reactive intelligence. In: International Joint Conference on Artificial Intelligence, Seattle, Washington (2001)
Land, M.F., Mennie, N., Rusted, J.: Eye movements and the roles of vision in activities of daily living: making a cup of tea. J. Neuroscience 21, 6917–6932 (2001)
Hayhoe, M.M., Bensinger, D., Ballard, D.H.: Task constraints in visual working memory. Vision Research 38, 125–137 (1998)
Johansson, R., Westling, G., Backstrom, A., Flanagan, J.R.: Eye-hand coordination in object manipulation. Perception 28, 1311–1328 (1999) 21
Sprague, N., Ballard, D.: Eye movements for reward maximization. In: Advances in Neural Information Processing Systems 15 (2003)
Ullman, S.: Visual routines. Cognition 18, 97–159 (1985)
Roelfsema, P., Lamme, V., Spekreijse, H.: The implementation of visual routines. Vision Research 40, 1385–1411 (2000)
Ballard, D., Hayhoe, M., Pook, P.: Deictic codes for the embodiment of cognition. Behavioral and Brain Sciences 20, 723–767 (1997)
Ballard, D., Sprague, N.: Attentional resource allocation in extended natural tasks [abstract]. Journal of Vision 2, 568a (2002)
Maloney, L., Landy, M.: When uncertainty matters: the selection of rapid goaldirected movements [abstract]. Journal of Vision (to appear)
Suri, R.E., Schultz, W.: Temporal difference model reproduces anticipatory neural activity. Neural Computation 13, 841–862 (2001)
Watkins, C.J.C.H., Dayan, P.: Technical note: Q-learning. Machine Computation 8, 279–292 (1992)
Humphrys, M.: Action selection methods using reinforcement learning. In: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior (1996)
Karlsson, J.: Learning to Solve Multiple Goals. PhD thesis, University of Rochester (1997)
Roelfsema, P.R., Spekreijse, H.: Subtask sequencing in the primary visual cortex. Proceedings of the National Academy of Sciences USA 100, 5467–5472 (2003)
Itti, L., Koch, C.: A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research 40, 1489–1506 (2000)
Miller, G.: The magic number seven plus or minus two: Some limits on your capacity for processing information. Psychological Review 63, 81–96 (1956)
Luck, S.J., Vogel, E.K.: The capacity of visual working memory for features and conjunctions. Nature 390, 279–281 (1997)
von der Malsburg, C.: The what and why of binding: the modeler’s perspective. Neuron. 24, 95–104 (1999)
Pashler, H.: The Psychology of Attention. MIT Press, Cambridge (1998)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Watkins, C.J.C.H.: Learning from Delayed Rewards. PhD thesis, King’s College, Oxford (1989)
Sutton, R.: Generalization in reinforcement learning: Successful examples using sparse coarse coding. In: Advances in Neural Information Processing Systems, vol. 8 (1996)
Sprague, N., Ballard, D.: Multiple goal learning for a virtual human (in preparation)
Sprague, N., Ballard, D.: Multiple-goal reinforcement learning with modular sarsa(0). In: International Joint Conference on Artificial Intelligence (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ballard, D., Sprague, N. (2005). Modeling the Brain’s Operating System. In: De Gregorio, M., Di Maio, V., Frucci, M., Musio, C. (eds) Brain, Vision, and Artificial Intelligence. BVAI 2005. Lecture Notes in Computer Science, vol 3704. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11565123_34
Download citation
DOI: https://doi.org/10.1007/11565123_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29282-1
Online ISBN: 978-3-540-32029-6
eBook Packages: Computer ScienceComputer Science (R0)