Abstract
One of the main claims of the active vision framework is that finding data on the basis of task requirements is more efficient than reconstructing the whole scene by performing a complete visual scan. To be successful, this approach requires that agents learn visual routines to direct overt attention to locations with the information needed to accomplish the task. In ecological conditions, learning such visual routines is difficult due to the partial observability of the world, the changes in the environment, and the fact that learning signals might be indirect. This paper uses a reinforcement-learning actor-critic model to study how visual routines can be formed, and then adapted when the environment changes, in a system endowed with a controllable gaze and reaching capabilities. The tests of the model show that: (a) the autonomously-developed visual routines are strongly dependent on the task and the statistical properties of the environment; (b) when the statistics of the environment change, the performance of the system remains rather stable thanks to the re-use of previously discovered visual routines while the visual exploration policy remains for long time sub-optimal. We conclude that the model has a robust behaviour but the acquisition of an optimal visual exploration policy is particularly hard given its complex dependence on statistical properties of the environment, showing another of the difficulties that adaptive active vision agents must face.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Marr, D.: Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Freeman, New York (1982)
Fermuller, C., Aloimonos, Y.: Vision and action. Image Vision Comput. 13(10), 725–744 (1995)
Ballard, D.: Animate vision. AI 48, 57–86 (1991)
Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cognit. Psychol. 12(1), 97–136 (1980)
Ullman, S.: Visual routines. Cognition 18(1-3), 97–159 (1984)
Hayhoe, M.: Vision using routines: A functional account of vision. Visual Cognition 7(1-2-3), 43–64 (2000)
Heisz, J.J., Shore, D.I.: More efficient scanning for familiar faces. J. Vis. 8(1), 1–10 (2008)
Sailer, U., Flanagan, J.R., Johansson, R.S.: Eye-hand coordination during learning of a novel visuomotor task. J. Neurosci. 25(39), 8833–8842 (2005)
Land, M.F.: Eye movements and the control of actions in everyday life. Prog. Retin. Eye Res. 25(3), 296–324 (2006)
Ognibene, D., Balkenius, C., Baldassarre, G.: Integrating epistemic action (active vision) and pragmatic action (reaching): A neural architecture for camera-arm robots. In: Asada, M., Hallam, J.C.T., Meyer, J.-A., Tani, J. (eds.) SAB 2008. LNCS (LNAI), vol. 5040, pp. 220–229. Springer, Heidelberg (2008)
Ognibene, D., Rega, A., Baldassarre, G.: A model of reaching that integrates reinforcement learning and population encoding of postures. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) From Animals to Animats 9: Proceedings of the Ninth International Conference on the Simulation of Adaptive Behavior (SAB 2006), September 2006, pp. 381–393. Springer, Heidelberg (2006)
Schmidhuber, J., Huber, R.: Learning to generate artificial fovea trajectories for target detection. Int. J. Neural Syst. 2(1-2), 135–141 (1991)
Minut, S., Mahadevan, S.: A reinforcement learning model of selective visual attention. In: AGENTS 2001: Proceedings of the Fifth International Conference on Autonomous Agents, pp. 457–464. ACM, New York (2001)
Kwok, C., Fox, D.: Reinforcement learning for sensing strategies. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2004 (2004)
de Croon, G.: Adaptive Active Vision. PhD thesis, Universiteit Maastricht (2008)
Suzuki, M., Floreano, D.: Enactive robot vision. Adapt. Behav. 16(2-3), 122–128 (2008)
Mirolli, M., Ferrauto, T., Nolfi, S.: Categorisation through evidence accumulation in an active vision system. Connection Science (subm)
Pouget, A., Ducom, J.C., Torri, J., Bavelier, D.: Multisensory spatial representations in eye-centered coordinates for reaching. Cognition 83(1), B1–B11 (2002)
Pouget, A., Zhang, K., Deneve, S., Latham, P.E.: Statistically efficient estimation using population coding. Neural Comput. 10(2), 373–401 (1998)
Cisek, P.: Integrated neural processes for defining potential actions and deciding between them: a computational model. J. Neurosci. 26, 9761–9770 (2006)
Erlhagen, W., Schöner, G.: Dynamic field theory of movement preparation. Psychol. Rev. 109(3), 545–572 (2002)
Itti, L., Koch, C.: Feature combination strategies for saliency-based visual attention systems. Journal of Electronic Imaging 10(1), 161–169 (2001)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Houk, J., Adams, J., Barto, A.: A model of how the basal ganglia generate and use neural signals that predict reinforcement, pp. 249–270 (1995)
Mannella, F., Baldassarre, G.: A neural-network reinforcement-learning model of domestic chicks that learn to localise the centre of closed arenas. Phil. Trans. R. Soc. B 362(383-401), 333–356 (2007)
Dominey, P.F., Arbib, M.A.: A cortico-subcortical model for generation of spatially accurate sequential saccades. Cereb. Cortex. 2(2), 153–175 (1992)
Klein: Inhibition of return. Trends Cogn. Sci. 4(4), 138–147 (2000)
Thompson, K.G., Bichot, N.P.: A visual salience map in the primate frontal eye field. Prog. Brain Res. 147, 251–262 (2005)
Treue, S.: Visual attention: the where, what, how and why of saliency. Curr. Opin. Neurobiol. 13(4), 428–432 (2003)
Chun: Contextual cueing of visual attention. Trends Cogn. Sci. 4(5), 170–178 (2000)
Silver, M.A., Kastner, S.: Topographic maps in human frontal and parietal cortex. Trends Cogn. Sci. 13(11), 488–495 (2009)
Cutsuridis, V.: A cognitive model of saliency, attention, and picture scanning. Cognitive Computation 1, 292–299 (2009)
Hikosaka, O., Takikawa, Y., Kawagoe, R.: Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80(3), 953–978 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ognibene, D., Pezzulo, G., Baldassare, G. (2010). Learning to Look in Different Environments: An Active-Vision Model Which Learns and Readapts Visual Routines. In: Doncieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, JA., Mouret, JB. (eds) From Animals to Animats 11. SAB 2010. Lecture Notes in Computer Science(), vol 6226. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15193-4_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-15193-4_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15192-7
Online ISBN: 978-3-642-15193-4
eBook Packages: Computer ScienceComputer Science (R0)