Abstract
The innovation of this work is the provision of a system that learns visual encodings of attention patterns and that enables sequential attention for object detection in real world environments. The system embeds the saccadic decision procedure in a cascaded process where visual evidence is probed at the most informative image locations. It is based on the extraction of information theoretic saliency by determining informative local image descriptors that provide selected foci of interest. Both the local information in terms of code book vector responses, and the geometric information in the shift of attention contribute to the recognition state of a Markov decision process. A Q-learner performs then explorative search on useful actions towards salient locations, developing a strategy of useful action sequences being directed in state space towards the optimization of information maximization. The method is evaluated in experiments on real world object recognition and demonstrates efficient performance in outdoor tasks.
This work is supported in part by the European Commission funded projects MACS under grant number FP6-004381 and MOBVIS under grant number FP6-511051, and by the FWF Austrian Joint Research Project Cognitive Vision under sub-project S9104-N13.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bandera, C., Vico, F.J., Bravo, J.M., Harmon, M.E., Baird, L.C.: Residual Q-learning applied to visual attention. In: International Conference on Machine Learning, pp. 20–27 (1996)
Deco, G.: The computational neuroscience of visual cognition: Attention, memory and reward. In: Proc. International Workshop on Attention and Performance in Computational Vision, pp. 49–58 (2004)
Deubel, H.: Localization of targets across saccades: Role of landmark objects. Visual Cognition (11), 173–202 (2004)
Fritz, G., Paletta, L., Bischof, H.: Object recognition using local information content. In: ICPR 2004. Proc. International Conference on Pattern Recognition, Cambridge, UK, vol. II, pp. 15–18 (2004)
Fritz, G., Seifert, C., Paletta, L., Bischof, H.: Rapid object recognition from discriminative regions of interest. In: AAAI 2004. Proc. National Conference on Artificial Intelligence, San Jose, CA, pp. 444–449 (2004)
Fritz, G., Seifert, C., Paletta, L., Bischof, H.: Building recognition using informative local descriptors from mobile imagery. In: Kalviainen, H., Parkkinen, J., Kaarna, A. (eds.) SCIA 2005. LNCS, vol. 3540, Springer, Heidelberg (in print)
Gorea, A., Sagi, D.: Selective attention as the substrate of optimal decision behaviour in environments with multiple stimuli. In: Proc. European Conference on Visual Perception (2003)
Henderson, J.M.: Human gaze control in real-world scene perception. Trends in Cognitive Sciences 7, 498–504 (2003)
Itti, L., Koch, C.: Computational modeling of visual attention. Nature Reviews Neuroscience 2(3), 194–203 (2001)
Li, M., Clark, J.J.: Learning of position and attention-shift invariant recognition across attention shifts. In: Proc. International Workshop on Attention and Performance in Computational Vision, pp. 41–48 (2004)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
Minut, S., Mahadevan, S.: A reinforcement learning model of selective visual attention. In: Proc. International Conference on Autonomous Agents, pp. 457–464 (2001)
Puterman, M.L.: Markov Decision Processes. John Wiley & Sons, New York, NY (1994)
Rensink, R.A., O’Regan, J.K., Clark, J.J.: To see or not to see: The need for attention to perceive changes in scenes. Psychological Science 8, 368–373 (1997)
Rybak, I.A., Gusakova, I.V., Golovan, A.V., Podladchikova, L.N., Shevtsova, N.A.: A model of attention-guided visual perception and recognition. Vision Research 38, 2387–2400 (1998)
Schall, J.D., Thompson, K.G.: Neural selection and control of visually guided eye movements. Annual Review of Neuroscience 22(22), 241–259 (1999)
Stark, L.W., Choi, Y.S.: Experimental metaphysics: The scanpath as an epistemological mechanism. In: Zangemeister, W.H., Stiehl, H.S., Freska, C. (eds.) Visual attention and cognition, pp. 3–69. Elsevier Science, Amsterdam, Netherlands (1996)
Tipper, S.P., Grisson, S., Kessler, K.: Long-term inhibition of return of attention. Psychological Science 14, 19–25–105 (2003)
Watkins, C., Dayan, P.: Q-learning. Machine Learning 8(3,4), 279–292 (1992)
Weber, M., Welling, M., Perona, P.: Unsupervised learning of models for recognition. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 18–32. Springer, Heidelberg (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Paletta, L., Fritz, G. (2007). Reinforcement Learning for Decision Making in Sequential Visual Attention . In: Paletta, L., Rome, E. (eds) Attention in Cognitive Systems. Theories and Systems from an Interdisciplinary Viewpoint. WAPCV 2007. Lecture Notes in Computer Science(), vol 4840. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77343-6_19
Download citation
DOI: https://doi.org/10.1007/978-3-540-77343-6_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77342-9
Online ISBN: 978-3-540-77343-6
eBook Packages: Computer ScienceComputer Science (R0)