Abstract
A notion of visual information is introduced as the complexity not of the raw images, but of the images after the effects of nuisance factors such as viewpoint and illumination are discounted. It is rooted in ideas of J. J. Gibson, and stands in contrast to traditional information as entropy or coding length of the data regardless of its use, and regardless of the nuisance factors affecting it. The non-invertibility of nuisances such as occlusion and quantization induces an “information gap” that can only be bridged by controlling the data acquisition process. Measuring visual information entails early vision operations, tailored to the structure of the nuisances so as to be “lossless” with respect to visual decision and control tasks (as opposed to data transmission and storage tasks implicit in communications theory). The definition of visual information suggests desirable properties that a visual representation should possess to best accomplish vision-based decision and control tasks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aloimonos, J., Weiss, I., Bandyopadhyay, A.: Active vision. International Journal of Computer Vision 1(4), 333–356 (1988)
Alvarez, L., Guichard, F., Lions, P.L., Morel, J.M.: Axioms and fundamental equations of image processing. Arch. Rational Mechanics 123 (1993)
Arbel, T., Ferrie, F.P.: Informative views and sequential recognition. In: Conference on Computer Vision and Pattern Recognition (1995)
Arrow, K.J.: Information and economic behavior. In: Federation of Swedish Industries Stockholm, Sweden (1973)
Ayvaci, A., Raptis, M., Soatto, S.: Optical flow and occlusion detection with convex optimization. In: Proc. of Neuro Information Processing Systems (NIPS) (December 2010)
Ayvaci, A., Soatto, S.: Efficient model selection for detachable object detection. UCLA Technical Report, March 23 (2011), (submitted)
Ayvaci, A., Soatto, S.: Detachable object detection. Technical Report UCLA-CSD-100036 (November 19 February 22, 2011) (2010) (submitted)
Bajcsy, R.: Active perception. Proceedings of the IEEE 76(8), 966–1005 (1988)
Bajcsy, R., Maver, J.: Occlusions as a guide for planning the next view. IEEE Trans. Pattern Anal. Mach. Intell. 15(5) (May 1993)
Batalin, M.A., Sukhatme, G.S.: Efficient exploration without localization. In: Proceedings of IEEE International Conference on Robotics and Automation, 2003. ICRA 2003, vol. 2 (2003)
Bernardo, J.M.: Expected information as expected utility. Annals of Stat. 7(3), 686–690 (1979)
Blake, A., Yuille, A.: Active vision. MIT Press Cambridge (1993)
Bourgault, F., Makarenko, A.A., Williams, S., Grocholsky, B., Durrant-Whyte, H.: Information based adaptive robotic exploration. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), vol. 1 (2002)
Brooks, R.: Visual map making for a mobile robot. In: Proceedings of 1985 IEEE International Conference on Robotics and Automation, vol. 2 (1985)
Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2005, vol. 2 (2005)
Burns, J.B., Weiss, R.S., Riseman, E.M.: The non-existence of general-case view-invariants. In: Geometric Invariance in Computer Vision (1992)
Butler, R.B.: The effect of deprivation of visual incentives on visual exploration motivation in monkeys. Journal of Comparative and Physiological Psychology 50(2), 177 (1957)
Candès, E.J., Donoho, D.L.: Curvelets, multiresolution representation, and scaling laws. In: Wavelet Applications in Signal and Image Processing (2000)
Caselles, V., Coll, B., Morel, J.-M.: Topographic maps and local contrast changes in natural images. Int. J. Comput. Vision 33(1), 5–27 (1999)
Castro, R., Kalish, C., Nowak, R., Qian, R., Rogers, T., Zhu, X.: Human active learning. In: Proc. of NIPS (2008)
Chen, H.F., Belhumeur, P.N., Jacobs, D.W.: In search of illumination invariants. In: Proc. IEEE Conf. on Comp. Vision and Pattern Recogn. (2000)
Claxton, K., Neumann, P.J., Araki, S., Weinstein, M.C.: Bayesian value-of-information analysis. International Journal of Technology Assessment in Health Care 17(01), 38–55 (2001)
Cremers, D., Soatto, S.: Motion competition: a variational approach to piecewise parametric motion segmentation. Intl. J. of Comp. Vision, 249–265 (May 2005)
Da Cunha, A.L., Do, M.N., Vetterli, M.: On the information rates of the plenoptic function. In: ICIP, Atlanta, GA (2006)
Davison, A.J., Murray, D.W.: Simultaneous localization and map-building using active vision. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 865–880 (2002)
Denzler, J., Brown, C.M.: Information theoretic sensor data selection for active object recognition and state estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(2), 145–157 (2002)
Fogel, E., Huang, Y.F.: Value of Information in System Identification-Bounded Noise Case. Automatica 18(2), 229–238 (1982)
Franz, M.O., Schölkopf, B., Mallot, H.A., Bülthoff, H.H.: Learning view graphs for robot navigation. Autonomous Robots 5(1), 111–125 (1998)
Gibson, J.J.: The theory of information pickup. In: Contemp. Theory and Research in Visual Perception, p. 662 (1968)
Gibson, J.J.: The myths of passive perception. Philosophy and Phenomenological Research 37(2), 234–238 (1976)
Gibson, J.J.: The ecological approach to visual perception. LEA (1984)
Golubitsky, M., Guillemin, V.: Stable mappings and their singularities. Graduate texts in mathematics, vol. 14 (1974)
Good, I.J., Osteyee, D.B.: Information, weight of evidence. The singularity between probability measures and signal detection. Springer, Heidelberg (1974)
Gould, J.P.: Risk, stochastic preference, and the value of information. Journal of Economic Theory 8(1), 64–84 (1974)
Guo, C., Zhu, S., Wu, Y.N.: Toward a mathematical theory of primal sketch and sketchability. In: Proc. 9th Int. Conf. on Computer Vision (2003)
Huang, J., Mumford, D.: Statistics of natural images and models. In: Proc. CVPR, pp. 541–547 (1999)
Hughes, S.B., Lewis, M.: Task-driven camera operations for robotic exploration. IEEE Transactions on Systems, Man and Cybernetics, Part A 35(4), 513–522 (2005)
Itti, L., Koch, C.: Computational modelling of visual attention. Nature Rev. Neuroscience 2(3), 194–203 (2001)
James, K.: On some possible characteristics of information in J. J. Gibson’s ecological approach to visual perception. Leonardo 13(2) (1980)
Jin, H., Soatto, S., Yezzi, A.: Multi-view stereo reconstruction of dense shape and complex appearance. Intl. J. of Comp. Vis. 63(3), 175–189 (2005)
Jojic, N., Frey, B., Kannan, A.: Epitomic analysis of apperance and shape. In: Proc. ICCV (2003)
Jones, E., Soatto, S.: Visual-inertial navigation, localization and mapping: A scalable real-time large-scale approach. Intl. J. of Robotics Research (January 17, 2011)
Jones, S.D., Andersen, C., Crowley, J.L.: Appearance based processes for visual navigation. In: Processings of the 5th International Symposium on Intelligent Robotic Systems (SIRS 1997), pp. 551–557 (1997)
Karatzas, I., Shreve, S.E.: Brownian Motion and Stochastic Calculus. Springer, Heidelberg (1988)
Keeler, K.C.: Map representation and optimal encoding for image segmentation. PhD dissertation. Harvard University (October 1990)
Kunita, H.: Stochastic differential equations on manifolds. Cambridge University Press (1991)
Kutulakos, K.N., Dyer, C.R.: Global surface reconstruction by purposive control of observer motion. Artificial Intelligence 78(1-2), 147–177 (1995)
Kutulakos, K.N., Jagersand, M.: Exploring objects by invariant-based tangential viewpoint control. In: Proceedings of International Symposium on Computer Vision, pp. 503–508 (1995)
Lee, T., Soatto, S.: Learning and matching multiscale template descriptors for real-time detection, localization and tracking. In: Proc. IEEE Conf. on Comp. Vision and Pattern Recogn., pp. 1457–1464 (2011)
Lindeberg, T.: Edge Detection and Ridge Detection with Automatic Scale Selection, vol. 30. Cambridge University Press (1998)
Lindeberg, T.: Principles for automatic scale selection. Technical report, KTH, Stockholm, CVAP (1998)
Lindley, D.V.: On a measure of the information provided by an experiment. Annals of Math. Stat. 27(4), 986–1005 (1956)
Ma, Y., Soatto, S., Kosecka, J., Sastry, S.: An invitation to 3D vision, from images to geometric models. Springer, Heidelberg (2003)
Marr, D.: Vision. W.H.Freeman & Co. (1982)
Marschak, J.: Remarks on the economics of information. Contributions to Scientific Research in Management (1960)
Mikolajczyk, K., Schmid, C.: An Affine Invariant Interest Point Detector. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 128–142. Springer, Heidelberg (2002)
Mumford, D., Gidas, B.: Stochastic models for generic images. Quarterly of Applied Mathematics 54(1), 85–111 (2001)
Newman, P., Ho, K.: SLAM-loop closing with visually salient features. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation. ICRA 2005, pp. 635–642 (2005)
Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy employed by V1? In: Vision Research (1998)
Peruch, P., Vercher, J.-L., Gauthier, G.M.: Acquisition of spatial knowledge through visual exploration of simulated environments. Ecological Psychology 7(1), 1–20 (1995)
Pito, R., Co, I.T., Boston, M.A.: A solution to the next best view problem for automated surface acquisition. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(10), 1016–1030 (1999)
Pretto, A., Chiuso, A., Soatto, S.: Sufficient exploration for navigation and recognition. UCLA Technical Report (January 17, 2011) (submitted)
Robert, C.P.: The Bayesian Choice. Springer, New York (2001)
Se, S., Lowe, D., Little, J.: Vision-based mobile robot localization and mapping using scale-invariant features. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA (2001)
Shao, J.: Mathematical Statistics. Springer, Heidelberg (1998)
Sim, R., Dudek, G.: Effective exploration strategies for the construction of visual maps. In: Proceedings of 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003), vol. 4 (2003)
Soatto, S.: Actionable information in vision. In: Proc. of the Intl. Conf. on Comp. Vision (October 2009)
Soatto, S.: Steps Toward a Theory of Visual Information. Technical Report UCLA-CSD100028 (September 13, 2010)
Soatto, S.: Texture, structure and visual matching. Technical Report UCLA-CSD-100033 (October 11, 2010)
Soatto, S., Chiuso, A.: Controlled recognition bounds for scaling and occlusion channels. In: Proc. of the Data Compression Conference (March 2011)
Stachniss, C., Grisetti, G., Burgard, W.: Information gain-based exploration using rao-blackwellized particle filters. In: Proc. of RSS (2005)
Sundaramoorthi, G., Petersen, P., Soatto, S.: On the set of images modulo viewpoint and contrast changes. Technical Report UCLA-CSD090005 (2009) (submitted)
Taylor, C.J., Kriegman, D.J.: Vision-based motion planning and exploration algorithms for mobile robots. IEEE Trans. on Robotics and Automation 14(3), 417–426 (1998)
Thrun, S.: Learning metric-topological maps for indoor mobile robot navigation. Artificial Intelligence 99(1), 21–71 (1998)
Tishby, N., Pereira, F.C., Bialek, W.: The information bottleneck method. In: Proc. of the Allerton Conf. (2000)
Todorovic, S., Ahuja, N.: Extracting subimages of an unknown category from a set of images. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Recog. (2006)
Turing, A.M.: The chemical basis of morphogenesis. Phil. Trans. of the Royal Society of London, ser. B (1952)
Vedaldi, A., Soatto, S.: Features for recognition: viewpoint invariance for non-planar scenes. In: Proc. of the Intl. Conf. of Comp. Vision, pp. 1474–1481 (October 2005)
Vedaldi, A., Soatto, S.: Quick Shift and Kernel Methods for Mode Seeking. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 705–718. Springer, Heidelberg (2008)
Wertheimer, M.: Laws of organization in perceptual forms. In: Ellis, W.D. (ed.) A Sourcebook of Gestalt Psychology, pp. 331–363. Harcourt, Brace and Company (1939)
Whaite, P., Ferrie, F.P.: From uncertainty to visual exploration. IEEE Transactions on Pattern Analysis and Machine Intelligence 13(10), 1038–1049 (1991)
Wiener, N.: Cybernetics, or Control and Communication in Men and Machines. MIT Press (1949)
Wu, Y.N., Guo, C., Zhu, S.C.: From information scaling of natural images to regimes of statistical models. Quarterly of Applied Mathematics 66, 81–122 (2008)
Zhang, H., Ostrowski, J.P.: Visual motion planning for mobile robots. IEEE Transactions on Robotics and Automation 18(2), 199–208 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Berlin Heidelberg
About this chapter
Cite this chapter
Soatto, S. (2013). Actionable Information in Vision. In: Cipolla, R., Battiato, S., Farinella, G. (eds) Machine Learning for Computer Vision. Studies in Computational Intelligence, vol 411. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28661-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-28661-2_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28660-5
Online ISBN: 978-3-642-28661-2
eBook Packages: EngineeringEngineering (R0)