Skip to main content
SpringerLink
Log in

Information: Theoretical Model for Saliency Prediction—Application to Attentive CBIR

  • Chapter
  • First Online:
Book cover Visual Content Indexing and Retrieval with Psycho-Visual Models

Part of the book series: Multimedia Systems and Applications ((MMSA))

  • 421 Accesses

Abstract

This work presents an original informational approach to extract visual information, model attention and evaluate the efficiency of the results. Even if the extraction of salient and useful information, i.e. observation, is an elementary task for human and animals, its simulation is still an open problem in computer vision. In this article, we define a process to derive optimal laws to extract visual information without any constraints or a priori. Starting from saliency definition and measure through the prism of information theory, we present a framework in which we develop an ecological inspired approach to model visual information extraction. We demonstrate that our approach provides a fast and highly configurable model, moreover it is as plausible as existing models designed for high biological fidelity. It proposes an adjustable trade-off between nondeterministic attentional behavior and properties of stability, reproducibility and reactiveness. We apply this approach to enhance the performance in an object recognition task. As a conclusion, this article proposes a theoretical framework to derive an optimal model validated by many experimentations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    EPI used in an open system.

  2. 2.

    Bruce database is available at http://www-sop.inria.fr/members/Neil.Bruce.

  3. 3.

    Le Meur database is available at http://www.irisa.fr/temics/staff/lemeur/visualAttention.

  4. 4.

    White Box Testing is a software testing method in which the internal structure/design/ implementation of the item being tested is known to the tester.

References

  1. Berthoz, A.: La simplexité, odile jacob edn. Paris (2009)

    Google Scholar 

  2. Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell. 99(Xxx) (2012). doi: 10.1109/TPAMI.2012.89. http://doi.ieeecomputersociety.org/10.1109/TPAMI.2012.89?utm_source=dlvr.it&utm_medium=feed

  3. Broadbent, D.E.: Perception and Communication. Pergamon Press, Elmsford, NY (1958)

    Book  Google Scholar 

  4. Bruce, B., Jernigan, E.: Evolutionary design of context-free attentional operators. In: Proceedings of ICIP’03, pp. 0–3. Citeseer (2003)

    Google Scholar 

  5. Bruce, N.D.B., Tsotsos, J.K.: Spatiotemporal saliency: towards a hierarchical representation of visual saliency. In: Proceedings of the 5th International Workshop on Attention in Cognitive Systems, pp. 98–111. Springer, Heidelberg (2008)

    Google Scholar 

  6. Bruce, N.D.B., Tsotsos, J.K.: Saliency, attention, and visual search: an information theoretic approach. J. Vis. 9(3), 5 (2009)

    Article  Google Scholar 

  7. Cabezas, H., Fath, B.D.: Towards a theory of sustainable systems. Fluid Phase Equilib. 194–197, 3–14 (2002)

    Article  Google Scholar 

  8. Courboulay, V.: Une nouvelle approche variationnelle du traitement d’images. Application à la coopération détection-reconstruction. Ph.D. thesis, La Rochelle (2002)

    Google Scholar 

  9. Courboulay, V., Mancas, M.: CuriousMind photographer: distract the robot from its initial task. EAI Endorsed Trans. Creative Technol. 2(2), 1–9 (2014). doi:10.4108/ct.2.2.e4. https://hal.archives-ouvertes.fr/hal-01062621

  10. Diamant, E.: Modeling human-like intelligent image processing: an information processing perspective and approach. Signal Process. Image Commun. 22(6), 583–590 (2007). doi:10.1016/j.image.2007.05.007 http://linkinghub.elsevier.com/retrieve/pii/S0923596507000781

  11. Diamant, E.: Unveiling the mystery of visual information processing in human brain. Brain Res. 1225, 171–178 (2008). doi:10.1016/j.brainres.2008.05.017. http://www.ncbi.nlm.nih.gov/pubmed/18585686

  12. Diamant, E., Box, P.O., Ono, K.: Does a plane imitate a bird? Does computer vision have to follow biological paradigms? In: Vision, and Artificial Intelligence, First International Symposium Proceedings. Lecturer Notes in Computer Science, pp. 108–115. Springer, Berlin/Heidelberg (2005)

    Google Scholar 

  13. Everingham, M., Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2009). doi:10.1007/s11263-009-0275-4. http://www.springerlink.com/index/10.1007/s11263-009-0275-4

  14. Fath, B.: Exergy and Fisher Information as ecological indices. Ecol. Model. 174(1-2), 25–35 (2004). doi: 10.1016/j.ecolmodel.2003.12.045. http://linkinghub.elsevier.com/retrieve/pii/S0304380003005660

  15. Fath, B.D., Cabezas, H., Pawlowski, C.W.: Regime changes in ecological systems: an information theory approach. J. Theor. Biol. 222, 517–530 (2003)

    Article  MathSciNet  Google Scholar 

  16. Fawcett, T.: An introduction to ROC analysis. Pattern Recogn. Lett. 27(8), 861–874 (2006). doi:10.1016/j.patrec.2005.10.010. http://linkinghub.elsevier.com/retrieve/pii/S016786550500303X

  17. Foo, J.J.: Pruning SIFT for scalable near-duplicate image matching. In: Australasian Database Conference, Ballarat, p. 9 (2007)

    Google Scholar 

  18. Fox, M.D., Snyder, A.Z., Vincent, J.L., Raichle, M.E.: Intrinsic fluctuations within cortical systems account for intertrial variability in human behavior. Neuron 56(1), 171–184 (2007). doi:10.1016/j.neuron.2007.08.023

    Article  Google Scholar 

  19. Frieden, B.R.: Physics from Fisher Information: A Unification. Cambridge University Press, Cambridge (1998)

    Book  MATH  Google Scholar 

  20. Frieden, B.R.: Science from Fisher Information: A Unification, Cambridge edn. Cambridge University Press, Cambridge (2004). http://www.amazon.com/Science-Fisher-Information-Roy-Frieden/dp/0521009111

    Book  MATH  Google Scholar 

  21. Frintrop, S.: Towards attentive robots. Paladyn. J. Behav. Robot. 2, 64–70 (2011). doi: 10.2478/s13230-011-0018-4, http://dx.doi.org/10.2478/s13230-011-0018-4

  22. Frintrop, S., Jensfelt, P.: Attentional landmarks and active gaze control for visual slam. IEEE Trans. Robot. 24(5), 1054–1065 (2008). doi:10.1109/TRO.2008.2004977

    Article  Google Scholar 

  23. Frintrop, S., Klodt, M., Rome, E.: A real-time visual attention system using integral images. In: 5th International Conference on Computer Vision Systems (ICVS). Applied Computer Science Group, Bielefeld (2007)

    Google Scholar 

  24. Gilles, S.: Description and Experimentation of Image Matching Using Mutual Information. Robotics Research Group, Oxford University (1996)

    Google Scholar 

  25. Histace, A., Ménard, M., Courboulay, V.: Selective image diffusion for oriented pattern extraction. In: 4th International Conference on Informatics in Control, Automation and Robotics (ICINCO), France (2008). http://hal.archives-ouvertes.fr/hal-00377679/en/

  26. Histace, A., Ménard, M., Cavaro-ménard, C.: Selective diffusion for oriented pattern extraction: application to tagged cardiac MRI enhancement. Pattern Recogn. Lett. 30(15), 1356–1365 (2009). doi:10.1016/j.patrec.2009.07.012. http://dx.doi.org/10.1016/j.patrec.2009.07.012

  27. Itti, L., Koch, C., Niebur, E., Others: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  28. Kadir, T., Brady, M.: Saliency, scale and image description. Int. J. Comput. Vis. 45(2), 83–105 (2001)

    Article  MATH  Google Scholar 

  29. Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4(4), 219–227 (1985)

    Google Scholar 

  30. Kondor, R., Jebara, T.: A kernel between sets of vectors. Mach. Learn. 361–368 (2003)

    Google Scholar 

  31. Laaksonen, J.: PicSOM? Content-based image retrieval with self-organizing maps. Pattern Recogn. Lett. 21(13/14), 1199–1207 (2000). doi:10.1016/S0167-8655(00)00082-9. http://linkinghub.elsevier.com/retrieve/pii/S0167865500000829

  32. Le Meur, O., Le Callet, P., Dominique, B., Thoreau, D.: A coherent computational approach to model bottom-up visual attention. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 802–817 (2006)

    Article  Google Scholar 

  33. Lesser, M., Dinah, M.: Mind as a dynamical system: implications for autism. In: Psychobiology of Autism: Current Research & Practice (1998). http://www.autismusundcomputer.de/mind.en.html

  34. Lindeberg, T.: Feature detection with automatic scale selection. Comput. Vis. 30(2), 96 (1998)

    Google Scholar 

  35. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004). doi:10.1023/B:VISI.0000029664.99615.94. http://www.springerlink.com/openurl.asp?id=doi:10.1023/B:VISI.0000029664.99615.94

  36. Mancas, M.: Computational attention: towards attentive computers. Ph.D., Faculté Polytechnique de Mons (2007)

    Google Scholar 

  37. Mikolajczyk, K.: Scale & affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004). doi:10.1023/B:VISI.0000027790.02288.f2. http://www.springerlink.com/openurl.asp?id=doi:10.1023/B:VISI.0000027790.02288.f2

  38. Murray, J.D.: Mathematical Biology: An Introduction. Springer, Berlin/Heidelberg (2003)

    Google Scholar 

  39. Murray, J.D.: Mathematical Biology: Spatial Models and Biomedical Applications. Springer, New York (2003)

    MATH  Google Scholar 

  40. Neisser, U.: Cognitive Psychology. Appleton-Century-Crofts, New York (1967)

    Google Scholar 

  41. Ouerhani, N., Hugli, H.: A model of dynamic visual attention for object tracking in natural image sequences. Lecture Notes in Computer Science, pp. 702–709. Springer, Berlin (2003)

    Google Scholar 

  42. Park, S.J., An, K.H., Lee, M.: Saliency map model with adaptive masking based on independent component analysis. Neurocomputing 49(1), 417–422 (2002)

    Article  Google Scholar 

  43. Perreira Da Silva, M., Courboulay, V.: Implementation and evaluation of a computational model of attention for computer vision. In: Developing and Applying Biologically-Inspired Vision Systems: Interdisciplinary Concepts, pp. 273–306. IGI Global, Hershey (2012)

    Google Scholar 

  44. Perreira Da Silva, M., Courboulay, V., Estraillier, P.: Objective validation of a dynamical and plausible computational model of visual attention. In: IEEE European Workshop on Visual Information Processing, Paris, pp. 223–228 (2011)

    Google Scholar 

  45. Perreira Da Silva, M., Courboulay, V., Estraillier, P.: Une nouvelle mesure de complexité pour les images basée sur l’attention visuelle. In: GRETSI, Bordeaux (2011)

    Google Scholar 

  46. Pisharady, P., Vadakkepat, P., Loh, A.: Attention based detection and recognition of hand postures against complex backgrounds. Int. J. Comput. Vis. 1–17 (2012). doi:10.1007/s11263-012-0560-5. http://dx.doi.org/10.1007/s11263-012-0560-5

  47. Santini, S., Gupta, A., Jain, R.: Emergent semantics through interaction in image databases. IEEE Trans. Knowl. Data Eng. 13(3), 337–351 (2001). doi:10.1109/69.929893. http://dx.doi.org/10.1109/69.929893

  48. Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering objects and their location in images. In: Proceeding of the International Conference on Computer Vision, vol. 1, pp. 370–377 (2005)

    Google Scholar 

  49. Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J. Vis. 7, 1–17 (2007). doi:10.1167/7.14.4.Introduction

    Article  Google Scholar 

  50. Treisman, A.: Strategies and models of selective attention. Psychol. Rev. 76, 282–299 (1969)

    Article  Google Scholar 

  51. Treisman, A., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 136(12), 97–136 (1980)

    Article  Google Scholar 

  52. Treisman, A.M., Kanwisher, N.G.: Perceiving visually presented objets: recognition, awareness, and modularity. Curr. Opin. Neurobiol. 8(2), 218–226 (1998). http://linkinghub.elsevier.com/retrieve/pii/S0959438898801438

    Article  Google Scholar 

  53. Tsotsos, J.K., Culhane, S.M., Kei Wai, W.Y., Lai, Y., Davis, N., Nuflo, F.: Modeling visual attention via selective tuning. Artif. Intell. 78(1-2), 507–545 (1995)

    Article  Google Scholar 

  54. Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends Comput. Graph. Vis. 3(3), 177–280 (2007). doi:10.1561/0600000017. http://www.nowpublishers.com/product.aspx?product=CGV&doi=0600000017

  55. Volterra, V.: Variations and fluctuations of the number of individuals in animal species living together. ICES J. Mar. Sci. 3(1), 3–51 (1928)

    Article  Google Scholar 

  56. Walther, D.: Selective visual attention enables learning and recognition of multiple objects in cluttered scenes. Comput. Vis. Image Underst. 100(1-2), 41–63 (2005). doi:10.1016/j.cviu.2004.09.004

    Article  Google Scholar 

  57. Wolfe, J.M., Cave, K.R., Franzel, S.L.: Guided search: an alternative to the feature integration model for visual search. J. Exp. Psychol. Hum. Percept. Perform. 15(3), 419–433 (1989). http://www.ncbi.nlm.nih.gov/pubmed/2527952

    Article  Google Scholar 

  58. Zhao, X., Hou, Y., Song, D., Li, W.: Extending the extreme physical information to universal cognitive models via a confident information first principle. Entropy 16(7), 3670–3688 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. L3i - University of La Rochelle, 17000 La Rochelle, France

    Vincent Courboulay & Arnaud Revel

Authors
  1. Vincent Courboulay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arnaud Revel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vincent Courboulay .

Editor information

Editors and Affiliations

  1. LaBRI UMR 5800, Univ. Bordeaux, CNRS, Bordeaux INP, Univ. Bordeaux, Talence, France

    Jenny Benois-Pineau

  2. LS2N, UMR CNRS 6004, Université de Nantes, Nantes Cedex 3, France

    Patrick Le Callet

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Courboulay, V., Revel, A. (2017). Information: Theoretical Model for Saliency Prediction—Application to Attentive CBIR. In: Benois-Pineau, J., Le Callet, P. (eds) Visual Content Indexing and Retrieval with Psycho-Visual Models. Multimedia Systems and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-57687-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-57687-9_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-57686-2

  • Online ISBN: 978-3-319-57687-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation