Skip to main content
SpringerLink
Log in

Effects of individual’s emotions on saliency and visual search

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Psychological studies have already proven a connection between emotional stimuli and visual attention. However, there is a lack of evidence of how much an individual’s mood influences visual information processing of emotionally neutral stimuli. In contrast to prior studies, we explored the effect of positive mood on bottom-up low-level visual saliency. In our extensive experimental studies, we induced positive or neutral emotions in 106 subjects using a psychological method of autobiographical memories and listening to positive music and then performed an eye-tracking study with three task types—free viewing, memorizing the image content and visual search tasks. We explored differences in the human gaze behaviour between both emotions and relate them to the bottom-up attention correlation between emotionally aroused and non-aroused subjects. We observed that positive emotions induce a strong broadening and engagement effect on the bottom-up attention while performing all task types. On the other hand, we found out that certain tasks are solved less efficiently when experiencing a positive mood. Therefore, we claim that positive mood may also act as a distractor while performing certain types of tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://vgg.fiit.stuba.sk/2018-08/emotions-attention/.

  2. Tobii X2-60 specification available online on January 2020 at: https://www.tobiipro.com/siteassets/tobii-pro/user-manuals/tobii-pro-x2-60-eye-tracker-user-manual.pdf/?v=1.0.3.

References

  1. Armony, J.L., Dolan, R.J.: Modulation of spatial attention by fear-conditioned stimuli: an event-related fmri study. Neuropsychologia 40(7), 817–826 (2002)

    Google Scholar 

  2. Barbot, A., Carrasco, M.: Emotion and anxiety potentiate the way attention alters visual appearance. Sci. Rep. 8(1), 5938 (2018)

    Google Scholar 

  3. Becker, M.W., Leinenger, M.: Attentional selection is biased toward mood-congruent stimuli. Emotion 11(5), 1248 (2011)

    Google Scholar 

  4. Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 185–207 (2013)

    Google Scholar 

  5. Borji, A., Itti, L.: Cat2000: a large scale fixation dataset for boosting saliency research. arXiv preprint arXiv:1505.03581 (2015)

  6. Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)

    Google Scholar 

  7. Bylinskii, Z., Judd, T., Oliva, A., Torralba, A., Durand, F.: What do different evaluation metrics tell us about saliency models? arXiv preprint arXiv:1604.03605 (2016)

  8. Bylinskii, Z., Judd, T., Oliva, A., Torralba, A., Durand, F.: What do different evaluation metrics tell us about saliency models? IEEE Trans. Pattern Anal. Mach. Intell. 41(3), 740–757 (2018)

    Google Scholar 

  9. Ding, X., Huang, L., Li, B., Lang, C., Hua, Z., Wang, Y.: A novel emotional saliency map to model emotional attention mechanism. In: International Conference on Multimedia Modeling, pp. 197–206. Springer, Berlin (2016)

  10. Egermann, H., Fernando, N., Chuen, L., McAdams, S.: Music induces universal emotion-related psychophysiological responses: comparing Canadian listeners to Congolese Pygmies. Front. Psychol. 5, 1341 (2015)

    Google Scholar 

  11. Eysenck, M.W., Keane, M.T.: Cognitive Psychology: A Student’s Handbook. Psychology Press, London (2013)

    Google Scholar 

  12. Fan, S., Jiang, M., Shen, Z., Koenig, B.L., Kankanhalli, M.S., Zhao, Q.: The role of visual attention in sentiment prediction. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 217–225 (2017)

  13. Fan, S., Shen, Z., Jiang, M., Koenig, B.L., Xu, J., Kankanhalli, M.S., Zhao, Q.: Emotional attention: a study of image sentiment and visual attention. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7521–7531 (2018)

  14. Ferrer, R.A., Stanley, J.T., Graff, K., Klein, W.M., Goodman, N., Nelson, W.L., Salazar, S.: The effect of emotion on visual attention to information and decision making in the context of informed consent process for clinical trials. J. Behav. Decis. Mak. 29(2–3), 245–253 (2016)

    Google Scholar 

  15. Fredrickson, B.L.: The broaden-and-build theory of positive emotions. Philos. Trans. R. Soc. B Biol. Sci. 359(1449), 1367 (2004)

    Google Scholar 

  16. Fredrickson, B.L., Branigan, C.: Positive emotions broaden the scope of attention and thought-action repertoires. Cogn. Emot. 19(3), 313–332 (2005)

    Google Scholar 

  17. Goldstein, E.B.: Encyclopedia of Perception, vol. 1. Sage, London (2010)

    Google Scholar 

  18. Grol, M., Koster, E.H., Bruyneel, L., De Raedt, R.: Effects of positive mood on attention broadening for self-related information. Psychol. Res. 78(4), 566–573 (2014)

    Google Scholar 

  19. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2007)

  20. Holmes, E., Hackmann, A.: A healthy imagination? Editorial for the special issue of memory: mental imagery and memory in psychopathology. Memory 12(4), 387–388 (2004)

    Google Scholar 

  21. Hsu, J.L., Zhen, Y.L., Lin, T.C., Chiu, Y.S.: Affective content analysis of music emotion through eeg. Multimed. Syst. 24(2), 195–210 (2018)

    Google Scholar 

  22. Jefferies, L.N., Smilek, D., Eich, E., Enns, J.T.: Emotional valence and arousal interact in attentional control. Psychol. Sci. 19(3), 290–295 (2008)

    Google Scholar 

  23. Jiang, J., Scolaro, A.J., Bailey, K., Chen, A.: The effect of music-induced mood on attentional networks. Int. J. Psychol. 46(3), 214–222 (2011)

    Google Scholar 

  24. Judd, T., Durand, F., Torralba, A.: A benchmark of computational models of saliency to predict human fixations. In: MIT Technical Report (2012)

  25. Karim, J., Weisz, R., Rehman, S.U.: International positive and negative affect schedule short-form (I-PANAS-SF): testing for factorial invariance across cultures. Procedia Soc. Behav. Sci. 15, 2016–2022 (2011)

    Google Scholar 

  26. Kaspar, K., Gameiro, R.R., König, P.: Feeling good, searching the bad: positive priming increases attention and memory for negative stimuli on webpages. Comput. Hum. Behav. 53, 332–343 (2015)

    Google Scholar 

  27. Killgore, W.D., Yurgelun-Todd, D.A.: Positive affect modulates activity in the visual cortex to images of high calorie foods. Int. J. Neurosci. 117(5), 643–653 (2007)

    Google Scholar 

  28. Krahé, B., Bieneck, S.: The effect of music-induced mood on aggressive affect, cognition, and behavior 1. J. Appl. Soc. Psychol. 42(2), 271–290 (2012)

    Google Scholar 

  29. Lang, P.J., Kozak, M.J., Miller, G.A., Levin, D.N., McLean Jr., A.: Emotional imagery: conceptual structure and pattern of somato-visceral response. Psychophysiology 17(2), 179–192 (1980)

    Google Scholar 

  30. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: common objects in context. In: European Conference on Computer Vision, pp. 740–755. Springer, Berlin (2014)

  31. Liu, H., Xu, M., Wang, J., Rao, T., Burnett, I.: Improving visual saliency computing with emotion intensity. IEEE Trans. Neural Netw. Learn. Syst. 27(6), 1201–1213 (2016)

    MathSciNet  Google Scholar 

  32. Maekawa, T., Anderson, S.J., de Brecht, M., Yamagishi, N.: The effect of mood state on visual search times for detecting a target in noise: an application of smartphone technology. PLoS ONE 13(4), e0195865 (2018)

    Google Scholar 

  33. Mills, C., D’Mello, S.: On the validity of the autobiographical emotional memory task for emotion induction. PLoS ONE 9(4), e95837 (2014)

    Google Scholar 

  34. Miyazawa, S., Iwasaki, S.: Effect of negative emotion on visual attention: automatic capture by fear-related stimuli. Jpn. Psychol. Res. 51(1), 13–23 (2009)

    Google Scholar 

  35. Most, S.B., Smith, S.D., Cooter, A.B., Levy, B.N., Zald, D.H.: The naked truth: positive, arousing distractors impair rapid target perception. Cogn. Emot. 21(5), 964–981 (2007)

    Google Scholar 

  36. Öhman, A., Flykt, A., Esteves, F.: Emotion drives attention: detecting the snake in the grass. J. Exp. Psychol. Gen. 130(3), 466 (2001)

    Google Scholar 

  37. Olivers, C.N., Nieuwenhuis, S.: The beneficial effect of concurrent task-irrelevant mental activity on temporal attention. Psychol. Sci. 16(4), 265–269 (2005)

    Google Scholar 

  38. Olsen, A.: The Tobii I-VT Fixation Filter. Tobii Technology, Stockholm (2012)

    Google Scholar 

  39. Pêcher, C., Lemercier, C., Cellier, J.M.: Emotions drive attention: effects on driver’s behaviour. Saf. Sci. 47(9), 1254–1259 (2009)

    Google Scholar 

  40. Phelps, E.A., Ling, S., Carrasco, M.: Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychol. Sci. 17(4), 292–299 (2006)

    Google Scholar 

  41. Ramanathan, S., Katti, H., Sebe, N., Kankanhalli, M., Chua, T.S.: An eye fixation database for saliency detection in images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) Computer Vision—ECCV 2010, pp. 30–43. Springer, Berlin (2010)

  42. Rowe, G., Hirsh, J.B., Anderson, A.K.: Positive affect increases the breadth of attentional selection. Proc. Natl. Acad. Sci. 104(1), 383–388 (2007)

    Google Scholar 

  43. Smilek, D., Enns, J.T., Eastwood, J.D., Merikle, P.M.: Relax! Cognitive strategy influences visual search. Vis. Cogn. 14(4–8), 543–564 (2006)

    Google Scholar 

  44. Sutherland, M.R., McQuiggan, D.A., Ryan, J.D., Mather, M.: Perceptual salience does not influence emotional arousal’s impairing effects on top–down attention. Emotion 17(4), 700 (2017)

    Google Scholar 

  45. Talarico, J.M., Berntsen, D., Rubin, D.C.: Positive emotions enhance recall of peripheral details. Cogn. Emot. 23(2), 380–398 (2009)

    Google Scholar 

  46. Tsujimura, A., Miyagawa, Y., Takada, S., Matsuoka, Y., Takao, T., Hirai, T., Matsushita, M., Nonomura, N., Okuyama, A.: Sex differences in visual attention to sexually explicit videos: a preliminary study. J. Sex. Med. 6(4), 1011–1017 (2009)

    Google Scholar 

  47. Wadlinger, H.A., Isaacowitz, D.M.: Positive mood broadens visual attention to positive stimuli. Motiv. Emot. 30(1), 87–99 (2006)

    Google Scholar 

  48. Watkins, E.R., Moberly, N.J.: Concreteness training reduces dysphoria: a pilot proof-of-principle study. Behav. Res. Ther. 47(1), 48–53 (2009)

    Google Scholar 

  49. Watson, D., Clark, L.A., Tellegen, A.: Development and validation of brief measures of positive and negative affect: the panas scales. J. Pers. Soc. Psychol. 54(6), 1063 (1988)

    Google Scholar 

  50. Xu, Y., Gao, S., Wu, J., Li, N., Yu, J.: Personalized saliency and its prediction. IEEE Trans. Pattern Anal. Mach. Intell. 41, 2975–2989 (2018)

    Google Scholar 

  51. Xu, Y., Li, N., Wu, J., Yu, J., Gao, S.: Beyond universal saliency: personalized saliency prediction with multi-task CNN. In: IJCAI, pp. 3887–3893 (2017)

  52. Zadra, J.R., Clore, G.L.: Emotion and perception: the role of affective information. Wiley Interdiscip. Rev. Cogn. Sci. 2(6), 676–685 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Informatics and Information Technologies, Slovak University of Technology in Bratislava, Bratislava, Slovak Republic

    Miroslav Laco, Patrik Polatsek, Šimon Dekrét & Wanda Benesova

  2. EmotionID, Bratislava, Slovak Republic

    Martina Baránková & Mária Gablíková

  3. Institute of Applied Psychology, Faculty of Social and Economic Sciences, Comenius University in Bratislava, Bratislava, Slovak Republic

    Martina Baránková, Bronislava Strnádelová, Jana Koróniová & Mária Gablíková

  4. Institute of Experimental Psychology, Centre of Social and Psychological Sciences, Slovak Academy of Sciences, Bratislava, Slovak Republic

    Jana Koróniová

Authors
  1. Miroslav Laco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrik Polatsek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Šimon Dekrét
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wanda Benesova
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Martina Baránková
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bronislava Strnádelová
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jana Koróniová
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Mária Gablíková
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wanda Benesova.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The authors would like to thank for financial contribution from UVP Project STU and from the Research and Development Operational Programme for the Project “University Science Park of STU Bratislava”, ITMS 26240220084, co-funded by the European Regional Development Fund and Grant VEGA 1/075/19 2019–2022.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Laco, M., Polatsek, P., Dekrét, Š. et al. Effects of individual’s emotions on saliency and visual search. Vis Comput 37, 1581–1592 (2021). https://doi.org/10.1007/s00371-020-01912-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01912-7

Keywords

Search

Navigation