Savvas Petridis
ABSTRACT
Visual metaphors are a creative technique used in print media to convey a message through images. This message is not said directly, but implied through symbols and how those symbols are juxtaposed in the image. The messages we see affect our thoughts and lives, and it is an open research challenge to get machines to automatically understand the implied messages in images. However, it is unclear how people process these images or to what degree they understand the meaning. We test several theories about how people interpret visual metaphors and find people can interpret the visual metaphor correctly without explanatory text with 41.3% accuracy. We provide evidence for four distinct types of errors people make in their interpretation, which speaks to the cognitive processes people use to infer the meaning. We also show that people's ability to interpret a visual message is not simply a function of image content but also of message familiarity. This implies that efforts to automatically understand visual images should take into account message familiarity.
- Chun-Tuan Chang and Ching-Ting Yen. 2013. Missing ingredients in metaphor advertising: The right formula of metaphor type, product type, and need for cognition. Journal of advertising 42, 1 (2013), 80--94.Google Scholar
Cross Ref
- Lydia B. Chilton, Savvas Petridis, and Maneesh Agrawala. 2019. VisiBlends: A flexible workflow for visual blends. In Proceedings of the 2019 ACM Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA.Google ScholarDigital Library
- Joao M Cunha, Joao Goncc alves, Pedro Martins, Penousal Machado, and Am'ilcar Cardoso. 2017. A pig, an angel and a cactus walk into a blender: A descriptive approach to visual blending. arXiv preprint arXiv:1706.09076 (2017).Google Scholar
- Joao M Cunha, Pedro Martins, and Penousal Machado. 2018. How shell and horn make a unicorn: Experimenting with visual blending in emoji. In Proceedings of the Ninth International Conference on Computational Creativity.Google Scholar
- Charles Forceville. 1994. Pictorial Metaphor in Advertisements. Metaphor and Symbolic Activity 9, 1 (1994), 1--29.Google ScholarCross Ref
- Charles Forceville. 2002. The identification of target and source in pictorial metaphors. Journal of Pragmatics 34, 1 (2002), 1--14.Google ScholarCross Ref
- Charles Forceville and others. 2017. Visual and multimodal metaphor in advertising: Cultural perspectives. Styles of Communication 9, 2 (2017), 26--41.Google Scholar
- Jos Hornikx and Rob le Pair. 2017. The influence of high-/low-context culture on perceived Ad complexity and liking. Journal of Global Marketing 30, 4 (2017), 228--237.Google ScholarCross Ref
- Zaeem Hussain, Mingda Zhang, Xiaozhong Zhang, Keren Ye, Christopher Thomas, Zuha Agha, Nathan Ong, and Adriana Kovashka. 2017. Automatic understanding of image and video advertisements. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1100--1110.Google ScholarCross Ref
- Se-Hoon Jeong. 2008. Visual metaphor in advertising: Is the persuasive effect attributable to visual argumentation or metaphorical rhetoric? Journal of marketing communications 14, 1 (2008), 59--73.Google ScholarCross Ref
- Allison J Lazard, Benita A Bamgbade, Jennah M Sontag, and Carolyn Brown. 2016. Using visual metaphors in health messages: a strategy to increase effectiveness for mental illness communication. Journal of health communication 21, 12 (2016), 1260--1268.Google ScholarCross Ref
- RG Le Pair and MJP Van Mulken. 2008. Perceived complexity and appreciation of visual metaphors by consumers with different cultural backgrounds. (2008).Google Scholar
- Edward F McQuarrie and David Glen Mick. 1999. Visual rhetoric in advertising: Text-interpretive, experimental, and reader-response analyses. Journal of consumer research 26, 1 (1999), 37--54.Google ScholarCross Ref
- Edward F McQuarrie and Barbara J Phillips. 2005. Indirect persuasion in advertising: How consumers process metaphors presented in pictures and words. Journal of advertising 34, 2 (2005), 7--20.Google ScholarCross Ref
- Marijn HC Meijers, Paola Remmelswaal, and Anke Wonneberger. 2018. Using Visual Impact Metaphors to Stimulate Environmentally Friendly Behavior: The Roles of Response Efficacy and Evaluative Persuasion Knowledge. Environmental Communication (2018), 1--14.Google Scholar
- Jean-Baptiste Michel, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K Gray, Joseph P Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, Jon Orwant, and others. 2011. Quantitative analysis of culture using millions of digitized books. science 331, 6014 (2011), 176--182.Google Scholar
- Barbara J Phillips and Edward F McQuarrie. 2004. Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing theory 4, 1--2 (2004), 113--136.Google Scholar
- Keith Rayner, Sarah J White, and SP Liversedge. 2006. Raeding wrods with jubmled lettres: There is a cost. (2006).Google Scholar
- Elisabeth El Refaie. 2003. Understanding visual metaphor: The example of newspaper cartoons. Visual communication 2, 1 (2003), 75--95.Google Scholar
- Renske van Enschot and Hans Hoeken. 2015. The occurrence and effects of verbal and visual anchoring of tropes on the perceived comprehensibility and liking of TV commercials. Journal of Advertising 44, 1 (2015), 25--36.Google ScholarCross Ref
- Margot Van Mulken, Rob Le Pair, and Charles Forceville. 2010. The impact of perceived complexity, deviation and comprehension on the appreciation of visual metaphor in advertising across three European countries. Journal of Pragmatics 42, 12 (2010), 3418--3430.Google ScholarCross Ref
- Margot Van Mulken, Andreu van Hooft, and Ulrike Nederstigt. 2014. Finding the tipping point: Visual metaphor and conceptual complexity in advertising. Journal of Advertising 43, 4 (2014), 333--343.Google ScholarCross Ref
- Stephanie K Van Stee. 2018. Meta-analysis of the persuasive effects of metaphorical vs. literal messages. Communication Studies 69, 5 (2018), 545--566.Google ScholarCross Ref
- Thomas Whiteside. 1971. Selling death: cigarette advertising and public health. Liveright.Google Scholar
- Keren Ye and Adriana Kovashka. 2018. Advise: Symbolism and external knowledge for decoding advertisements. In Proceedings of the European Conference on Computer Vision (ECCV). 837--855.Google ScholarDigital Library
Index Terms
- Human Errors in Interpreting Visual Metaphor
Recommendations
Conveying Semantics through Visual Metaphor
Special Issue on Linking Social Granularity and FunctionsIn the field of visual art, metaphor is a way to communicate meaning to the viewer. We present a computational system for communicating visual metaphor that can identify adjectives for describing an image based on a low-level visual feature ...
Comments