Keywords

1 Introduction

People intrinsically interact with the world in a multimodal manner. In User Interface Design (UID), multimodal metaphors aim to influence and enhance natural human capabilities to communicate information using audio, visual, graphical 3-D, and other modalities. Changes in consumer behavior have forced businesses to rethink their marketing presentation strategies on the digital domain. The majority of the firms are now advertising using social media. Social Media marketing has caused a shift in consumer behavior. According to recent statistics [16], last year the annual average weekly spend online was £718.7 million.

It is generally accepted that social media marketing lacks “human warmth” as it is impersonal and automated compared to traditional marketing. In the online advertisement context, multimodality provides an additional bandwidth to communicate information. As the complexity of an online advertisement presentation increases, its clarity and effectiveness may decrease. In order to reduce complexity, the design and interactive features need to be balanced in a manner that the presented media impacts favourably on the purchase decision of a potential customer. However, the effect of multimodal metaphors to present and promote products on social platforms has still not been fully investigated. This paper aims to gather an overall viewpoint of user perception relating to the use or lack of multimodality.

2 Social Media Marketing and Multimodality

Consumer behavior can be defined as “a study of the process involved when individuals or groups select, purchase, use or dispose of products, services, ideas, or experiences to satisfy needs and desires” [9]. Bagozzi et al. [18] defines consumer behaviour as a combination of social and psychological processes, which people endure through the possession, use and disposal of products or services. It is the understanding of the interpersonal and social reasons underpinning consumer decision-making and the transactions associated with it. Consumer behaviour is complex as it varies depending on individual characteristics and idiosyncrasies. Purchase can be either rational or emotional [17]. Consumers employ multiple senses in order to directly and indirectly explore their environment to improve belief and identify new information [10]. External stimuli contribute during information search and evaluation. When a product is presented on social media platform, multimodal metaphors provide additional bandwidth to communicate information and enable the better user understanding of the information communicated. Despite the existing knowledge of customers for a product, they will still require to enhance their knowledge with some form of external search. Some of this supplementary data is obtained through advertisement, friends and increasingly social media platforms [17].

Social media can be defined as a medium through which people interact in order to create, share and exchange information and ideas in virtual networks and communities. Social media is differentiated from traditional media by attributes such as permanence, immediacy, frequency, cost and reach [21]. The importance of social media has been widely accepted today and it is pervasive in both personal and professional environments. Over the last decade, the Internet and social media has grown in usage and acceptability. This has changed purchasing patterns and consumer behavior. Human interaction has fundamentally and profoundly changed by the extensive use of online social networks [15]. This change has been accompanied by a strong growth of web-based platforms that facilitate this interaction. Some of the in vivo social interactions and relationships have been replaced with an equivalent counterpart in virtual worlds using online communities. These online communities enable people to share their knowledge, experiences and engage and promote dialogues within different cultures [4]. Studies suggest that consumers show greater value to peer reviews and judgments rather than firm promotions, demonstrating a shift in the focus of persuasive power [5].

The growth of internet and online communities have significantly transformed consumers, cultures and organizations with easy access to vast information, enriched communications and improved social networking [19]. Social media has not only changed marketing and advertisement of products [11], but also altered the focus from product information search to post purchase behavior [8]. Social media allows organizations to communicate with customers in a way that increases intimacy and enables the growth of a relationship [12]. This could result to a positive or negative perception of a brand as the reviews cannot be controlled. Even a small amount of negative information or user feedback can have a significant impact on consumer attitudes [13]. Online groups exercise an evident influence on the behavior and consumer buying intent and indirectly on the purchase decision. Social media platforms provide individual consumers with a public forum which not only gives them own voice but access to product information which then aids their purchase decisions [14].

Multimodality is an integrated approach of different types of media. According to Bunt et al. [6], human interaction with the world is inherently multimodal. People use their senses sequentially and simultaneously to perceive their environment. Multimodal metaphors describe the interactive systems that aim to enhance computer communication using audio, visual, graphical and other modalities. Fortin and Dholakia [7] suggest that as the presentation complexity of an advertisement increases, the effectiveness of positive results decreases. This complexity is often alleviated through balancing design and interactive features in such way that they contribute to a positive consumer purchase decision.

Rigas and Alseid [1] demonstrated the successful use of multimodal metaphors to communicate information. Further Rigas and Alty [20] and Almutairi and Rigas [2] prescribed the use of multimodal interfaces and avatars to communicate with the target audience. Additionally, Rigas and Almutairi [3] identified the positive impact of multimodal metaphors on ease of use of learning materials. Interfaces that communicate information to users using several communication channels often provide a more engaging interaction [2]. The use of interactive multimodal features aids user understanding to complete online purchases. However existing literature lacks the understanding of the impact of specific metaphors (text, video, audio, graphical illustrations) used in online advertisement towards consumer purchase decision.

3 Methodology

A positivist approach was taken in this study. Structured questionnaire was designed to investigate the user comprehension on the use or lack of multimodality and explore whether its presence help users develop an understanding of a product advertisement. The questionnaire had questions of multiple format, including multiple choice, dichotomous questions, self-assessment Likert-scale questions and open ended question. This enabled the gathering of an overall user viewpoint relating to multimodal metaphors and their perceived impact on purchasing decisions. The opportunistic sample consisted of 58 participants, 29 male and 29 female aged between 18 and 35 years old. The age profile of respondents consisted of 18–21 years old (8.6 %), 21–25 (34.5 %), 26–30 (37.9 %) and 31–35 (19 %).

4 Data Analysis and Discussion

4.1 Sample Profile

A total number of 58 respondents took part in the survey, 29 (50 %) male and 29 (50 %) female and 58 valid responses were received as shown in Fig. 1. All the respondents fall in the age group between 18–35, with the largest group falling in the age group of 26–30, accounting for 37.9 %, out of which 20.70 % account for females and 17.20 % males.

Fig. 1.
figure 1

Sample profile against gender

Full size image

The second largest group falls in the age group of 21–25, making up 34.5 %, out of which 17.20 % account for females and 17.20 % for males. However, the smallest group of respondents fall in the age group of 18–20, accounting for only 8.6 %, of which 3.40 % are female and 5.20 % male. Out of the 29 female respondents, the educational profile consists of 14 (24.10 %), with Degree or Diploma, and 9 (15.5 %) with a Postgraduate degree. Whereas, out of the 29 male respondents, 10 (17.2 %) hold a degree and 11 (19 %) have acquired postgraduate degree. Out of the 58 respondents who took part in the survey 63.8 % prefer to shop in-store, out of which 34.50 % account for male and 29.30 % account for female. On contrary 32.8 % prefer to shop online, of which 20.70 % are female and 12.10 % male. This question had an invalid response of 3.4 %. Figure 2 highlights Internet proficiency, 25 respondents (43.1 % of which 22.40 % were female and 20.70 % male) indicated that they use internet most on their smart phone. The second largest group of 22 (37.9 % of which 15.50 % were female and 22.40 % male) indicated that they prefer using internet on their Laptop. This data suggests that females prefer to access the Internet on their smart phone and males on their laptop. However, from the results, smartphone seems to be the device on which majority of the respondents prefer using internet on as compared to laptop, tablet and desktop. This can be largely due to the convenience factor; as smart phones allow one to perform most tasks that one could do on a desktop or laptop.

A cross tabulation between gender and reasons for using internet most indicated that the largest group of females accounting for 13.8 % used internet most for work purposes. The second largest group (17.2 %) used internet for browsing, followed by 8.6 % who use internet for shopping. Whereas, 24.1 % of male respondents indicated that they use internet most for browsing, followed by 10.30 % who use for work purposes. However, the overall results indicate browsing to be the reason for which majority of total respondents (41.4 %) use internet for, followed by work (24.1 %), personal reasons (19.0 %) and shopping (10.3 %).

It is interesting to notice that the smallest group, consisting of 1.7 % male use internet for shopping purposes, which clearly indicates towards the difference in male and female shopping behavior.

Fig. 2.
figure 2

Respondent internet proficiency against gender

Full size image

The survey aimed to gather an understanding of the frequency of online shopping. Results indicated that a large number of respondents, (31.0 %) shop online on monthly basis, followed by 25.9 % who shop once in every 3 months. Only 13.8 % respondents shop on weekly basis. However, results indicate equal number of male and female (50 %) who shop online on monthly basis. These statistics indicate towards ever growing trend on shopping online on significantly regular basis, equally by both genders.

Figure 3 demonstrates user perception of different marketing modes. 41.40 % of the respondents strongly agreed that social media marketing is more ‘influential’ as compared to conventional marketing. 22.40 % agree, whereas only a small percentage of respondents (8.60 %) disagree that advertisements on social media are more influential than conventional marketing. On contrary, the largest group accounting for 34.50 % thought that advertisements through conventional marketing are more influential. However, 6.90 % disagree and 12.10 % strongly disagree to this claim. 69 % perceive social media marketing to be more information rich as compared to conventional marketing. Out of which 32.80 % strongly agreed and 36.20 % agreed. Only 17.2 % respondents did not consider social media to be more information rich. On the other hand, 34.50 % of respondents agreed and 13.80 % strongly agreed that conventional marketing is more information rich. These results highlight the importance of social media as a medium to transmit information. The ability of social media to allow organizations and individuals to upload and share information. The main aim of marketing along with product/service promotion and brand creation is to be able to aid user understanding of the product or service advertised. Hence it is vital to ensure that the users do not find the advertisement context and the information marketed too complex to digest. Results from the survey illustrated that a large group of respondents, accounting for 43.1 % found advertisements via social media marketing to be a lot more complex as compared to conventional marketing. However, 20.7 % respondents disagree that social media marketing is more complex. This indicates that out of the respondents who took part in the survey, most of them find advertisements on social to be more complex as compared to conventional marketing.

Fig. 3.
figure 3

User perception of different advertisement contexts

Full size image
Fig. 4.
figure 4

Respondent viewpoint of different advertisement modes

Full size image

4.2 User Perception of Advertisement Contexts and Modes

Figure 4 indicates the user perception of most convincing advertisement modes. 37.90 % strongly agreed and 31 % agreed that advertisements on social media are most convincing. The second most convincing advertisement mode form the results is magazine (32.80 % agreed), followed by TV (31 % respondents agreed). The least convincing mode was radio (12.10 % agreed and 24.10 % disagreed). In conclusion, the social media was found to be the most convincing mode of advertisement (Fig. 5).

Fig. 5.
figure 5

User view point of social media marketing

Full size image

4.3 User Understanding of Social Media Marketing

In order to develop an understanding of respondents view point of social media marketing, the social media marketing was tested against 5 variables including Open systems, Two-way communication, Authentic content, Influencing and Real time creation. The aim was to identify which variables are accepted amongst the most respondents as significant traits of social media marketing. 56.9 % (39.70 % agreed and 17.20 strongly agreed) believed social media marketing to be compromised of an open system. 55.1 % perceived social media marketing offers authentic content. 51 % claim it to be influencing and 50 % believed that offers a two-way communication. The smallest group which accounts for 44.8 % of the respondents found social media marketing offering real time creation. Open system and authentic content were considered as two major traits.

4.4 Impact of Multimodal Metaphors to Aid User Understanding

Figure 6 demonstrates the impact of different metaphors to aid user understanding. Multimodal metaphors are present on the digital domain to enhance user understanding of the product/service advertised. In order to identify the impact of individual multimodal metaphors to aid user understanding a series of Likert scale questions were posed to the respondents. Results illustrated that the largest group of respondents accounting for 81.1 %, where 28.30 agree and 52.80 strongly agree, find Video to be the most significant metaphor in developing a clear understanding of the product advertised. The next metaphor which aids user understanding most after video is Illustrations for which 38.30 % agree and 34.70 % strongly agree, making a total group of 73.5 %. 60 % of the respondents find Text to aid their product understanding and only 43.5 % find Audio to help them develop a clear understanding of the product advertised online.

Fig. 6.
figure 6

Impact of multimodal metaphors to aid user understanding

Full size image

The graph clearly illustrates that all 4 metaphors are significantly important to aid user understanding of the product advertised online. However, Video (which compromises of all remaining metaphors) and Illustrations appear to be the most critical metaphors in enhancing user understanding (Fig. 7).

Fig. 7.
figure 7

Impact of multimodal interactivity on user purchase decision

Full size image

4.5 Impact of Multimodal Interactivity on Purchase Decision

It is important to determine whether multimodal metaphors only act as informant tools to aid user understanding or if they have any impact on the consumer purchase decision. This has been addressed in the survey through dichotomous questions. 50 % of the respondents stated that absence of sales assistant (human) does not affect their purchase decision, whereas 48.30 % stated that it would affect their decision. However, a large percentage of females accounting for 29.30 % agree that absence of sales assistant would affect their purchase decision, whereas a large proportion of the male respondents (31 %) stated that their purchase decision would not be affected by the human absence. This clearly shows that although majority users feel that the absence of human would not have an impact on their purchase decision there is clearly a difference of shopping behavior and preferences across genders.

The results have also revealed that a large proportion of respondents, both male and female, accounting for 91.4 % agree that interactive features increase their product knowledge. Only a mere 8.6 % disagree. This highlights the importance of interactive features to increase product knowledge. The respondents were further questioned to identify whether the absence of interactive features affects their purchase decision, to which 74.20 % answered yes. A small group of 24.1 % claim that interactive features have no influence on their purchase decision. 1.70 % were unsure. These results demonstrate the importance of interactive features in not only increasing product knowledge and aiding user understanding but also affecting the purchase decision. Further, a large number of respondents (39.6 %) stated that the product presentation affects their purchase decision. Only 1.70 % are barely influence by the presentation of the product.

4.6 Interactive Features and User Influence

Figure 8 shows 53.10 % strongly agreed and 26.50 % agreed that advertisements with a combination of video and text are most likely to positively affect purchase decision. For illustrations and text, 19.10 % strongly agreed and 53.20 % agreed. The advertisements via audio perceived to have a limited impact.

Fig. 8.
figure 8

Impact of interactive features on consumer purchase decision

Full size image

Figure 9 indicates that 49.10 % agreed that advertisements on social media are more efficient. 60.40 % wanted a more secure payment online procedure. 57.50 % strongly agreed that online shopping should have more clear refund and exchange policies. 29.30 % desired high human presence and 27.20 % strongly agreed that high human presence would improve their online purchase experience. The interactive multimodal features present on social media and integrated in online advertisements largely contributed to aid product understanding, influence and positively affecting consumer purchase decision.

Fig. 9.
figure 9

Online buying effectiveness, efficiency and user satisfaction

Full size image

5 Conclusion

Smartphone was the most common device on which majority of the respondents accessed Internet. The results indicate that the highest online shopping frequency of the respondents is on monthly basis, indicating equal number of male and female (50 %). A large number of respondents believed that social media marketing is more ‘influential’ and ‘information rich’ compared to conventional marketing. However most of the respondents find advertisements on social media to be more complex, but still find social media to be the most convincing mode of advertisement. Open system and authentic content are the two major traits of social media. The research also highlights that text, illustration, audio and video are perceived as significantly important to aid user understanding of the product advertised online. However, video and illustrations appear to be the most critical metaphors in enhancing the user understanding. Interactive features are perceived to increase product knowledge, aid user understanding and impact upon purchase decisions. The majority of users felt that the absence of human presence would not have an impact on their purchase decision. However, females felt that the absence of human presence might impact their purchase decision. It appears that there is a difference of shopping behavior and preferences across genders. The study also identified that combining metaphors in online advertisement is most likely to have a positive effect on the purchase decision (video with text and illustrations with text). Audio used on its own in advertisement context does not seem to have a significant impact to positively affect the purchase decision. Most of the respondents agreed that advertisements on social media are more efficient and effective, hence leading to a positive purchase decision. However, advertisements on social media are perceived as more complex. In conclusion, multimodal metaphors based on social media advertisement act as an informant tool by helping consumers to gain a better understanding of the product advertised and positively influence the final purchase decision.