Skip to main content
SpringerLink
Log in

A study of product experience obtained from multimodal interactive displays

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Experiencing a product by human-computer interactions can effectively convey valid information to users, and a surreal experience can be simulated by multi-sensory interactions. The effects of multimodal interactive displays on the cognition of products have been investigated in this study. The effects of multimodal human-computer interactions on multi-sensory modal applications have also been investigated by a questionnaire survey. And the effect of multimodal interactive displays on users’ cognition can be further realized. The results indicated that the exhibit scenario display can assist users in simulating the real feeling of usage and understanding the displayed products. For the feedback stereoscopic display, it is harder for users to perceive the authenticity of product usage. For the simulation of reality, external links supply more product information, and users’ cognition of products has been strengthened. The panoramic view is for the display only but fails to assist users in determining the quality of products.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Bruno F, Muzzupappa M (2010) Product interface design: a participatory approach based on virtual reality. Int J Human Comput Stud 68(5):254–269

    Article  Google Scholar 

  2. Chang D (2009) Thinking and comparison of interactive exhibits in museums. Technol Museum Rev 13(4):1–16

    Google Scholar 

  3. Charfi S, Dubois E, Bastide R (2007) Articulating interaction and task models for the design of advanced interactive systems. Proceedings of the 6th international conference on task models and diagrams for user interface design. Springer, Berlin, pp 70–83

    Book  Google Scholar 

  4. Chen F (2006) Multimodal interaction systems, Designing human interface in speech technology. Springer, Berlin, pp 212–223

    Google Scholar 

  5. Chen LF, Li MS (2006) The application of augmented reality commodity demonstration on designing the interface of a electronic commerce. Proceedings of the symposium of information management research and application conference. Jinwen University of Science and Technology, Department of Information Management, Taipei, pp 135–143

    Google Scholar 

  6. Eastern Integrated Consumer Profile Survey (2010) Taiwan’s top five lifestyle trends – The Internet demographic profile and Internet use behavior. Retrieved September 30, 2010, From Tainan, National Cheng Kung University Library, Retrieved for Eastern Integrated Consumer Profile Web site: http://www.isurvey.com.tw/3_product/1_eicp.aspx

  7. Fiore AM, Jin H-J, Kim J (2005) For fun and profit: hedonic value from image interactivity and responses toward an online store. Psychol Mark 22(8):669–694

    Article  Google Scholar 

  8. Gao Y, Wang M, Tao D, Ji R, Dai Q (2012) 3-D object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303

    Article  MathSciNet  Google Scholar 

  9. Gosling DC (1990) The design of educational exhibits (Shih-Huei Huang, trans.). Museol Q 4(2):49–53

    Google Scholar 

  10. Grundy SM, Pasternak R, Greenland P, Smith S Jr, Fuster V (1999) Assessment of cardiovascular risk by use of multiple-risk-factor assessment equations. J Am Coll Cardiol 34:1348–1359

    Article  Google Scholar 

  11. Hassenzahl M, Diefenbach S, Göritz A (2010) Needs, affect, and interactive products—facets of user experience. Interacting Comput 22(5):353–362

    Article  Google Scholar 

  12. Jhang HM (1994) The practice of display design. Suncolor, Taipei, in Chinese, semantic translation

    Google Scholar 

  13. Kaplan AM, Haenlein M (2009) The fairyland of second life: virtual social worlds and how to use them. Bus Horiz 52(6):563–572

    Article  Google Scholar 

  14. Katherine AA (2009) Multimedia learning: cognitive individual differences and display design techniques predict transfer learning with multimedia learning modules. Comput Educ 53(4):1339–1354

    Article  Google Scholar 

  15. Kim J, Fiore AM, Lee HH (2007) Influences of online store cognition, shopping enjoyment, and shopping involvement on consumer patronage behavior towards an online retailer. J Retail Consum Serv 14(2):95–107

    Article  Google Scholar 

  16. Lin MH, Huang CC (2001) The logic of the figurative expressions and cognition in design practices. J Des 7(2):1–22

    Google Scholar 

  17. Lucquiaud V (2005) Sémantique et Outil pour la Modélisation des Tâches Utilisateur: N-MDA (Unpublished doctoral dissertation). University of Poitiers, France

  18. Mayer RE (2002) Multimedia learning. Psychol Learn Motiv 41:85–139

    Article  Google Scholar 

  19. Messinger PR, Stroulia E, Lyons K, Bone M, Niu RH, Smirnov K, Perelgut S (2009) Virtual worlds- past, present, and future: new directions in social computing. Decis Support Syst 47(3):204–228

    Article  Google Scholar 

  20. O’Brien HL (2010) The influence of hedonic and utilitarian motivations on user engagement: the case of online shopping experiences. Interacting Comput 22(5):344–352

    Article  Google Scholar 

  21. Olson JC, Jacoby J (1972) Cue utilization in the quality cognition process. In: Venkatesan M (ed) Proceedings of the third annual conference of the association for consumer research. Association for Consumer Research, College Park, pp 167–179

    Google Scholar 

  22. Oviatt S (2003) Multimodal interfaces. In: Jacko JA, Sears A (eds) The human–computer interaction handbook: fundamentals, evolving technologies and emerging applications. Lawrence Erlbaum Associates, Mahwah, pp 286–304

    Google Scholar 

  23. Pegler M (2001) Visual merchandising and display. Fairchild, New York

    Google Scholar 

  24. Richardson PS, Dick AS, Jain AK (1994) Extrinsic and extrinsic cue effect on cognitions of store brand quality. J Mark Res 58(4):28–36

    Google Scholar 

  25. Sankar S, Lauren GB, Sucharita C (2002) Window displays and consumer shopping decisions. J Retail Consum Serv 9(5):277–290

    Article  Google Scholar 

  26. Shen ES, Liang CY (2000) Research of interactive exhibition design for the web-based virtual reality museum. J Educ Media Libr Sci 37(3):275–298

    Google Scholar 

  27. Srinath BR, Otman AB (2010) Concept-based evidential reasoning for multimodal fusion in human–computer interaction. Appl Soft Comput 10(2):567–577

    Article  Google Scholar 

  28. Then NK, DeLong MR (1999) Apparel shopping on the web. J Fam Consum Sci 91(3):65–68

    Google Scholar 

  29. Wright PC, McCarthy J, Meekison L (2003) Making sense of experience. In: Blythe M, Overbeeke C, Monk A, Wright P (eds) Funology: from usability to enjoyment. Kluwer, Dordrecht, pp 43–53

    Chapter  Google Scholar 

  30. Xu XC (2001) A study on product display design for website shopping – a case of mobile phone. (Unpublished master's thesis). National Yunlin University of Science and Technology, Yunlin

  31. Zhao S, Chen L, Yao H, Zhang Y, Sun X (2015) Strategy for dynamic 3D depth data matching towards robust action retrieval. Neurocomputing 151:533–543

    Article  Google Scholar 

  32. Zhao S, Yao H, Zhang Y, Wang Y, Liu S (2015) View-based 3d object retrieval via multi-modal graph learning. Signal Process 112:110–118

    Article  Google Scholar 

Download references

Acknowledgments

The expenditure of this study is supported by the National Science Council of Republic of China under the project number of NSC 99-2221-E-006-168.

Author information

Authors and Affiliations

  1. Department of Industrial Design, National Cheng Kung University, Tainan 701, Taiwan

    Shuo-Fang Liu, Jui-Hung Cheng & Wan-Jiun Chen

Authors
  1. Shuo-Fang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jui-Hung Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wan-Jiun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuo-Fang Liu.

Additional information

Liu holds a Ph.D, National Cheng Kung University.

Cheng holds a PhD Candidate, National Cheng Kung University.

Chen holds a MS, National Cheng Kung University.

Appendices

Appendix 1

There are 25 types of display methods obtained and they are three-dimensional billboard, display rack - floor mount, display rack - hanging, wall mount, display cabinet - open type, display cabinet - closed type, display props - manikin, display props - mobile catering, physical arrangement - scenario type, physical arrangement - random type, slides, 360° display, marquee, neon lamp effect, flickering sign, tooltip effect, function presentation, blossom effect, page flip, menu type, scenario simulation type, game, web-page manipulation, real-part simulation, and voice robot respectively. There are ten types of three-dimensional models for static displays, eight types of activated modes for dynamic displays, three types of operative modes for dynamic displays, and four interactive modes for dynamic displays.

Table 11

Table 11 The analysis of the current status of product display types in virtual malls
Full size table

Appendix 2

To investigate users’ requirements for displays, a questionnaire survey was designed (29 questions for display patterns and 18 questions for population statistics). The guidelines for designing these questions are as follows.

  1. 1.

    Space functions of virtual shopping centers: the external environment (activity piazza, pleasure ground, landscaping, etc.) and the internal environment (reception lobby, mall, pleasure ground, etc.).

  2. 2.

    Product display patterns of virtual shopping centers: product display patterns of virtual shopping centers: product display patterns, static product display, three-dimensional display, dynamic product display, preference degree, product activation display pattern, product operation display pattern, product interactive display patterns, etc.

  3. 3.

    Population statistics: gender, place of residence, age, marriage condition, family members, level of education, occupation, earnings, etc.

  4. 4.

    Status of community website usage: frequency of community website visits (hours, days, and weeks), mall games in 3D virtual world, virtual game types and frequency of visits (hour, days, and weeks), participating in virtual shopping centers in games, buying items or props in games, motivation of participating in virtual games, etc.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, SF., Cheng, JH. & Chen, WJ. A study of product experience obtained from multimodal interactive displays. Multimed Tools Appl 75, 6143–6172 (2016). https://doi.org/10.1007/s11042-015-2564-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2564-y

Keywords

Search

Navigation