skip to main content
research-article

Convergence of interactive displays with smart mobile devices for effective advertising: A survey

Published: 14 February 2014 Publication History

Abstract

The trend of replacing public static signages with digital displays creates opportunities for interactive display systems, which can be used in collaborative workspaces, social gaming platforms and advertising. Based on marketing communication concepts and existing models for consumer behavior, three stages, namely attraction, interaction and conation, are defined in this article to analyze the effectiveness of interactive display advertising. By reviewing various methods and strategies employed by existing systems with attraction, interaction and conation stages, this article concludes that smart mobile devices should be integrated as a component to increase the effectiveness of interactive displays as advertising tools. Future research challenges related to this topic are also discussed.

References

[1]
S. Agamanolis. 2003. Designing displays for human connectedness. In Public and Situated Displays: Social and Interactional Aspects of Shared Display Technologies, vol. 2, 309--334.
[2]
R. Ballagas, J. Borchers, M. Rohs, and J. G. Sheridan. 2006. The smart phone: A ubiquitous input device. IEEE Pervasive Comput. 5. 70--77.
[3]
R. Ballagas, M. Rohs, and J. G. Sheridan. 2005. Sweep and point and shoot: Phonecam-based interactions for large public displays. In CHI Extended Abstracts on Human Factors in Computing Systems. 1200--1203.
[4]
L. Barkhuus and A. Dey. 2003. Is context-aware computing taking control away from the user? Three levels of interactivity examined. In Proceedings of the 5th International Conference on Ubiquitous Computing. 149--156.
[5]
H. Brignull and Y. Rogers. 2003. Enticing people to interact with large public displays in public spaces. In Proceedings of the IFIP International Conference on Human-Computer Interaction. 17--24.
[6]
J. C. Cardoso and R. José. 2009. A framework for context-aware adaptation in public displays. In Proceedings of the Confederated International Workshops and Posters On the Move to Meaningful Internet Systems. 118--127.
[7]
F. Chehimi, N. Nijdam, D. Schmidt, E. Rukzio, and N. Magnenat-Thalmann. 2009. An interactive table supporting mobile phone interaction and 3d content. In Adjunct Proceedings of the 11th International Conference on Ubiquitous Computing. http://eprints.lancs.ac.uk/42501/.
[8]
K. Cheverst, A. Dix, D. Fitton, C. Kray, M. Rouncefield, C. Sas, G. Saslis-Lagoudakis, and J. G. Sheridan. 2005a. Exploring Bluetooth based mobile phone interaction with the hermes photo display. In Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices & Services. 47--54.
[9]
K. Cheverst, A. Dix, D. Fitton, C. Kray, M. Rouncefield, G. Saslis-Lagoudakis, and J. G. Sheridan. 2005b. Exploring mobile phone interaction with situated displays. In Proceedings of Pervasive Mobile Interaction Devices Workshop. 43--47.
[10]
T. Cuypers, Y. Francken, C. Vanaken, F. Van Reeth, and P. Bekaert. 2009. Smartphone localization on interactive surfaces using the built-in camera. In Proceedings of the IEEE International Workshop on Projector-Camera Systems. 61--68.
[11]
R. Dachselt and R. Buchholz. 2008. Throw and tilt-seamless interaction across devices using mobile phone gestures. In Proceedings of 2nd Workshop on Mobile and Embedded Interactive Systems. 272--278.
[12]
N. Davies, A. Friday, P. Newman, S. Rutlidge, and O. Storz. 2009. Using bluetooth device names to support interaction in smart environments. In Proceedings of the 7th International Conference on Mobile Systems, Applications, and Services. 151--164.
[13]
Dentsu. 2012. iButterfly. http://mobileartlab.jp/ibutterfly_ en.html.
[14]
G. Ding and B. Bhargava. 2004. Peer-to-peer file-sharing over mobile ad hoc networks. In Proceedings of the 2nd IEEE Annual Conference on Pervasive Computing and Communications Workshops. 104--108.
[15]
F. Echtler, S. Nestler, A. Dippon, and G. Klinker. 2009. Supporting casual interactions between board games on public tabletop displays and mobile devices. Personal and Ubiquitous Comput. 13. 609--617.
[16]
A. Erbad, M. Blackstock, A. Friday, R. Lea, and J. Al-Muhtadi. 2008. Magic broker: A middleware toolkit for interactive public displays. In Proceedings of the 6th Annual IEEE International Conference on Pervasive Computing and Communications. 509--514.
[17]
S. Feiner, N. Dedual, J. Jacobs, O. Oda, C. Robbins, M. Sukan, and S. White. 2010. Putting the surface in context. Tech. rep., Department of Computer Science, Columbia University. http://research.microsoft.com/en-us/projects/beneath-the-surface/bts_columbia_finalreport.pdf.
[18]
M. Finke, A. Tang, R. Leung, and M. Blackstock. 2008. Lessons learned: game design for large public displays. In Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts. 26--33.
[19]
R. Hardy, E. Rukzio, P. Holleis, and M. Wagner. 2010. Mobile interaction with static and dynamic nfc-based displays. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services. 123--132.
[20]
R. Hardy, E. Rukzio, M. Wagner, and M. Paolucci. 2009. Exploring expressive nfc-based mobile phone interaction with large dynamic displays. In Proceedings of the 1st International Workshop on Near Field Communication. 36--41.
[21]
M. K. Hogg and D. Yorke. 2004. Hierarchy of effects model. In The Blackwell Encyclopedia of Management, Wiley-Blackwell. http://www.blackwellreference.com/subscriber/tocnode.html?id=g9780631233176_ chunk_g978140510254411_ss1-2.
[22]
S. Hosio, H. Kukka, M. Jurmu, T. Ojala, and J. Riekki. 2010. Enhancing interactive public displays with social networking services. In Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia. 23:1--23:9.
[23]
E. Huang, A. Koster, and J. Borchers. 2008. Overcoming assumptions and uncovering practices: When does the public really look at public displays? In Pervasive Computing, Lecture Notes in Computer Science, vol. 5013, 228--243.
[24]
JCDecaux. 2012. U tie. http://www.u-tie.com.hk/front/en/home.
[25]
N. Katzakis, M. Hori, K. Kiyokawa, and H. Takemura. 2011. Smartphone game controller. In Proceedings of the 74th HIS SigVR Workshop. http://www.lab.ime.cmc.osaka-u.ac.jp/paper/datas/2011/06/Katzakis_ 0408/Katzakis_201106_paper.pdf.
[26]
N. Kaviani, M. Finke, S. Fels, R. Lea, and H. Wang. 2009. What goes where? Designing interactive large public display applications for mobile device interaction. In Proceedings of the 1st International Conference on Internet Multimedia Computing and Service. 129--138.
[27]
D. Kern, M. Harding, O. Storz, N. Davis, and A. Schmidt. 2008. Shaping how advertisers see me: User views on implicit and explicit profile capture. In CHI Extended Abstracts on Human Factors in Computing Systems. 3363--3368.
[28]
J. Leikas, H. Stromberg, V. Ikonen, R. Suomela, and J. Heinila. 2006. Multi-user mobile applications and a public display: Novel ways for social interaction. In Proceedings of the 4th Annual IEEE International Conference on Pervasive Computing and Communication. 66--70.
[29]
A. Lorenz and M. Jentsch. 2010. The ambient media player: a media application remotely operated by the use of mobile devices and gestures. In Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia. 15:1--15:10.
[30]
D. Majoe, S. Schubiger, A. Clay, and S. Arisona. 2007. Sqeak: A mobile multi platform phone and networks gesture sensor. In Proceedings of the 2nd International Conference on Pervasive Computing and Application. 699--704.
[31]
D. Michelis and J. Müller. 2011. The audience funnel: Observations of gesture based interaction with multiple large displays in a city center. Int. J. Human-Comput. Interact. 27. 562--579.
[32]
Microsoft. 2011. Microsoft Surface (PixelSense). http://en.wikipedia.org/wiki/Microsoft_PixelSense.
[33]
K. Mitchell and N. Race. 2006. Oi: Capturing user attention within pervasive display environments. In Proceedings of the Workshop on Pervasive Display Infrastructures, Interfaces and Applications. http://www.nrsp.lancs.ac.uk/mandmdesigns/oi.pdf.
[34]
J. Müller, F. Alt, D. Michelis, and A. Schmidt. 2010. Requirements and design space for interactive public displays. In Proceedings of the International Conference on Multimedia. 1285--1294.
[35]
J. Müller, J. Exeler, M. Buzeck, and A. Krüger. 2009. Reflectivesigns: Digital signs that adapt to audience attention. In Proceedings of the 7th International Conference on Pervasive Computing. 17--24.
[36]
J. Müller, M. Jentsch, C. Kray, and A. Krüger. 2008. Exploring factors that influence the combined use of mobile devices and public displays for pedestrian navigation. In Proceedings of the 5th Nordic Conference on Human-Computer Interaction. 308--317.
[37]
J. Müller, A. Schlottmann, and A. Krüger. 2007. Self-optimizing digital signage advertising. In Adjunct Proceedings of the 9th International Conference on Ubiquitous Computing (UbiComp). http://www.joergmueller.info/pdf/Ubicomp07MuellerSelfOptimizing.pdf.
[38]
M. Nancel, J. Wagner, E. Pietriga, O. Chapuis, and W. Mackay. 2011. Mid-air pan-and-zoom on wall-sized displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 177--186.
[39]
T. Payne, E. David, N. R. Jennings, and M. Sharifi. 2006. Auction mechanisms for efficient advertisement selection on public displays. In Proceedings of the 17th European Conference on Artificial Intelligence. 285--289.
[40]
N. Pears, D. G. Jackson, and P. Olivier. 2009. Smart phone interaction with registered displays. IEEE Pervasive Comput. 8, 14--21.
[41]
T. Pering, R. Ballagas, and R. Want. 2005. Spontaneous marriages of mobile devices and interactive spaces. Commun. ACM 48, 53--59.
[42]
M. L. Ray. 1973. Marketing communications and the hierarchy-of-effects. In New Models for Mass Communication Research, Sage Publications, Beverly Hills, CA, 147--176.
[43]
RPA. 2011. Mobile mouse for the iPhone, iPod, & iPad. http://www.mobilemouse.com/overview.html.
[44]
H. Ruan, Y. Qian, Y. Zhang, and M. Zhou. 2010. Touchinteract: An interaction technique with large displays using touchscreen-phone. In Proceedings of the Symposia and Workshops on Ubiquitous, Autonomic and Trusted Computing. 262--265.
[45]
J. Scheible and T. Ojala. 2005. Mobilenin combining a multi-track music video, personal mobile phones and a public display into multi-user interactive entertainment. In Proceedings of the 13th Annual ACM International Conference on Multimedia. 199--208.
[46]
J. Scheible, T. Ojala, and P. Coulton. 2008. Mobitoss: a novel gesture based interface for creating and sharing mobile multimedia art on large public displays. In Proceedings of the 16th ACM International Conference on Multimedia. 957--960.
[47]
D. Schmidt, F. Chehimi, E. Rukzio, and H. Gellersen. 2010. Phonetouch: a technique for direct phone interaction on surfaces. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology. 13--16.
[48]
K. Seewoonauth, E. Rukzio, R. Hardy, and P. Holleis. 2009. Touch & connect and touch & select: Interacting with a computer by touching it with a mobile phone. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 36:1--36:9.
[49]
A. Shirazi, T. Döring, P. Parvahan, B. Ahrens, and A. Schmidt. 2009a. Poker surface: combining a multi-touch table and mobile phones in interactive card games. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 73:1--73:2.
[50]
A. S. Shirazi, C. Winkler, and A. Schmidt. 2009b. Flashlight interaction: a study on mobile phone interaction techniques with large displays. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 93:1--93:2.
[51]
R. Suomela, E. Räsänen, A. Koivisto, and J. Mattila. 2004. Open-source game development with the multi-user publishing environment (mupe) application platform. In Proceedings of 3rd International Conference on Entertainment Computing. 308--320.
[52]
L. Terrenghi, A. Quigley, and A. Dix. 2009. A taxonomy for and analysis of multi-person-display ecosystems. Personal Ubiquitous Comput. 13, 583--598.
[53]
Tesco. 2011. The QR code Tesco store: From concept to reality. http://2d-code.co.uk/tesco-qr-code-store/.
[54]
V. H. Tuulos, J. Scheible, and H. Nyholm. 2007. Combining web, mobile phones and public displays in large-scale: Manhattan story mashup. In Proceedings of the 5th International Conference on Pervasive Computing. 37--54.
[55]
T. Vajk, P. Coulton, W. Bamford, and R. Edwards. 2008. Using a mobile phone as a “wii-like” controller for playing games on a large public display. Int. J. Computer Games Tech. 4:1--4:6.
[56]
A. D. Wilson and R. Sarin. 2007. Bluetable: Connecting wireless mobile devices on interactive surfaces using vision-based handshaking. In Proceedings of Graphics Interface. 119--125.

Cited By

View all
  • (2024)NearFetch: Enhancing Touch-Based Mobile Interaction on Public Displays with an Embedded Programmable NFC ArrayProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685749(234-243)Online publication date: 4-Nov-2024
  • (2021)Users’ Content Memorization in Multi-User Interactive Public DisplaysInternational Journal of Human–Computer Interaction10.1080/10447318.2021.194868638:5(447-455)Online publication date: 6-Jul-2021
  • (2019)A Design of Digital Signage with that the Character in Display Handing a Flyer Directly to Consumers映像中のキャラクターから手渡しでチラシを受け取るデジタルサイネージThe Journal of the Society for Art and Science10.3756/artsci.18.118:1(1-9)Online publication date: 15-Mar-2019
  • Show More Cited By

Index Terms

  1. Convergence of interactive displays with smart mobile devices for effective advertising: A survey

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 10, Issue 2
      February 2014
      142 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/2579228
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 14 February 2014
      Accepted: 01 August 2013
      Revised: 01 May 2013
      Received: 01 December 2012
      Published in TOMM Volume 10, Issue 2

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Interactive display
      2. advertising
      3. human factors
      4. marketing
      5. smart mobile device

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)50
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 17 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)NearFetch: Enhancing Touch-Based Mobile Interaction on Public Displays with an Embedded Programmable NFC ArrayProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685749(234-243)Online publication date: 4-Nov-2024
      • (2021)Users’ Content Memorization in Multi-User Interactive Public DisplaysInternational Journal of Human–Computer Interaction10.1080/10447318.2021.194868638:5(447-455)Online publication date: 6-Jul-2021
      • (2019)A Design of Digital Signage with that the Character in Display Handing a Flyer Directly to Consumers映像中のキャラクターから手渡しでチラシを受け取るデジタルサイネージThe Journal of the Society for Art and Science10.3756/artsci.18.118:1(1-9)Online publication date: 15-Mar-2019
      • (2019)Learning Sequence-Based Fingerprint for Magnetic Indoor Positioning SystemIEEE Access10.1109/ACCESS.2019.29525647(163231-163244)Online publication date: 2019
      • (2019)Understanding How Advertising Gamification Influences Consumers: The Effect of Image Experience and InteractivityAdvances in Human Factors in Communication of Design10.1007/978-3-030-20500-3_8(71-85)Online publication date: 8-Jun-2019
      • (2018)Sensor Ball Raffle – Gamification of Billboard Advertising: How to Engage the Audience?Entertainment Computing – ICEC 201810.1007/978-3-319-99426-0_14(164-174)Online publication date: 25-Aug-2018
      • (2017)When Smart Devices Interact With Pervasive ScreensACM Transactions on Multimedia Computing, Communications, and Applications10.1145/311593313:4(1-23)Online publication date: 12-Aug-2017
      • (2017)Designing Interactive Advertisements for Public DisplaysProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025531(1518-1529)Online publication date: 2-May-2017
      • (2017)Formal Model-Based Synthesis of Application-Specific Static RTOSACM Transactions on Embedded Computing Systems10.1145/301577716:4(1-25)Online publication date: 11-May-2017
      • (2017)Finding the Sweet Spot: Public Displays for Advertising on Campus2017 IEEE First Summer School on Smart Cities (S3C)10.1109/S3C.2017.8501382(108-113)Online publication date: Aug-2017
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media