skip to main content
10.1145/3382507.3418811acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article

Mimicker-in-the-Browser: A Novel Interaction Using Mimicry to Augment the Browsing Experience

Published: 22 October 2020 Publication History

Abstract

Humans are known to have a better subconscious impression of other humans when their movements are imitated in social interactions. Despite this influential phenomenon, its application in human-computer interaction is currently limited to specific areas, such as an agent mimicking the head movements of a user in virtual reality, because capturing user movements conventionally requires external sensors. If we can implement the mimicry effect in a scalable platform without such sensors, a new approach for designing human-computer interaction will be introduced. Therefore, we have investigated whether users feel positively toward a mimicking agent that is delivered by a standalone web application using only a webcam. We also examined whether a web page that changes its background pattern based on head movements can foster a favorable impression. The positive effect confirmed in our experiments supports mimicry as a novel design practice to augment our daily browsing experiences.

Supplementary Material

MP4 File (3382507.3418811.mp4)
Humans are known to have a better subconscious impression of other humans when their movements are imitated in social interactions. Despite this influential phenomenon, its application in human-computer interaction is currently limited to specific areas, such as an agent mimicking the head movements of a user in virtual reality, because capturing user movements conventionally requires external sensors. If we can implement the mimicry effect in a scalable platform without such sensors, a new approach for designing human-computer interaction will be introduced. Therefore, we have investigated whether users feel positively toward a mimicking agent that is delivered by a standalone web application using only a webcam. We also examined whether a web page that changes its background pattern based on head movements can foster a favorable impression. The positive effect confirmed in our experiments supports mimicry as a novel design practice to augment our daily browsing experiences.
MP4 File (icmi1016vf.mp4)
Supplemental video

References

[1]
Riku Arakawa and Hiromu Yakura. 2019. REsCUE: A framework for REal-time feedback on behavioral CUEs using multimodal anomaly detection. In CHI. ACM, New York, NY, 572.
[2]
Jeremy N. Bailenson et al. 2008. Detecting digital chameleons. Comput. Hum. Behav. 24, 1 (2008), 66--87.
[3]
Jeremy N. Bailenson and Nick Yee. 2005. Digital Chameleons: Automatic Assimilation of Nonverbal Gestures in Immersive Virtual Environments. Psychol. Sci. 16, 10 (2005), 814--819.
[4]
François Bérard. 1999. The Perceptual Window: Head Motion as a New Input Stream. In Interact. IFIP, Laxenburg, Austria, 238--237.
[5]
Cigdem Beyan et al. 2018. Investigation of Small Group Social Interactions Using Deep Visual Activity-Based Nonverbal Features. In Multimedia. ACM, New York, NY, 311--319.
[6]
Milan Z. Bjelica et al. 2015. Minimizing impact of loading time and presentation to user experience in modern Over the Top television. In ICCE. IEEE, Washington, DC, 228--231.
[7]
Joseph N. Cappella and Sally Planalp. 1981. Talk and Silence Sequences in Informal Conversations III: Interspeaker Influence. Hum. Commun. Res. 7, 2 (1981), 117--132.
[8]
Caroline Catmur et al. 2008. Through the looking glass: counter-mirror activation following incompatible sensorimotor learning. Eur. J, Neurosci, 28, 6 (2008), 1208-- 1215.
[9]
Caroline Catmur and Cecilia Heyes. 2013. Is It What You Do, or When You Do It? The Roles of Contingency and Similarity in Pro-Social Effects of Imitation. Cogn. Sci. 37, 8 (2013), 1541--1552.
[10]
Youli Chang. 2015. Understanding Users? Touch Behavior on Large Mobile TouchScreens and Assisted Targeting by Tilting Gesture. In CHI. ACM, New York, NY, 1499--1508.
[11]
Tanya L. Chartrand and John A. Bargh. 1999. The chameleon effect: The perception--behavior link and social interaction. J. Pers. Soc. Psychol. 76, 6 (1999), 893--910.
[12]
Tanya L. Chartrand and Jessica L. Lakin. 2013. The Antecedents and Consequences of Human Behavioral Mimicry. Annu. Rev. Psychol. 64, 1 (2013), 285--308.
[13]
Jean Decety et al. 2002. A PET Exploration of the Neural Mechanisms Involved in Reciprocal Imitation. NeuroImage 15, 1 (2002), 265--272.
[14]
Christine Distefano et al. 2009. Understanding and using factor scores: Considerations for the applied researcher. Pract. Assess. Res. Evaluation 14, 20 (2009), 1--11.
[15]
Annerose Engel et al. 2008. How moving objects become animated: The human mirror neuron system assimilates non-biological movement patterns. Soc. Neurosci 3, 3--4 (2008), 368--387.
[16]
Dede Frederick et al. 2015. The Effects of Parallax Scrolling on User Experience in Web Design. J. Usability Stud. 10, 2 (2015), 87--95.
[17]
Luis A. Fuente et al. 2015. Influence of Upper Body Pose Mirroring in HumanRobot Interaction. In ICSR. Springer, Cham, Switzerland, 214--223.
[18]
Linda Di Geronimo et al. 2015. Tilt-and-Tap: Framework to Support Motion-Based Web Interaction Techniques. In ICWE. Springer, Cham, Switzerland, 565--582.
[19]
Lisa Graham. 2008. Gestalt Theory in Interactive Media Design. J. Humanit. Soc. Sci. 2, 1 (2008), 1--12.
[20]
Julie Grzes and Jean Decety. 2000. Functional anatomy of execution, mental simulation, observation, and verb generation of actions: A meta-analysis. Hum. Brain Mapp. 12, 1 (2000), 1--19.
[21]
Sophie Guionnet et al. 2011. Reciprocal Imitation: Toward a Neural Basis of Social Interaction. Cereb. Cortex 22, 4 (2011), 971--978.
[22]
Joanna Hale and Antonia F. de C. Hamilton. 2016. Cognitive mechanisms for responding to mimicry from others. Neurosci. Biobehav. Rev. 63 (2016), 106--123.
[23]
Joanna Hale and Antonia F. De C. Hamilton. 2016. Testing the relationship between mimicry, trust and rapport in virtual reality conversations. Sci. Rep. 6 (2016), 35295.
[24]
Cecilia Heyes. 2001. Causes and consequences of imitation. Trends Cogn. Sci. 5, 6 (2001), 253--261.
[25]
Sture Holm. 1979. A Simple Sequentially Rejective Multiple Test Procedure. Scand. J. Stat. 6, 2 (1979), 65--70.
[26]
Mohammed (Ehsan) Hoque et al. 2013. MACH: my automated conversation coach. In UbiComp. ACM, New York, NY, 697--706.
[27]
Christopher K. Hsee et al. 1990. The effect of power on susceptibility to emotional contagion. Cogn. Emot. 4, 4 (1990), 327--340.
[28]
Rex Hsieh et al. 2019. Evaluation of Avatar and Voice Transform in Programming E-Learning Lectures. In IVA. ACM, New York, NY, 197--199.
[29]
Marco Iacoboni et al. 1999. Cortical Mechanisms of Human Imitation. Sci. 286, 5449 (1999), 2526--2528.
[30]
Robert J. K. Jakob. 1991. The Use of Eye Movements in Human-Computer Interaction Techniques: What You Look At is What You Get. ACM Trans. Inf. Syst. 9, 2 (1991), 152--169.
[31]
Conor Kelton et al. 2017. Improving User Perceived Page Load Times Using Gaze. In NSDI. USENIX, Berkeley, CA, 545--559.
[32]
James M. Kilner et al. 2003. An Interference Effect of Observed Biological Movement on Action. Curr. Biol. 13, 6 (2003), 522--525.
[33]
Woojoo Kim et al. 2017. Effect of Loading Symbol of Online Video on Perception of Waiting Time. Int. J. Hum. Comput. Interact. 33, 12 (2017), 1001--1009.
[34]
Idil Kokal et al. 2011. Synchronized Drumming Enhances Activity in the Caudate and Facilitates Prosocial Commitment - If the Rhythm Comes Easily. PLoS ONE 6, 11 (2011), e27272.
[35]
Chandan Kumar et al. 2017. Chromium based Framework to Include Gaze Interaction in Web Browser. In WWW. ACM, New York, NY, 219--223.
[36]
Shusaku Kumazaki and Yugo Takeuchi. 2014. Estimation of Artifact's Emotion through Sharing the Background of Interaction. In HAI Symposium (in Japanese). HAI Symposium Steering Committee, Gifu, Japan, 23--30.
[37]
Jessica L. Lakin and others. 2003. The Chameleon Effect as Social Glue: Evidence for The Evolutionary Significance of Nonconscious Mimicry. J. Nonverbal Behav. 27, 3 (2003), 145--162.
[38]
Dominique Machuletz et al. 2018. Webcam Covering as Planned Behavior. In CHI. ACM, New York, NY, 180.
[39]
William W. Maddux et al. 2008. Chameleons bake bigger pies and take bigger pieces: Strategic behavioral mimicry facilitates negotiation outcomes. J. Exp. Soc. Psychol. 44, 2 (2008), 461--468.
[40]
Päivi Majaranta. 2011. Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies: Advances in Assistive Technologies. IGI Global, Hershey, PA.
[41]
Fridanna Maricchiolo et al. 2009. Effects of different types of hand gestures in persuasive speech on receivers? evaluations. Lang. Cogn. Process. 24, 2 (2009), 239--266.
[42]
Raphael Menges et al. 2017. GazeTheWeb: A Gaze-Controlled Web Browser. In Web4All. ACM, New York, NY, 25:1--25:2.
[43]
Roland Neumann and Fritz Strack. 2000. 'Mood contagion?: The automatic transfer of mood between persons. J. Pers. Soc. Psychol. 79, 2 (2000), 211--223.
[44]
Rebecca S. Portnoff et al. 2015. Somebody's Watching Me?: Assessing the Effectiveness of Webcam Indicator Lights. In CHI. ACM, New York, NY, 1649--1658.
[45]
Pei-Luen Patrick Rau et al. 2007. The Effect of Rich Web Portal Design and Floating Animations on Visual Search. Int. J. Hum. Comput. Interact. 22, 3 (2007), 195--216.
[46]
Giacomo Rizzolatti and Laila Craighero. 2004. The Mirror-Neuron System. Annu. Rev. Neurosci. 27, 1 (2004), 169--192.
[47]
Jeffrey Sanchez-Burks et al. 2009. Performance in intercultural interactions at work: Cross-cultural differences in response to behavioral mirroring. J. Appl. Psychol. 94, 1 (2009), 216--223.
[48]
Nava A. Shaked. 2017. Avatars and virtual agents -- relationship interfaces for the elderly. Healthc. Technol. Lett. 4, 3 (2017), 83--87.
[49]
Michihiro Shimada et al. 2008. Studying the Influence of the Chameleon Effect on Humans using an Android. In IROS. IEEE, Washington, DC, 767--772.
[50]
Daniel Smilkov et al. 2019. TensorFlow.js: Machine Learning for the Web and Beyond. arXiv 1901.05350 (2019), 1--12.
[51]
Julie S. Stevenson et al. 2000. Webpage Background and Viewer Attitudes. J. Advert. Res. 40, 1--2 (2000), 29--34.
[52]
Janis Stolzenwald and Paul Bremner. 2017. Gesture mimicry in social humanrobot interaction. In RO-MAN. IEEE, Washington, DC, 430--436.
[53]
Keita Suzuki et al. 2017. FaceShare: Mirroring with Pseudo-Smile Enriches Video Chat Communications. In CHI. ACM, New York, NY, 5313--5317.
[54]
Louis L. Thurstone. 1934. The vectors of mind. Psychol. Rev. 41, 1 (1934), 1--32.
[55]
Rick B. van Baaren et al. 2003. Mimicry for money: Behavioral consequences of imitation. J. Exp. Soc. Psychol. 39, 4 (2003), 393--398.
[56]
Rick B. van Baaren et al. 2009. Being Imitated: Consequences of Nonconsciously Showing Empathy. In The Social Neuroscience of Empathy. The MIT Press, Cambridge, MA, 31--42.
[57]
Frank M. F. Verberne et al. 2013. Trusting Digital Chameleons: The Effect of Mimicry by a Virtual Social Agent on User Trust. In Persuasive Technology. Springer, Cham, Switzerland, 234--245.
[58]
Ruoxu Wang and S. Shyam Sundar. 2018. How does Parallax Scrolling Influence User Experience? A Test of TIME (Theory of Interactive Media Effects). Int. J. Hum. Compt. Interact. 34, 6 (2018), 533--543.
[59]
Yin Wang and Antonia F. de C. Hamilton. 2012. Social top-down response modulation (STORM): a model of the control of mimicry in social interaction. Front. Hum. Neurosci. 6 (2012), 153.
[60]
Jeffrey Yim et al. 2008. Experience in the design and development of a game based on head-tracking input. In FuturePlay. ACM, New York, NY, 236--239.
[61]
Shumin Zhai et al. 1999. Manual and Gaze Input Cascaded (MAGIC) Pointing. In CHI. ACM, New York, NY, 246--253.
[62]
Hong-Bo Zhang et al. 2019. A Comprehensive Survey of Vision-Based Human Action Recognition Methods. Sensors 19, 5 (2019), 1005.
[63]
Leshao Zhang and Patrick G. T. Healey. 2018. Human, Chameleon or Nodding Dog?. In ICMI. ACM, New York, NY, 428--436.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction
October 2020
920 pages
ISBN:9781450375818
DOI:10.1145/3382507
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. browser-based interaction
  2. computer-mediated communication
  3. mimicry

Qualifiers

  • Research-article

Conference

ICMI '20
Sponsor:
ICMI '20: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
October 25 - 29, 2020
Virtual Event, Netherlands

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 182
    Total Downloads
  • Downloads (Last 12 months)13
  • Downloads (Last 6 weeks)0
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media