research-article

MyTraces: Investigating Correlation and Causation between Users’ Emotional States and Mobile Phone Interaction

Authors:

Abhinav Mehrotra,

Robert Hendley,

Mirco MusolesiAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 1, Issue 3

Article No.: 83, Pages 1 - 21

https://doi.org/10.1145/3130948

Published: 11 September 2017 Publication History

Abstract

Most of the existing work concerning the analysis of emotional states and mobile phone interaction has been based on correlation analysis. In this paper, for the first time, we carry out a causality study to investigate the causal links between users’ emotional states and their interaction with mobile phones, which could provide valuable information to practitioners and researchers. The analysis is based on a dataset collected in-the-wild. We recorded 5,118 mood reports from 28 users over a period of 20 days.

Our results show that users’ emotions have a causal impact on different aspects of mobile phone interaction. On the other hand, we can observe a causal impact of the use of specific applications, reflecting the external users’ context, such as socializing and traveling, on happiness and stress level. This study has profound implications for the design of interactive mobile systems since it identifies the dimensions that have causal effects on users’ interaction with mobile phones and vice versa. These findings might lead to the design of more effective computing systems and services that rely on the analysis of the emotional state of users, for example for marketing and digital health applications.

References

[1]

2017. Android’s Notification Listener Service. (2017). http://developer.android.com/reference/android/service/notification/NotificationListenerService.html.

[2]

2017. Google Now. (2017). http://www.google.com/landing/now/.

[3]

2017. Google Play Store. (2017). https://play.google.com/store/apps.

[4]

2017. Google’s Activity Recognition Application. http://developer.android.com/training/location/activity-recognition.htmll. (2017).

[5]

2017. HeadSpace. (2017). https://www.headspace.com.

[6]

2017. HelloMind. (2017). http://www.hellomind.com.

[7]

Jorge Alvarez-Lozano, Venet Osmani, Oscar Mayora, Mads Frost, Jakob Bardram, Maria Faurholt-Jepsen, and Lars Vedel Kessing. 2014. Tell me your apps and I will tell you your mood: correlation of apps usage with bipolar disorder state. In PETRA’14.

Digital Library

[8]

C Daniel Batson, Laura L Shaw, and Kathryn C Oleson. 1992. Differentiating affect, mood, and emotion: toward functionally based conceptual distinctions. Sage Publications.

[9]

Christopher Beedie, Peter Terry, and Andrew Lane. 2005. Distinctions between emotion and mood. Cognition 8 Emotion 19, 6 (2005), 847--878.

[10]

Luca Canzian and Mirco Musolesi. 2015. Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis. In UbiComp’15.

Digital Library

[11]

Chun-Chu Chen and James F Petrick. 2013. Health and wellness benefits of travel experiences a literature review. Journal of Travel Research 52, 6 (2013), 709--719.

[12]

Sunny Consolvo, David W McDonald, Tammy Toscos, Mike Y Chen, Jon Froehlich, Beverly Harrison, Predrag Klasnja, Anthony LaMarca, Louis LeGrand, Ryan Libby, and others. 2008. Activity Sensing in the Wild: a Field Trial of UbiFit Garden. In CHI’08.

Digital Library

[13]

Mihaly Csikszentmihalyi and Reed Larson. 1983. The experience sampling method. New Directions for Methodology of Social and Behavioral Science 15 (1983), 41--56.

[14]

Mihaly Csikszentmihalyi and Reed Larson. 2014. Validity and reliability of the experience-sampling method. In Flow and the Foundations of Positive Psychology. 35--54.

[15]

Martin Fishbein. 1995. Developing effective behavior change interventions: some lessons learned from behavioral research. NIDA Research Monograph 155 (1995), 246--261.

[16]

Ronald Aylmer Fisher and Frank Yates. 1938. Statistical tables for biological, agricultural and medical research. Longman.

[17]

Joseph P Forgas, Gordon H Bower, and Susan E Krantz. 1984. The influence of mood on perceptions of social interactions. Journal of Experimental Social Psychology 20, 6 (1984), 497--513.

[18]

Ke Huang, Chunhui Zhang, Xiaoxiao Ma, and Guanling Chen. 2012. Predicting mobile application usage using contextual information. In UbiComp’12.

Digital Library

[19]

Neal Lathia, Veljko Pejovic, Kiran K. Rachuri, Cecilia Mascolo, Mirco Musolesi, and Peter J. Rentfrow. 2013. Smartphones for Large-Scale Behaviour Change Intervention. Pervasive Computing 12, 12 (July 2013), 66--73.

Digital Library

[20]

Peter D Leathwood and Patricia Pollet. 1983. Diet-induced mood changes in normal populations. Journal of Psychiatric Research 17, 2 (1983), 147--154.

[21]

Hosub Lee, Young Sang Choi, Sunjae Lee, and I.P. Park. 2012. Towards unobtrusive emotion recognition for affective social communication. In CCNC’12.

[22]

Robert LiKamWa, Yunxin Liu, Nicholas D Lane, and Lin Zhong. 2013. Moodscope: Building a mood sensor from smartphone usage patterns. In MobiSys’13.

Digital Library

[23]

Bo Lu, Elaine Zanutto, Robert Hornik, and Paul R Rosenbaum. 2001. Matching with doses in an observational study of a media campaign against drug abuse. J. Amer. Statist. Assoc. 96, 456 (2001), 1245--1253.

[24]

Hong Lu, Gokul T. Chittaranjan Mashfiqui Rabbi, Denise Frauendorfer, Marianne Schmid Mast, Andrew T. Campbell, Daniel Gatica-Perez, and Tanzeem Choudhury. 2012. StressSense: Detecting Stress in Unconstrained Acoustic Environments using Smartphones. In UbiComp’12.

Digital Library

[25]

Hong Lu, Wei Pan, Nicholas D. Lane, Tanzeem Choudhury, and Andrew T. Campbell. 2009. SoundSense: Scalable Sound Sensing for People-centric Applications on Mobile Phones. In MobiSys’09.

Digital Library

[26]

Prasanta Chandra Mahalanobis. 1936. On the generalized distance in statistics. Proceedings of the National Institute of Sciences (Calcutta) 2 (1936), 49--55.

[27]

Abhinav Mehrotra, Robert Hendley, and Mirco Musolesi. 2016. PrefMiner: Mining User’s Preferences for Intelligent Mobile Notification Management. In Proceedings of UbiComp’16. Heidelberg, Germany.

Digital Library

[28]

Abhinav Mehrotra, Mirco Musolesi, Robert Hendley, and Veljko Pejovic. 2015. Designing Content-driven Intelligent Notification Mechanisms for Mobile Applications. In UbiComp’15.

Digital Library

[29]

Abhinav Mehrotra, Veljko Pejovic, Jo Vermeulen, Robert Hendley, and Mirco Musolesi. 2016. My Phone and Me: Understanding User’s Receptivity to Mobile Notifications. In CHI’16.

Digital Library

[30]

Abhinav Mehrotra, Jo Vermeulen, Veljko Pejovic, and Mirco Musolesi. 2015. Ask, But Don’t Interrupt: The Case for Interruptibility-Aware Mobile Experience Sampling. In UbiComp’15 Adjunct.

Digital Library

[31]

Kewei Ming and Paul R Rosenbaum. 2001. A note on optimal matching with variable controls using the assignment algorithm. Journal of Computational and Graphical Statistics 10, 3 (2001), 455--463.

[32]

Veljko Pejovic and Mirco Musolesi. 2014. InterruptMe: designing intelligent prompting mechanisms for pervasive applications. In UbiComp’14.

Digital Library

[33]

Richard E Petty, Joseph R Priester, and Duane T Wegener. 1994. Cognitive processes in attitude change. Handbook of Social Cognition 2 (1994), 69--142.

[34]

Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver. 2015. When attention is not scarce-detecting boredom from mobile phone usage. In UbiComp’15.

Digital Library

[35]

Karl H Pribram and Diane McGuinness. 1975. Arousal, activation, and effort in the control of attention. Psychological Review 82, 2 (1975), 116--149.

[36]

Mashfiqui Rabbi, Shahid Ali, Tanzeem Choudhury, and Ethan Berke. 2011. Passive and In-situ Assessment of Mental and Physical Well-being using Mobile Sensors. In UbiComp’11.

Digital Library

[37]

Kiran K. Rachuri, Mirco Musolesi, Cecilia Mascolo, Jason Rentfrow, Chris Longworth, and Andrius Aucinas. 2010. EmotionSense: A Mobile Phones based Adaptive Platform for Experimental Social Psychology Research. In UbiComp’10.

Digital Library

[38]

JA Ressel. 1980. A circumplex model of affect. Journal of Personality and Social Psychology 39 (1980), 1161--78.

[39]

Donald B Rubin. 1973. Matching to remove bias in observational studies. Biometrics 29, 1 (1973), 159--183.

[40]

Donald B Rubin. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 66, 5 (1974), 688--701.

[41]

Adam Sadilek and John Krumm. 2012. Far Out: Predicting Long-Term Human Mobility. In AAAI’12.

Digital Library

[42]

Antti Salovaara, Antti Lindqvist, Tero Hasu, and Jonna Häkkilä. 2011. The phone rings but the user doesn’t answer: unavailability in mobile communication. In MobileHCI’11.

Digital Library

[43]

Ulrich Schimmack and Reisenzein Rainer. 2002. Experiencing activation: energetic arousal and tense arousal are not mixtures of valence and activation. Emotion 2, 4 (2002), 412--417.

[44]

Martin EP Seligman. 2004. Can happiness be taught? Daedalus 133, 2 (2004), 80--87.

[45]

Hans Selye. 1956. The stress of life. McGraw-Hill.

[46]

Sandra Servia-Rodríguez, Kiran K Rachuri, Cecilia Mascolo, Peter J Rentfrow, Neal Lathia, and Gillian M Sandstrom. 2017. Mobile Sensing at the Service of Mental Well-being: a Large-scale Longitudinal Study. In WWW’17.

Digital Library

[47]

William R Shadish, Thomas D Cook, and Donald Thomas Campbell. 2002. Experimental and quasi-experimental designs for generalized causal inference. Houghton, Mifflin and Company.

[48]

Choonsung Shin, Jin-Hyuk Hong, and Anind K Dey. 2012. Understanding and prediction of mobile application usage for smart phones. In UbiComp’12.

Digital Library

[49]

Vijay Srinivasan, Saeed Moghaddam, Abhishek Mukherji, Kiran K Rachuri, Chenren Xu, and Emmanuel Munguia Tapia. 2014. MobileMiner: Mining your frequent patterns on your phone. In UbiComp’14.

Digital Library

[50]

Elizabeth A Stuart. 2010. Matching methods for causal inference: A review and a look forward. Statistical science: a review journal of the Institute of Mathematical Statistics 25, 1 (2010), 1--21.

[51]

Robert E Thayer. 1989. The biopsychology of mood and arousal. Oxford University Press.

[52]

Arvind Thiagarajan, Lenin Ravindranath, Katrina LaCurts, Samuel Madden, Hari Balakrishnan, Sivan Toledo, and Jakob Eriksson. 2009. VTrack: Accurate, Energy-Aware Road Traffic Delay Estimation Using Mobile Phones. In SenSys’09.

Digital Library

[53]

Dorthe Kirkegaard Thomsen, Mimi Yung Mehlsen, Søren Christensen, and Robert Zachariae. 2003. Rumination--relationship with negative mood and sleep quality. Personality and Individual Differences 34, 7 (2003), 1293--1301.

[54]

Fani Tsapeli and Mirco Musolesi. 2015. Investigating causality in human behavior from smartphone sensor data: a quasi-experimental approach. EPJ Data Science 4, 1 (2015), 1--15.

[55]

Fani Tsapeli, Mirco Musolesi, and Peter Tino. 2017. Model-free Causality Detection: An Application to Social Media and Financial Data. Physica A 183, 1 (2017), 139--155.

[56]

Tianyu Wang, Giuseppe Cardone, Antonio Corradi, Lorenzo Torresani, and Andrew T Campbell. 2012. WalkSafe: A Pedestrian Safety App for Mobile Phone Users Who Walk and Talk While Crossing Roads. In HotMobile’12.

Digital Library

[57]

Tingxin Yan, David Chu, Deepak Ganesan, Aman Kansal, and Jie Liu. 2012. Fast app launching for mobile devices using predictive user context. In MobiSys’12.

Digital Library

Cited By

Lin YSu LChen ULee YWang PChang Y(2024)Pinning, Sorting, and Categorizing Notifications: A Mixed-methods Usage and Experience Study of Mobile Notification-management FeaturesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785798:3(1-27)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678579
Lee YWu MChang CChang XChang Y(2024)Investigating User-perceived Impacts of Contextual Factors on Opportune MomentsProceedings of the ACM on Human-Computer Interaction10.1145/36765148:MHCI(1-28)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676514
Chang TSu LLin YTsai JTang ZChang Y(2024)"I Want Lower Tone for Work-Related Notifications": Exploring the Effectiveness of User-Assigned Notification Alerts in Improving User Speculation of and Attendance to Mobile NotificationsProceedings of the ACM on Human-Computer Interaction10.1145/36765128:MHCI(1-25)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676512
Show More Cited By

Index Terms

MyTraces: Investigating Correlation and Causation between Users’ Emotional States and Mobile Phone Interaction
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
      1. Causal reasoning and diagnostics
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
  2. Ubiquitous and mobile computing
    1. Empirical studies in ubiquitous and mobile computing

Recommendations

Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis
UbiComp '15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing

One of the most interesting applications of mobile sensing is monitoring of individual behavior, especially in the area of mental health care. Most existing systems require an interaction with the device, for example they may require the user to input ...
Participant selection for data collection through device-to-device communications in mobile sensing

The appearance of smart mobile devices with communication, computation and sensing capability and increasing popularity of various mobile applications have caused the explosion of mobile data recently. In the same time, mobile sensing has been emerging ...
Emotions on the Go: Mobile Emotion Assessment in Real-Time using Facial Expressions
AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

Exploiting emotions for user interface evaluation became an increasingly important research objective in Human-Computer Interaction. Emotions are usually assessed through surveys that do not allow information to be collected in real-time. In our work, we ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 1, Issue 3

September 2017

2023 pages

EISSN:2474-9567

DOI:10.1145/3139486

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 September 2017

Accepted: 01 July 2017

Revised: 01 May 2017

Received: 01 February 2017

Published in IMWUT Volume 1, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Engineering and Physical Sciences Research Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

62
Total Citations
View Citations
1,109
Total Downloads

Downloads (Last 12 months)82
Downloads (Last 6 weeks)7

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lin YSu LChen ULee YWang PChang Y(2024)Pinning, Sorting, and Categorizing Notifications: A Mixed-methods Usage and Experience Study of Mobile Notification-management FeaturesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785798:3(1-27)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678579
Lee YWu MChang CChang XChang Y(2024)Investigating User-perceived Impacts of Contextual Factors on Opportune MomentsProceedings of the ACM on Human-Computer Interaction10.1145/36765148:MHCI(1-28)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676514
Chang TSu LLin YTsai JTang ZChang Y(2024)"I Want Lower Tone for Work-Related Notifications": Exploring the Effectiveness of User-Assigned Notification Alerts in Improving User Speculation of and Attendance to Mobile NotificationsProceedings of the ACM on Human-Computer Interaction10.1145/36765128:MHCI(1-25)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676512
Lee HPark TLee UKostakos VKay JHoang T(2024)Smartphone-based Human Behavior Task Modeling for Explainable Mental Health Detection ModelCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3679003(894-896)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3679003
Gebhardt CBrombach ALuong THilliges OHolz C(2024)Detecting Users' Emotional States during Passive Social Media UseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596068:2(1-30)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659606
Dutt BMonteiro KMisra ATaneja YParnami A(2024)Kawach: Reducing Smartphone Distractions by Leveraging Tangible Skins to Disguise Smartphones as Everyday ObjectsCompanion Publication of the 2024 ACM Designing Interactive Systems Conference10.1145/3656156.3663696(204-208)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3656156.3663696
Rony RAhmed MSarcar SAhmed N(2024)Understanding Driving Stress in Urban Bangladesh: An Exploratory Study, Wearable Development and ExperimentACM Journal on Computing and Sustainable Societies10.1145/3648434Online publication date: 14-Feb-2024
https://dl.acm.org/doi/10.1145/3648434
Jung GPark SMa EKim HLee U(2024)Tutorial on Matching-based Causal Analysis of Human Behaviors Using Smartphone Sensor DataACM Computing Surveys10.1145/364835656:9(1-33)Online publication date: 24-Apr-2024
https://dl.acm.org/doi/10.1145/3648356
Nepal SLiu WPillai AWang WVojdanovski VHuckins JRogers CMeyer MCampbell A(2024)Capturing the College ExperienceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435018:1(1-37)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643501
Jung GPark SLee U(2024)DeepStress: Supporting Stressful Context Sensemaking in Personal Informatics Systems Using a Quasi-experimental ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642766(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642766
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents