skip to main content
research-article

Beyond Interruptibility: Predicting Opportune Moments to Engage Mobile Phone Users

Published: 11 September 2017 Publication History

Abstract

Many of today's mobile products and services engage their users proactively via push notifications. However, such notifications are not always delivered at the right moment, therefore not meeting products' and users' expectations. To address this challenge, we aim at developing an intelligent mobile system that automatically infers moments in which users are open to engage with suggested content. To inform the development of such a system, we carried out a field study with 337 mobile phone users. For 4 weeks, participants ran a study application on their primary phones. They were tasked to frequently report their current mood via a notification-administered experience-sampling questionnaire. In this study, however, we analyze whether they voluntarily engaged with content that we offered at the bottom of that questionnaire. In addition, the study app logged a wide range of data related to their phone use. Based on 120 Million phone-use events and 78,930 questionnaire notifications, we build a machine-learning model that before delivering a notification predicts whether a participant will click on the notification and subsequently engage with the offered content. When compared to a naïve baseline, which emulates current non-intelligent engagement strategies, our model achieves 66.6% higher success rate in its predictions. If the model also considers the user's past behavior, predictions improve 5-fold over the baseline. Based on these findings, we discuss the implications for building an intelligent service that identifies opportune moments for proactive user engagement, while, at the same time, reduces the number of undesirable interruptions.

References

[1]
Saeed Abdullah, Elizabeth L. Murnane, Mark Matthews, Matthew Kay, Julie A. Kientz, Geri Gay, and Tanzeem Choudhury. 2016. Cognitive Rhythms: Unobtrusive and Continuous Sensing of Alertness Using a Mobile Phone. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16). ACM, New York, NY, USA, 178--189.
[2]
Piotr D. Adamczyk and Brian P. Bailey. 2004. If Not Now, when?: The Effects of Interruption at Different Moments Within Task Execution. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’04). ACM, New York, NY, USA, 271--278.
[3]
Ionut Andone, Konrad Blaszkiewicz, Mark Eibes, Boris Trendafilov, Christian Montag, and Alexander Markowetz. 2016. How Age and Gender Affect Smartphone Usage. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp ’16). ACM, New York, NY, USA, 9--12.
[4]
Daniel Avrahami and Scott E. Hudson. 2006. Responsiveness in Instant Messaging: Predictive Models Supporting Inter-personal Communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’06). ACM, New York, NY, USA, 731--740.
[5]
L. Breiman. 2001. Random forests. Machine Learning 45, 1 (2001), 5--32.
[6]
L. Breiman and J. Friedman. 1984. Classification and regression trees. Chapman and Hall/CRC, Monterey, USA.
[7]
Yung-Ju Chang and John C. Tang. 2015. Investigating Mobile Users’ Ringer Mode Usage and Attentiveness and Responsiveness to Communication. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’15). ACM, New York, NY, USA, 6--15.
[8]
Peter McFaul Chapman. 1997. Models of engagement: Intrinsically motivated interaction with multimedia learning software. Ph.D. Dissertation. University of Waterloo.
[9]
T. Chen and C. Guestrin. 2016. XGBoost: a scalable tree boosting system. In Proc. of the ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD). 785--794.
[10]
Minsoo Choy, Daehoon Kim, Jae-Gil Lee, Heeyoung Kim, and Hiroshi Motoda. 2016. Looking Back on the Current Day: Interruptibility Prediction Using Daily Behavioral Features. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16). ACM, New York, NY, USA, 1004--1015.
[11]
Anind K Dey, Katarzyna Wac, Denzil Ferreira, Kevin Tassini, Jin-Hyuk Hong, and Julian Ramos. 2011. Getting closer: an empirical investigation of the proximity of user to their smart phones. In Proc. UbiComp ’11. ACM. http://www.mediateam.oulu.fi/publications/pdf/1432.pdf
[12]
Richard Farmer and Norman D. Sundberg. 1986. Boredom Proneness -- The development and correlates of a new scale. Journal of Personality Assessment 50 (1986), 4--17. Issue 1.
[13]
Joel E. Fischer, Chris Greenhalgh, and Steve Benford. 2011. Investigating Episodes of Mobile Phone Activity As Indicators of Opportune Moments to Deliver Notifications. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI ’11). ACM, New York, NY, USA, 181--190.
[14]
Joel E. Fischer, Nick Yee, Victoria Bellotti, Nathan Good, Steve Benford, and Chris Greenhalgh. 2010. Effects of Content and Time of Delivery on Receptivity to Mobile Interruptions. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI ’10). ACM, New York, NY, USA, 103--112.
[15]
James Fogarty, Scott E. Hudson, Christopher G. Atkeson, Daniel Avrahami, Jodi Forlizzi, Sara Kiesler, Johnny C. Lee, and Jie Yang. 2005. Predicting Human Interruptibility with Sensors. ACM Trans. Comput.-Hum. Interact. 12, 1 (March 2005), 119--146.
[16]
T. Hastie, R. Tibshirani, and J. Friedman. 2009. The elements of statistical learning (2nd ed.). Springer, Berlin, Germany.
[17]
Alexis Hiniker, Shwetak N. Patel, Tadayoshi Kohno, and Julie A. Kientz. 2016. Why Would You Do That? Predicting the Uses and Gratifications Behind Smartphone-usage Behaviors. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16). ACM, New York, NY, USA, 634--645.
[18]
Joyce Ho and Stephen S. Intille. 2005. Using Context-aware Computing to Reduce the Perceived Burden of Interruptions from Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’05). ACM, New York, NY, USA, 909--918.
[19]
Eric Horvitz, Carl Kadie, Tim Paek, and David Hovel. 2003. Models of Attention in Computing and Communication: From Principles to Applications. Commun. ACM 46, 3 (March 2003), 52--59.
[20]
Eric Horvitz, Paul Koch, Raman Sarin, Johnson Apacible, and Muru Subramani1. 2005. Bayesphone: Precomputation of Context-Sensitive Policies for Inquiry and Action in Mobile Devices. In Proc UM ’05.
[21]
Shamsi T. Iqbal and Brian P. Bailey. 2008. Effects of Intelligent Notification Management on Users and Their Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’08). ACM, New York, NY, USA, 93--102.
[22]
Shamsi T. Iqbal and Brian P. Bailey. 2010. Oasis: A Framework for Linking Notification Delivery to the Perceptual Structure of Goal-directed Tasks. ACM Trans. Comput.-Hum. Interact. 17, 4, Article 15 (Dec. 2010), 28 pages.
[23]
Yasumasa Kobayashi, Takahiro Tanaka, Kazuaki Aoki, and Kinya Fujita. 2015. Automatic Delivery Timing Control of Incoming Email Based on User Interruptibility. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’15). ACM, New York, NY, USA, 1779--1784.
[24]
Kostadin Kushlev, Bruno Cardoso, and Martin Pielot. 2017. Too Tense for Candy Crush: Affect Influences User Engagement With Proactively Suggested Content. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ‘17). ACM, New York, NY, USA.
[25]
Kostadin Kushlev, Jason Proulx, and Elizabeth W. Dunn. 2016. “Silence Your Phones”: Smartphone Notifications Increase Inattention and Hyperactivity Symptoms. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16). ACM, New York, NY, USA, 1011--1020.
[26]
R. Larson and M. Csikszentmihalyi. 1983. The experience sampling method. New Directions for Methodology of Social and Behavioral Science 15 (1983), 41?--56.
[27]
Hugo Lopez-Tovar, Andreas Charalambous, and John Dowell. 2015. Managing Smartphone Interruptions Through Adaptive Modes and Modulation of Notifications. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI ’15). ACM, New York, NY, USA, 296--299.
[28]
Gloria Mark, Daniela Gudith, and Ulrich Klocke. 2008. The Cost of Interrupted Work: More Speed and Stress. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’08). ACM, New York, NY, USA, 107--110.
[29]
Gloria Mark, Shamsi Iqbal, Mary Czerwinski, and Paul Johns. 2015. Focused, Aroused, but So Distractible: Temporal Perspectives on Multitasking and Communications. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work 8 Social Computing (CSCW ’15). ACM, New York, NY, USA, 903--916.
[30]
Gloria Mark, Shamsi T. Iqbal, Mary Czerwinski, and Paul Johns. 2014. Bored Mondays and Focused Afternoons: The Rhythm of Attention and Online Activity in the Workplace. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 3025--3034.
[31]
Afra Mashhadi, Akhil Mathur, and Fahim Kawsar. 2014. The Myth of Subtle Notifications. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp ’14 Adjunct). ACM, New York, NY, USA, 111--114.
[32]
Akhil Mathur, Nicholas D. Lane, and Fahim Kawsar. 2016. Engagement-aware Computing: Modelling User Engagement from Mobile Contexts. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’16). ACM, New York, NY, USA, 622--633.
[33]
Abhinav Mehrotra, Mirco Musolesi, Robert Hendley, and Veljko Pejovic. 2015. Designing Content-driven Intelligent Notification Mechanisms for Mobile Applications. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15). ACM, New York, NY, USA, 813--824.
[34]
Abhinav Mehrotra, Veljko Pejovic, Jo Vermeulen, Robert Hendley, and Mirco Musolesi. 2016. My Phone and Me: Understanding People’s Receptivity to Mobile Notifications. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16). ACM, New York, NY, USA, 1021--1032.
[35]
T. M. Mitchell. 1997. Machine Learning. McGraw-Hill, New York, USA.
[36]
Elizabeth L. Murnane, Saeed Abdullah, Mark Matthews, Matthew Kay, Julie A. Kientz, Tanzeem Choudhury, Geri Gay, and Dan Cosley. 2016. Mobile Manifestations of Alertness: Connecting Biological Rhythms with Patterns of Smartphone App Use. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’16). ACM, New York, NY, USA, 465--477.
[37]
Heather L O’Brien and Elaine G Toms. 2008. What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology 59, 6 (2008), 938--955.
[38]
Tadashi Okoshi, Hiroki Nozaki, Jin Nakazawa, Hideyuki Tokuda, Julian Ramos, and Anind K. Dey. 2016. Towards attention-aware adaptive notification on smart phones. Pervasive and Mobile Computing (2016).
[39]
Tadashi Okoshi, Julian Ramos, Hiroki Nozaki, Jin Nakazawa, Anind K. Dey, and Hideyuki Tokuda. 2015. Reducing Users’ Perceived Mental Effort Due to Interruptive Notifications in Multi-device Mobile Environments. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15). ACM, New York, NY, USA, 475--486.
[40]
Tadashi Okoshi, Kota Tsubouchi, Masaya Taji, Takanori Ichikawa, and Hideyuki Tokuda. 2017. Attention and Engagement-Awareness in the Wild: A Large-Scale Study with Adaptive Notifications. In IEEE International Conference on Pervasive Computing and Communications.
[41]
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: machine learning in python. Journal of Machine Learning Research 12 (2011), 2825--2830.
[42]
Veljko Pejovic and Mirco Musolesi. 2014. InterruptMe: Designing Intelligent Prompting Mechanisms for Pervasive Applications. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’14). ACM, New York, NY, USA, 897--908.
[43]
Veljko Pejovic, Mirco Musolesi, and Abhinav Mehrotra. 2015. Investigating The Role of Task Engagement in Mobile Interruptibility. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI ’15). ACM, New York, NY, USA, 1100--1105.
[44]
Martin Pielot. 2014. Large-scale Evaluation of Call-availability Prediction. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’14). ACM, New York, NY, USA, 933--937.
[45]
Martin Pielot, Karen Church, and Rodrigo de Oliveira. 2014. An In-situ Study of Mobile Phone Notifications. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI ’14). ACM, New York, NY, USA, 233--242.
[46]
Martin Pielot, Rodrigo de Oliveira, Haewoon Kwak, and Nuria Oliver. 2014. Didn’t You See My Message?: Predicting Attentiveness to Mobile Instant Messages. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 3319--3328.
[47]
Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver. 2015. When Attention is Not Scarce - Detecting Boredom from Mobile Phone Usage. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15). ACM, New York, NY, USA, 825--836.
[48]
Martin Pielot and Luz Rello. 2017. Productive, Anxious, Lonely ? 24 Hours Without Push Notifications. In MobileHCI ’17: Proceedings of the 18th International Conference on Human-computer Interaction with Mobile Devices and Services.
[49]
Benjamin Poppinga, Wilko Heuten, and Susanne Boll. 2014. Sensor-Based Identification of Opportune Moments for Triggering Notifications. IEEE Pervasive Computing 13, 1 (Jan. 2014), 22--29.
[50]
Stephanie Rosenthal, Anind K. Dey, and Manuela Veloso. 2011. Using decision-theoretic experience sampling to build personalized mobile phone interruption models. In Proc. Pervasive ’11. Springer-Verlag, 18. http://dl.acm.org/citation.cfm?id=2021975.2021991
[51]
Alireza Sahami Shirazi, Niels Henze, Tilman Dingler, Martin Pielot, Dominik Weber, and Albrecht Schmidt. 2014. Large-scale Assessment of Mobile Notifications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 3055--3064.
[52]
Hillol Sarker, Moushumi Sharmin, Amin Ahsan Ali, Md. Mahbubur Rahman, Rummana Bari, Syed Monowar Hossain, and Santosh Kumar. 2014. Assessing the Availability of Users to Engage in Just-in-time Intervention in the Natural Environment. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’14). ACM, New York, NY, USA, 909--920.
[53]
Florian Schulze and Georg Groh. 2016. Conversational Context Helps Improve Mobile Notification Management. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’16). ACM, New York, NY, USA, 518--528.
[54]
Liam D. Turner, Stuart M. Allen, and Roger M. Whitaker. 2015. Interruptibility Prediction for Ubiquitous Systems: Conventions and New Directions from a Growing Field. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp ’15). ACM, New York, NY, USA, 801--812.
[55]
Liam D. Turner, Stuart M. Allen, and Roger M. Whitaker. 2017. Reachable but not Receptive: Enhancing Smartphone Interruptibility Prediction by Modelling the Extent of User Engagement with Notifications. Pervasive and Mobile Computing (2017). accepted Jan 31, 2017.
[56]
Tilo Westermann, Sebastian Möller, and Ina Wechsung. 2015. Assessing the Relationship between Technical Affinity, Stress and Notifications on Smartphones. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI ’15). ACM, New York, NY, USA, 652--659.
[57]
Tilo Westermann, Ina Wechsung, and Sebastian Möller. 2016. Smartphone Notifications in Context: A Case Study on Receptivity by the Example of an Advertising Service. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’16). ACM, New York, NY, USA, 2355--2361.
[58]
Fred Zijlstra, Robert Roe, Anna B. Leonora, and Irene Krediet. 1999. Temporal factors in mental work: Effects of interrupted activities. Journal of Occupational and Organizational Psychology 72 (June 1999), 163--185. Issue 2.

Cited By

View all
  • (2024)Call to Action: Investigating Interaction Delay in Smartphone NotificationsSensors10.3390/s2408261224:8(2612)Online publication date: 19-Apr-2024
  • (2024)Pinning, Sorting, and Categorizing Notifications: A Mixed-methods Usage and Experience Study of Mobile Notification-management FeaturesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785798:3(1-27)Online publication date: 9-Sep-2024
  • (2024)Investigating User-perceived Impacts of Contextual Factors on Opportune MomentsProceedings of the ACM on Human-Computer Interaction10.1145/36765148:MHCI(1-28)Online publication date: 24-Sep-2024
  • Show More Cited By

Index Terms

  1. Beyond Interruptibility: Predicting Opportune Moments to Engage Mobile Phone Users

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 1, Issue 3
    September 2017
    2023 pages
    EISSN:2474-9567
    DOI:10.1145/3139486
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 11 September 2017
    Accepted: 01 June 2017
    Revised: 01 May 2017
    Received: 01 February 2017
    Published in IMWUT Volume 1, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Conversion
    2. Mobile Devices
    3. Proactive Recommendations
    4. Push Notifications

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)169
    • Downloads (Last 6 weeks)17
    Reflects downloads up to 03 Mar 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Call to Action: Investigating Interaction Delay in Smartphone NotificationsSensors10.3390/s2408261224:8(2612)Online publication date: 19-Apr-2024
    • (2024)Pinning, Sorting, and Categorizing Notifications: A Mixed-methods Usage and Experience Study of Mobile Notification-management FeaturesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785798:3(1-27)Online publication date: 9-Sep-2024
    • (2024)Investigating User-perceived Impacts of Contextual Factors on Opportune MomentsProceedings of the ACM on Human-Computer Interaction10.1145/36765148:MHCI(1-28)Online publication date: 24-Sep-2024
    • (2024)TIM: Temporal Interaction Model in Notification SystemProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3657614(1120-1124)Online publication date: 30-May-2024
    • (2024)Interrupting for Microlearning: Understanding Perceptions and Interruptibility of Proactive Conversational Microlearning ServicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642778(1-21)Online publication date: 11-May-2024
    • (2024)Investigating Contextual Notifications to Drive Self-Monitoring in mHealth Apps for Weight MaintenanceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641993(1-21)Online publication date: 11-May-2024
    • (2024)Greater mobile device‐prompted phone pickups are associated with daily parent stressActa Paediatrica10.1111/apa.17260113:8(1868-1875)Online publication date: 2-May-2024
    • (2024)My Model is Unfair, Do People Even Care? Visual Design Affects Trust and Perceived Bias in Machine LearningIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332719230:1(327-337)Online publication date: 1-Jan-2024
    • (2024)Design of Contextual Filtered Features for Better Smartphone-User Receptivity PredictionIEEE Internet of Things Journal10.1109/JIOT.2023.333171511:7(11707-11722)Online publication date: 1-Apr-2024
    • (2024)Smart Tiles: Tangible Interactions for Effective Notification Delivery2024 IEEE VII Congreso Internacional en Inteligencia Ambiental, Ingeniería de Software y Salud Electrónica y Móvil (AmITIC)10.1109/AmITIC62658.2024.10747631(1-8)Online publication date: 25-Sep-2024
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media