Skip to main content
Springer Nature Link
Log in

A Longitudinal Analysis of Real-World Self-report Data

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2023 (INTERACT 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14144))

Included in the following conference series:

  • 1305 Accesses

  • The original version of the chapter has been revised. A footnote has been added indicating that the label ‘Anonymised Company’ was revised to “Levell” for the published version. A correction to this chapter can be found at https://doi.org/10.1007/978-3-031-42286-7_38

Abstract

While self-report studies are common in Human-Computer Interaction research, few evaluations have assessed their long term use. We present a longitudinal analysis of a web-based workplace application that collects well-being assessments and offers suggestions to improve individual, team, and organisational performance. Our dataset covers 219 users. We assess their first year of application use, focusing on their usage patterns, well-being evaluations, and behaviour towards notifications. Our results highlight that the drop-off in use was the steepest in the first week (-24.2%). However, substantial breaks in usage were common and did not necessarily result in dropout. We found that latency periods of eight days or more predicted a stronger intention to drop out than stay engaged and that reminder notifications did not result in more completed self-reports but significantly prolonged the usage period. Our work strengthens findings related to high drop out rates, but also provides counter-evidence by showing that despite individuals appearing to drop-off in short-term studies, individuals can and do return to self-report applications after extensive breaks. We contribute an analysis of usage behaviour drivers in the area of technology-enabled well-being measurement, responding to the call for longer-term research to extend the growing literature on self-report studies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Change history

  • 25 August 2023

    A correction has been published.

References

  1. Attig, C., Franke, T.: Abandonment of personal quantification: A review and empirical study investigating reasons for wearable activity tracking attrition. Comput. Hum. Behav. 102, 223–237 (2020). https://doi.org/10.1016/j.chb.2019.08.025

    Article  Google Scholar 

  2. van Berkel, N., Ferreira, D., Kostakos, V.: The experience sampling method on mobile devices. ACM Comput. Surv. 50(6), 1–40 (2017). https://doi.org/10.1145/3123988

    Article  Google Scholar 

  3. van Berkel, N., Kostakos, V.: Recommendations for conducting longitudinal experience sampling studies. In: Karapanos, E., Gerken, J., Kjeldskov, J., Skov, M.B. (eds.) Advances in Longitudinal HCI Research. HIS, pp. 59–78. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67322-2_4

    Chapter  Google Scholar 

  4. van Berkel, N., Luo, C., Ferreira, D., Goncalves, J., Kostakos, V.: The curse of quantified-self: an endless quest for answers. In: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers, pp. 973–978. UbiComp/ISWC 2015 Adjunct (2015). https://doi.org/10.1145/2800835.2800946

  5. Berrocal, A., Concepcion, W., De Dominicis, S., Wac, K.: Complementing human behavior assessment by leveraging personal ubiquitous devices and social links: an evaluation of the peer-ceived momentary assessment method. JMIR Mhealth Uhealth 8(8), e15947 (2020). https://doi.org/10.2196/15947

    Article  Google Scholar 

  6. Blandford, A.: HCI for health and wellbeing: challenges and opportunities. Int. J. Hum Comput Stud. 131, 41–51 (2019). https://doi.org/10.1016/j.ijhcs.2019.06.007

    Article  Google Scholar 

  7. Caraban, A., Karapanos, E., Gonçalves, D., Campos, P.: 23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human-Computer Interaction. In: Proceeding of the 2019 CHI Conference, pp. 1–15 (2019). https://doi.org/10.1145/3290605.3300733

  8. Chalhoub, G., Kraemer, M.J., Nthala, N., Flechais, I.: “it did not give me an option to decline”: a longitudinal analysis of the user experience of security and privacy in smart home products. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2021). https://doi.org/10.1145/3411764.3445691

  9. Cherubini, M., Villalobos-Zuñiga, G., Boldi, M.O., Bonazzi, R.: The unexpected downside of paying or sending messages to people to make them walk: Comparing tangible rewards and motivational messages to improve physical activity. ACM Trans. Comput.-Hum. Interact. 27(2), 1–44 (2020). https://doi.org/10.1145/3365665

  10. Choe, E.K., Lee, N.B., Lee, B., Pratt, W., Kientz, J.A.: Understanding quantified-selfers’ practices in collecting and exploring personal data. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1143–1152 (2014). https://doi.org/10.1145/2556288.2557372

  11. Consolvo, S., Walker, M.: Using the experience sampling method to evaluate ubicomp applications. IEEE Pervasive Comput. 2(2), 24–31 (2003). https://doi.org/10.1109/MPRV.2003.1203750

    Article  Google Scholar 

  12. Eisele, G., et al.: The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population. Assessment (2020). https://doi.org/10.1177/1073191120957102

    Article  Google Scholar 

  13. Fowler, F.: Survey Research Methods. Thousand Oaks, California, 4th edn. (2009). https://doi.org/10.4135/9781452230184

  14. Fritz, T., Huang, E.M., Murphy, G.C., Zimmermann, T.: Persuasive technology in the real world: A study of long-term use of activity sensing devices for fitness. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 487–496. CHI 2014 (2014). https://doi.org/10.1145/2556288.2557383

  15. Golder, S.A., Macy, M.W.: Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science 333(6051), 1878–1881 (2011). https://doi.org/10.1126/science.1202775

    Article  Google Scholar 

  16. Gouveia, R., Karapanos, E., Hassenzahl, M.: How do we engage with activity trackers? a longitudinal study of habito. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 1305–1316. UbiComp 2015 (2015). https://doi.org/10.1145/2750858.2804290

  17. Grudin, J.: Why CSCW applications fail: Problems in the design and evaluation of organizational interfaces. In: Proceedings of the 1988 ACM Conference on Computer-Supported Cooperative Work, pp. 85–93. CSCW 1988 (1988). https://doi.org/10.1145/62266.62273

  18. Hollis, V., Konrad, A., Whittaker, S.: Change of Heart: Emotion Tracking to Promote Behavior Change. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 2643–2652 (2015). https://doi.org/10.1145/2702123.2702196

  19. Horvitz, E., Koch, P., Apacible, J.: Busybody: Creating and fielding personalized models of the cost of interruption. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, pp. 507–510. CSCW 2004 (2004). https://doi.org/10.1145/1031607.1031690

  20. Isaacs, E., Konrad, A., Walendowski, A., Lennig, T., Hollis, V., Whittaker, S.: Echoes from the past: How technology mediated reflection improves well-being. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1071–1080. CHI 2013 (2013). https://doi.org/10.1145/2470654.2466137

  21. Jun, E., Hsieh, G., Reinecke, K.: Types of motivation affect study selection, attention, and dropouts in online experiments. Proc. ACM Hum.-Comput. Interact. 1(CSCW), 1–15 (2017). https://doi.org/10.1145/3134691

  22. Kahneman, D., Krueger, A.B.: Developments in the measurement of subjective well-being. J. Econ. Perspect. 20(1), 3–24 (2006). https://doi.org/10.1257/089533006776526030

    Article  Google Scholar 

  23. Kaptein, M.: Experiments, longitudinal studies, and sequential experimentation: how using “intermediate’’ results can help design experiments. In: Karapanos, E., Gerken, J., Kjeldskov, J., Skov, M.B. (eds.) Advances in Longitudinal HCI Research. HIS, pp. 121–149. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67322-2_7

    Chapter  Google Scholar 

  24. Karapanos, E., Zimmerman, J., Forlizzi, J., Martens, J.B.: User experience over time: An initial framework. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 729–738 (2009). https://doi.org/10.1145/1518701.1518814

  25. Kim, Y.H., Lee, B., Srinivasan, A., Choe, E.K.: Data@hand: Fostering visual exploration of personal data on smartphones leveraging speech and touch interaction. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. CHI 2021 (2021). https://doi.org/10.1145/3411764.3445421

  26. Kjeldskov, J., Paay, J.: A longitudinal review of mobile HCI research methods. In: Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 69–78 (2012). https://doi.org/10.1145/2371574.2371586

  27. Kjærup, M., Skov, M.B., Nielsen, P.A., Kjeldskov, J., Gerken, J., Reiterer, H.: Longitudinal studies in HCI research: a review of CHI publications from 1982–2019. In: Karapanos, E., Gerken, J., Kjeldskov, J., Skov, M.B. (eds.) Advances in Longitudinal HCI Research. HIS, pp. 11–39. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67322-2_2

    Chapter  Google Scholar 

  28. Klasnja, P., Consolvo, S., Pratt, W.: How to Evaluate Technologies for Health Behavior Change in HCI Research. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 3063–3072 (2011). https://doi.org/10.1145/1978942.1979396

  29. Kuosmanen, E., et al.: How does sleep tracking influence your life? experiences from a longitudinal field study with a wearable ring. Proc. ACM Hum.-Comput. Interact. 6(MHCI), 1–19 (2022). https://doi.org/10.1145/3546720

  30. Larson, R., Csikszentmihalyi, M.: The experience sampling method. In: Flow and the Foundations of Positive Psychology, pp. 21–34. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-017-9088-8_2

    Chapter  Google Scholar 

  31. Lee, J.H., Schroeder, J., Epstein, D.A.: Understanding and supporting self-tracking app selection. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5(4), 1–25 (2021). https://doi.org/10.1145/3494980

    Article  Google Scholar 

  32. Lukoff, K., Yu, C., Kientz, J., Hiniker, A.: What makes smartphone use meaningful or meaningless? Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2(1), 1–26 (2018). https://doi.org/10.1145/3191754

    Article  Google Scholar 

  33. Lupton, D.: The Quantified Self. Wiley (2016)

    Google Scholar 

  34. Mattingly, S.M., et al.: The tesserae project: Large-scale, longitudinal, in situ, multimodal sensing of information workers. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–8 (2019). https://doi.org/10.1145/3290607.3299041

  35. McCambridge, J., Kypri, K., Elbourne, D.: Research participation effects: a skeleton in the methodological cupboard. J. Clin. Epidemiol. 67(8), 845–849 (2014). https://doi.org/10.1016/j.jclinepi.2014.03.002

    Article  Google Scholar 

  36. Meyer, J., Simske, S., Siek, K.A., Gurrin, C.G., Hermens, H.: Beyond quantified self: data for wellbeing. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 95–98. CHI EA 2014 (2014). https://doi.org/10.1145/2559206.2560469

  37. Neff, G., Nafus, D.: Self-tracking. MIT Press (2016)

    Google Scholar 

  38. Rabbi, M., Li, K., Yan, H.Y., Hall, K., Klasnja, P., Murphy, S.: ReVibe: a context-assisted evening recall approach to improve self-report adherence. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(4), 1–27 (2019). https://doi.org/10.1145/3369806

    Article  Google Scholar 

  39. Rahmati, A., Tossell, C., Shepard, C., Kortum, P., Zhong, L.: Exploring iphone usage: the influence of socioeconomic differences on smartphone adoption, usage and usability. In: Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 11–20. MobileHCI 2012 (2012). https://doi.org/10.1145/2371574.2371577

  40. Rapp, A., Cena, F.: Affordances for self-tracking wearable devices. In: Proceedings of the 2015 ACM International Symposium on Wearable Computers, pp. 141–142. ISWC 2015 (2015). https://doi.org/10.1145/2802083.2802090

  41. Rieman, J.: The diary study: A workplace-oriented research tool to guide laboratory efforts. In: Proceedings of the INTERACT 1993 and CHI 1993 Conference on Human Factors in Computing Systems, pp. 321–326. CHI 1993 (1993). https://doi.org/10.1145/169059.169255

  42. Rivera-Pelayo, V., Fessl, A., Müller, L., Pammer, V.: Introducing mood self-tracking at work: empirical insights from call centers. ACM Trans. Comput.-Hum. Interact. 24(1), 1–28 (2017). https://doi.org/10.1145/3014058

  43. Sanches, P., et al.: HCI and Affective Health: Taking Stock of a Decade of Studies and Charting Future Research Directions. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2019), pp. 1–17 (2019). https://doi.org/10.1145/3290605.3300475

  44. Sas, C., Höök, K., Doherty, G., Sanches, P., Leufkens, T., Westerink, J.: Mental wellbeing: future agenda drawing from design, HCI and big data. In: Companion Publication of the 2020 ACM Designing Interactive Systems Conference, pp. 425–428. DIS 2020 Companion (2020). https://doi.org/10.1145/3393914.3395920

  45. Schön, D.A.: The reflective practitioner: How professionals think in action. Routledge (2017)

    Google Scholar 

  46. Thieme, A., Wallace, J., Meyer, T.D., Olivier, P.: Designing for mental wellbeing: towards a more holistic approach in the treatment and prevention of mental illness. In: Proceedings of the 2015 British HCI Conference, pp. 1–10. British HCI 2015 (2015). https://doi.org/10.1145/2783446.2783586

  47. Visuri, A., van Berkel, N., Goncalves, J., Rawassizadeh, R., Ferreira, D., Kostakos, V.: Understanding usage style transformation during long-term smartwatch use. Pers. Ubiquit. Comput. 25(3), 535–549 (2021). https://doi.org/10.1007/s00779-020-01511-2

    Article  Google Scholar 

  48. Wang, R., et al.: StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 3–14 (2014). https://doi.org/10.1145/2632048.2632054

  49. Westermann, T., Möller, S., Wechsung, I.: Assessing the relationship between technical affinity, stress and notifications on smartphones. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp. 652–659. MobileHCI 2015 (2015). https://doi.org/10.1145/2786567.2793684

  50. Wilson, A.: Sensor- and Recognition-Based Input for Interaction, chap. 10, pp. 153–176. CRC Press (2009). https://doi.org/10.1201/b10368-13

  51. Yeager, C.M., Shoji, K., Luszczynska, A., Benight, C.C.: Engagement with a trauma recovery internet intervention explained with the health action process approach (HAPA): longitudinal study. JMIR Ment. Health 5(2), e9449 (2018)

    Article  Google Scholar 

  52. Zhang, X., Pina, L.R., Fogarty, J.: Examining unlock journaling with diaries and reminders for in situ self-report in health and wellness. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 5658–5664 (2016). https://doi.org/10.1145/2858036.2858360

Download references

Author information

Authors and Affiliations

  1. Aalborg University, Aalborg, Denmark

    Niels van Berkel & Sujay Shalawadi

  2. Levell, London, UK

    Madeleine R. Evans

  3. University of Oulu, Oulu, Finland

    Aku Visuri & Simo Hosio

Authors
  1. Niels van Berkel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sujay Shalawadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Madeleine R. Evans
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aku Visuri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Simo Hosio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niels van Berkel .

Editor information

Editors and Affiliations

  1. University of West London, London, UK

    José Abdelnour Nocera

  2. Reykjavik University, Reykjavik, Iceland

    Marta Kristín Lárusdóttir

  3. University of York, York, UK

    Helen Petrie

  4. University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  5. Université Côte d’Azur, Sophia Antipolis Cedex, France

    Marco Winckler

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

van Berkel, N., Shalawadi, S., Evans, M.R., Visuri, A., Hosio, S. (2023). A Longitudinal Analysis of Real-World Self-report Data. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14144. Springer, Cham. https://doi.org/10.1007/978-3-031-42286-7_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42286-7_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42285-0

  • Online ISBN: 978-3-031-42286-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships