extended-abstract

Following Topics Across All Apps and Media Formats: Mobile Keyword Tracking as a Privacy-Friendly Data Source in Mobile Media Research

Authors:

Philipp Krieter,

Patrick Zerrer,

Cornelius Puschmann,

Stephanie GeiseAuthors Info & Claims

UMAP Adjunct '24: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization

Pages 126 - 131

https://doi.org/10.1145/3631700.3664879

Published: 28 June 2024 Publication History

Abstract

Mining detailed content from mobile human-computer interactions often relies on broad log files, which limits the specificity of analyses. Our study presents a unique approach capitalizing on Optical Character Recognition to continuously detect keywords across all applications and media formats, correlating this with system logs for context and duration tracking. Such a strategy even allows tracing of topics within pictorial content like memes on social media. A privacy concept based on a whitelist for keywords and topics and anonymized log files addresses typical concerns of potential study participants regarding their personal data. In our four-month study involving 25 participants, we generated an expansive nine-million-point dataset. This detailed dataset not only validates the efficacy of our approach but also exemplifies its capacity for rich, cross-app, long-term interaction analysis. In order to clarify data protection issues, we conducted qualitative interviews with eight of the participants on a voluntary basis.

References

[1]

Pekka Aho, Teemu Kanstren, Tomi Räty, and Juha Röning. 2014. Advances in Computers. Academic Press.

[2]

Louise Barkuus and Anind Dey. 2003. Location-Based Services for Mobile Telephony: a Study of Users’ Privacy Concerns. In In 9th Ifip Tc13 International Conference on Human-Computer Interaction, Interact 2003.

[3]

Carol M. Barnum. 2020. Usability Testing Essentials: Ready, Set...Test!Elsevier.

[4]

Steven M. Boker, Timothy R. Brick, Joshua N. Pritikin, Yang Wang, Timo von Oertzen, Donald Brown, John Lach, Ryne Estabrook, Michael D. Hunter, Hermine H. Maes, and Michael C. Neale. 2015. Maintained Individual Data Distributed Likelihood Estimation (MIDDLE). Multivariate Behavioral Research 50, 6 (Nov. 2015), 706–720. https://doi.org/10.1080/00273171.2015.1094387

[5]

Matthias Böhmer, Brent Hecht, Johannes Schöning, Antonio Krüger, and Gernot Bauer. 2011. Falling Asleep with Angry Birds, Facebook and Kindle: A Large Scale Study on Mobile Application Usage. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services(MobileHCI ’11). ACM, New York, NY, USA, 47–56. https://doi.org/10.1145/2037373.2037383

Digital Library

[6]

Denzil Ferreira, Vassilis Kostakos, and Anind K. Dey. 2015. AWARE: Mobile Context Instrumentation Framework. Frontiers in ICT 2 (2015). https://doi.org/10.3389/fict.2015.00006

[7]

Christian Frisson, Sylvain Malacria, Gilles Bailly, and Thierry Dutoit. 2016. InspectorWidget: A System to Analyze Users Behaviors in Their Applications. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems(CHI EA ’16). ACM, New York, NY, USA, 1548–1554. https://doi.org/10.1145/2851581.2892388

Digital Library

[8]

Maaike van den Haak, Menno De Jong, and Peter Jan Schellens. 2003. Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue. Behaviour & Information Technology 22, 5 (Sept. 2003), 339–351. https://doi.org/10.1080/0044929031000

[9]

Uwe Hasebrink, Sascha Hölig, and Leonie Wunderlich. 2021. UseTheNews: Studie zur Nachrichtenkompetenz Jugendlicher und junger Erwachsener in der digitalen Medienwelt. (2021).

[10]

Uwe Hasebrink and Jutta Popp. 2006. Media repertoires as a result of selective media use. A conceptual approach to the analysis of patterns of exposure. (2006).

[11]

Jason I. Hong, Gaetano Boriello, James A. Landay, David W. Mcdonald, Bill N. Schilit, and J. D. Tygar. 2003. Privacy and security in the locationenhanced world wide web. In In Proceedings of Fifth International Conference on Ubiquitous Computing: Ubicomp 2003 (Workshop on Ubicomp Communities: Privacy as Boundary Negotiation. 2003.

[12]

Jason I. Hong and James A. Landay. 2004. An Architecture for Privacy-sensitive Ubiquitous Computing. In Proceedings of the 2Nd International Conference on Mobile Systems, Applications, and Services(MobiSys ’04). ACM, New York, NY, USA, 177–189. https://doi.org/10.1145/990064.990087

Digital Library

[13]

Philipp Krieter. 2019. Can I Record Your Screen? Mobile Screen Recordings as a Long-Term Data Source for User Studies. In Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia (Pisa, Italy) (MUM ’19). Association for Computing Machinery, New York, NY, USA, Article 23, 10 pages. https://doi.org/10.1145/3365610.3365618

Digital Library

[14]

Philipp Krieter. 2020. Looking Inside - Mobile Screen Recordings as a Privacy Friendly Long-Term Data Source to Analyze User Behavior. (2020).

[15]

Philipp Krieter. 2024. KeywordLogger. https://github.com/phil-iu/keywordlogger

[16]

Philipp Krieter and Andreas Breiter. 2018. Analyzing Mobile Application Usage: Generating Log Files from Mobile Screen Recordings. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services(MobileHCI ’18). ACM, New York, NY, USA, 9:1–9:10. https://doi.org/10.1145/3229434.3229450

Digital Library

[17]

Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research Methods in Human-Computer Interaction. Morgan Kaufmann. Google-Books-ID: hbkxDQAAQBAJ.

[18]

Donald McMillan, Moira McGregor, and Barry Brown. 2015. From in the Wild to in Vivo: Video Analysis of Mobile Device Use. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services(MobileHCI ’15). ACM, New York, NY, USA, 494–503. https://doi.org/10.1145/2785830.2785883

Digital Library

[19]

Douglas A. Parry, Britanny I. Davidson, Craig J. R. Sewall, Jacob T. Fisher, Hannah Mieczkowski, and Daniel S. Quintana. 2021. A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use. Nature human behaviour (2021). https://doi.org/10.1038/s41562-021-01117-5

[20]

Byron Reeves, Nilam Ram, Thomas N. Robinson, James J. Cummings, C. Lee Giles, Jennifer Pan, Agnese Chiatti, Mj Cho, Katie Roehrick, Xiao Yang, Anupriya Gagneja, Miriam Brinberg, Daniel Muise, Yingdan Lu, Mufan Luo, Andrew Fitzgerald, and Leo Yeykelis. 2019. Screenomics: A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them. Human–Computer Interaction 0, 0 (March 2019), 1–52. https://doi.org/10.1080/07370024.2019.1578652

[21]

Byron Reeves, Nilam Ram, Thomas N. Robinson, James J. Cummings, C. Lee Giles, Jennifer Pan, Agnese Chiatti, MJ Cho, Katie Roehrick, Xiao Yang, Anupriya Gagneja, Miriam Brinberg, Daniel Muise, Yingdan Lu, Mufan Luo, Andrew Fitzgerald, and Leo Yeykelis. 2021. Screenomics: A framework to capture and analyze personal life experiences and the ways that technology shapes them. Human–Computer Interaction 36, 2 (2021), 150–201. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8045984/pdf/nihms-1037619.pdf

[22]

Sarah Spiekermann and Lorrie F. Cranor. 2009. Engineering Privacy. IEEE Transactions on Software Engineering 35, 1 (Jan. 2009), 67–82. https://doi.org/10.1109/TSE.2008.88

Digital Library

[23]

John C. Tang, Sophia B. Liu, Michael Muller, James Lin, and Clemens Drews. 2006. Unobtrusive but Invasive: Using Screen Recording to Collect Field Data on Computer-mediated Interaction. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work(CSCW ’06). ACM, New York, NY, USA, 479–482. https://doi.org/10.1145/1180875.1180948

Digital Library

[24]

Tung Vuong, Giulio Jacucci, and Tuukka Ruotsalo. 2017. Watching inside the Screen: Digital Activity Monitoring for Task Recognition and Proactive Information Retrieval. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (Nov. 2017), 109. https://doi.org/10.1145/3130974

Digital Library

Cited By

Krieter P(2024)Recall for Good: Flexible Retrospective Mobile In-App Topic Tracking in a Privacy-Friendly Local-First ApproachProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3703382(451-453)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3703382

Index Terms

Following Topics Across All Apps and Media Formats: Mobile Keyword Tracking as a Privacy-Friendly Data Source in Mobile Media Research
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools

Recommendations

Recall for Good: Flexible Retrospective Mobile In-App Topic Tracking in a Privacy-Friendly Local-First Approach
MUM '24: Proceedings of the International Conference on Mobile and Ubiquitous Multimedia
Tracking emerging topics in mobile media research panels offers valuable insights into how people engage with new events. This paper introduces an innovative solution that can dynamically adapt to changing topics of interest, such as political events, ...
A Survey on Privacy in Social Media: Identification, Mitigation, and Applications

The increasing popularity of social media has attracted a huge number of people to participate in numerous activities on a daily basis. This results in tremendous amounts of rich user-generated data. These data provide opportunities for researchers and ...
The contribution of mobile social media to social capital and psychological well-being

Using a two-wave panel in Hong Kong, this study examines how different uses of mobile social media contribute to individuals' social capital and psychological well-being. Findings showed that communicative use and self-disclosure on mobile social media ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UMAP Adjunct '24: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization

June 2024

662 pages

ISBN:9798400704666

DOI:10.1145/3631700

General Chairs:
Ludovico Boratto
University of Cagliari, Italy
,
Cristina Gena
University of Turin, Italy
,
Mirko Marras
University of Cagliari, Italy
,
Program Chairs:
Panagiotis Germanakos
SAP SE, Germany
,
Elvira Popescus
University of Craiova, Romania

Copyright © 2024 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2024

Check for updates

Author Tags

Qualifiers

Extended-abstract
Research
Refereed limited

Conference

UMAP '24

Sponsor:

UMAP '24: 32nd ACM Conference on User Modeling, Adaptation and Personalization

July 1 - 4, 2024

Cagliari, Italy

Acceptance Rates

Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Sponsor:
sigchi
sigchi

33rd ACM Conference on User Modeling, Adaptation and Personalization

June 16 - 19, 2025

New York City , NY , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
69
Total Downloads

Downloads (Last 12 months)69
Downloads (Last 6 weeks)8

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Krieter P(2024)Recall for Good: Flexible Retrospective Mobile In-App Topic Tracking in a Privacy-Friendly Local-First ApproachProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3703382(451-453)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3703382

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten