skip to main content
10.1145/3500868.3559456acmconferencesArticle/Chapter ViewAbstractPublication PagescscwConference Proceedingsconference-collections
extended-abstract

Screenshot Journey Auditor: A Tool to Support Analysis of Smartphone Media Consumption Journey Using Screenshot Data

Published: 08 November 2022 Publication History

Abstract

Taking long series of screenshots for capturing and studying smartphone users’ phone usage and media consumption has recently attracted research attention due to its advantage of capturing rich contextual information from users’ phone use journeys. However, that approach creates a high volume of screenshots that take very considerable time and effort to inspect and annotate, especially when the granularity of analysis is low: such as when distinguishing among media-content units (e.g, single FB posts) and detecting events in them. We therefore developed Screenshot Journey Auditor (SJA), a web application that identifies individual social media posts, and detects news items and other events of interest in them. It then visualizes users’ journeys ‘flow’ among these media-content units. SJA also enables researchers/coders to collaboratively correct detections online. We evaluated SJA with five coders and received positive feedback on how the detections and visualizations made the analysis process more efficient and informative.

References

[1]
Frank Bentley, Katie Quehl, Jordan Wirfs-Brock, and Melissa Bica. 2019. Understanding Online News Behaviors. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3290605.3300820
[2]
Pablo J Boczkowski, Eugenia Mitchelstein, and Mora Matassi. 2018. “News comes across when I’m in a moment of leisure”: Understanding the practices of incidental news consumption on social media. New Media & Society 20, 10 (2018), 3523–3539. https://doi.org/10.1177/1461444817750396 arXiv:https://doi.org/10.1177/1461444817750396
[3]
Miriam Brinberg, Nilam Ram, Xiao Yang, Mu-Jung Cho, S. Shyam Sundar, Thomas N. Robinson, and Byron Reeves. 2021. The idiosyncrasies of everyday digital lives: Using the Human Screenome Project to study user behavior on smartphones. Computers in Human Behavior 114 (2021), 106570. https://doi.org/10.1016/j.chb.2020.106570
[4]
Marios Constantinides, John Dowell, David Johnson, and Sylvain Malacria. 2015. Exploring Mobile News Reading Interactions for News App Personalisation. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (Copenhagen, Denmark) (MobileHCI ’15). Association for Computing Machinery, New York, NY, USA, 457–462. https://doi.org/10.1145/2785830.2785860
[5]
Kristin Van Damme, Cédric Courtois, Karel Verbrugge, and Lieven De Marez. 2015. What’s APPening to news? A mixed-method audience-centred study on mobile news consumption. Mobile Media & Communication 3, 2 (2015), 196–213. https://doi.org/10.1177/2050157914557691 arXiv:https://doi.org/10.1177/2050157914557691
[6]
Kristin Van Damme, Marijn Martens, Sarah Van Leuven, Mariek Vanden Abeele, and Lieven De Marez. 2020. Mapping the Mobile DNA of News. Understanding Incidental and Serendipitous Mobile News Consumption. Digital Journalism 8, 1 (2020), 49–68. https://doi.org/10.1080/21670811.2019.1655461 arXiv:https://doi.org/10.1080/21670811.2019.1655461
[7]
Nir Grinberg. 2018. Identifying Modes of User Engagement with Online News and Their Relationship to Information Gain in Text. In Proceedings of the 2018 World Wide Web Conference(Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1745–1754. https://doi.org/10.1145/3178876.3186180
[8]
Donghan Hu and Sang Won Lee. 2020. ScreenTrack: Using a Visual History of a Computer Screen to Retrieve Documents and Web Pages. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376753
[9]
Martijn Kleppe and Marco Otte. 2017. Analysing and understanding news consumption patterns by tracking online user behaviour with a multimodal research design. Digital Scholarship in the Humanities 32 (06 2017), ii158–ii170. https://doi.org/10.1093/llc/fqx030
[10]
Tetsuro Kobayashi and Jeffrey Boase. 2012. No Such Effect? The Implications of Measurement Error in Self-Report Measures of Mobile Communication Use. Communication Methods and Measures 6, 2 (2012), 126–143. https://doi.org/10.1080/19312458.2012.679243 arXiv:https://doi.org/10.1080/19312458.2012.679243
[11]
Dmitry Lagun and Mounia Lalmas. 2016. Understanding User Attention and Engagement in Online News Reading. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining (San Francisco, California, USA) (WSDM ’16). Association for Computing Machinery, New York, NY, USA, 113–122. https://doi.org/10.1145/2835776.2835833
[12]
Danielle Lottridge, Katie Quehl, Frank Bentley, Max Silverman, Melissa Ong, Michael Dickard, Brooke White, and Ethan Plaut. 2022. Ubiquitous News Experienced Alone: Interviews with Americans and Their Devices. Proc. ACM Hum.-Comput. Interact. 6, CSCW1, Article 128 (apr 2022), 29 pages. https://doi.org/10.1145/3512975
[13]
Judith Möller, Robbert Nicolai van de Velde, Lisa Merten, and Cornelius Puschmann. 2020. Explaining Online News Engagement Based on Browsing Behavior: Creatures of Habit?Social Science Computer Review 38, 5 (2020), 616–632. https://doi.org/10.1177/0894439319828012 arXiv:https://doi.org/10.1177/0894439319828012
[14]
Nic Newman, Richard Fletcher, Anne Schulz, Simge Andi, Craig T Robertson, and Rasmus Kleis Nielsen. 2021. Reuters Institute digital news report 2021. Reuters Institute for the Study of Journalism (2021).
[15]
Jakob Ohme, Ewa Maslowska, and Cornelia Mothes. 2021. Mobile News Learning — Investigating Political Knowledge Gains in a Social Media Newsfeed with Mobile Eye Tracking. Political Communication 0, 0 (2021), 1–19. https://doi.org/10.1080/10584609.2021.2000082 arXiv:https://doi.org/10.1080/10584609.2021.2000082
[16]
Keith Rayner. 1998. Eye movements in reading and information processing: 20 years of research.Psychological bulletin 124, 3 (1998), 372.
[17]
Keith Rayner. 2009. Eye movements and attention in reading, scene perception, and visual search. The Quarterly Journal of Experimental Psychology 62, 8(2009), 1457–1506. https://doi.org/10.1080/17470210902816461 arXiv:https://doi.org/10.1080/17470210902816461PMID: 19449261.
[18]
Byron Reeves, Nilam Ram, Thomas N. Robinson, James J. Cummings, C. Lee Giles, Jennifer Pan, Agnese Chiatti, Mj Cho, Katie Roehrick, Xiao Yang, Anupriya Gagneja, Miriam Brinberg, Daniel Muise, Yingdan Lu, Mufan Luo, Andrew Fitzgerald, and Leo Yeykelis. 2021. Screenomics: A Framework to Capture and Analyze Personal Life Experiences and the Ways that Technology Shapes Them. Human–Computer Interaction 36, 2 (2021), 150–201. https://doi.org/10.1080/07370024.2019.1578652 arXiv:https://doi.org/10.1080/07370024.2019.1578652PMID: 33867652.
[19]
Adam Rule, Aurélien Tabard, and Jim Hollan. 2017. Using Visual Histories to Reconstruct the Mental Context of Suspended Activities. Human–Computer Interaction 32, 5-6 (2017), 511–558. https://doi.org/10.1080/07370024.2017.1300063 arXiv:https://doi.org/10.1080/07370024.2017.1300063
[20]
Yoonmo Sang, Jee Young Lee, Sora Park, Caroline Fisher, and Glen Fuller. 2020. Signalling and Expressive Interaction: Online News Users’ Different Modes of Interaction on Digital Platforms. Digital Journalism 8, 4 (2020), 467–485. https://doi.org/10.1080/21670811.2020.1743194 arXiv:https://doi.org/10.1080/21670811.2020.1743194
[21]
Elisa Shearer and Amy Mitchell. 2021. News Use Across Social Media Platforms in 2020. Pew Research Available at: https://www.pewresearch.org/journalism/2021/01/12/news-use-across-social-media-platforms-in-2020/(2021).
[22]
Arthur A Stone, Christine A Bachrach, Jared B Jobe, Howard S Kurtzman, and Virginia S Cain. 1999. The science of self-report: Implications for research and practice. Psychology Press.
[23]
Harsh Taneja, James G. Webster, Edward C. Malthouse, and Thomas B. Ksiazek. 2012. Media consumption across platforms: Identifying user-defined repertoires. New Media & Society 14, 6 (2012), 951–968. https://doi.org/10.1177/1461444811436146 arXiv:https://doi.org/10.1177/1461444811436146
[24]
David Tewksbury. 2006. What Do Americans Really Want to Know? Tracking the Behavior of News Readers on the Internet. Journal of Communication 53, 4 (01 2006), 694–710. https://doi.org/10.1111/j.1460-2466.2003.tb02918.x arXiv:https://academic.oup.com/joc/article-pdf/53/4/694/22330549/jjnlcom0694.pdf

Index Terms

  1. Screenshot Journey Auditor: A Tool to Support Analysis of Smartphone Media Consumption Journey Using Screenshot Data

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CSCW'22 Companion: Companion Publication of the 2022 Conference on Computer Supported Cooperative Work and Social Computing
    November 2022
    318 pages
    ISBN:9781450391900
    DOI:10.1145/3500868
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 November 2022

    Check for updates

    Author Tags

    1. Analysis tool
    2. Mobile phone
    3. News consumption
    4. Screenshot
    5. collaboration

    Qualifiers

    • Extended-abstract
    • Research
    • Refereed limited

    Funding Sources

    Conference

    CSCW '22
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

    Upcoming Conference

    CSCW '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 157
      Total Downloads
    • Downloads (Last 12 months)34
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media