skip to main content
10.1145/3025453.3025911acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Public Access

Can People Self-Report Security Accurately?: Agreement Between Self-Report and Behavioral Measures

Published: 02 May 2017 Publication History

Abstract

It is common for researchers to use self-report measures (e.g. surveys) to measure people's security behaviors. In the computer security community, we don't know what behaviors people understand well enough to self-report accurately, or how well those self-reports correlate with what people actually do. In a six week field study, we collected both behavior data and survey responses from 122 subjects. We found that a relatively small number of behaviors -- mostly related to tasks that require users to take a specific, regular action -- have non-zero correlations. Since security is almost never a user's primary task for everyday computer users, several important security behaviors that we directly measured were not self-reported accurately. These results suggest that security research based on self-report is only reliable for certain behaviors. Additionally, a number of important security behaviors are not sufficiently salient to users that they can self-report accurately.

References

[1]
2015. Block Shock. The Economist (June 6 2015). http://www.economist.com/node/21653644/print
[2]
Icek Ajzen. 1991. The theory of planned behavior. Organizational Behavior and Human Decision Processes 50, 2 (Dec. 1991), 179--211.
[3]
John R. Anderson. 2009. Cognitive Psychology and Its Limitations (7th ed.). Worth Publishers.
[4]
Serge Egelman, Marian Harbach, and Eyal Peer. 2016. Behavior Ever Follows Intention? A Validation of the Security Behavior Intentions Scale (SeBIS). In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 5257--5261.
[5]
Serge Egelman and Eyal Peer. 2015. Scaling the Security Wall: Developing a Security Behavior Intentions Scale. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). 2873--2882.
[6]
Dinei Florêncio and Cormac Herley. 2007. A large-scale study of web password habits. In Proceedings of the 16th International Conference on World Wide Web (WWW). 657--666.
[7]
Alain Forget, Sarah Pearman, Jeremy Thomas, Alessandro Acquisti, Nicolas Christin, Lorrie Faith Cranor, Serge Egelman, Marian Harbach, and Rahul Telang. 2016. Do or Do Not, There Is No Try: User Engagement May Not Improve Security Outcomes. In Twelfth Symposium on Usable Privacy and Security (SOUPS 2016). 97--111.
[8]
Andrew Gelman and John Carlin. 2014. Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors. Perspectives on Psychological Science 9, 6 (2014), 641--651.
[9]
Joel Hruska. 2016. Forbes forces readers to turn off ad blockers, promptly serves malware. (January 8 2016). http://www.extremetech.com/internet/220696-forbesforces-readers-to-turn-off-ad-blockers-promptlyserves-malware
[10]
Iulia Ion, Rob Reeder, and Sunny Consolvo. 2015. "... no one can hack my mind": Comparing Expert and Non-Expert Security Practices. ... On Usable Privacy and Security (SOUPS ... (2015).
[11]
Ivar Krumpal. 2013. Determinants of social desirability bias in sensitive surveys: a literature review. Quality & Quantity 47, 4 (2013), 2025--2047.
[12]
Robert LaRose, Nora J Rifon, and Richard Enbody. 2008. Promoting personal responsibility for internet safety. Commun. ACM 51, 3 (March 2008), 71--76.
[13]
Paschal Sheeran. 2011. Intention-Behavior Relations: A Conceptual and Empirical Review. European Review of Social Psychology 12, 1 (2011), 1--36.
[14]
Kami Vaniea, Emilee Rader, and Rick Wash. 2014. Betrayed by Updates: How Negative Experiences Affect Future Security. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2671--2674.
[15]
Rick Wash. 2010. Folk Models of Home Computer Security. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS). Seattle, WA.
[16]
Rick Wash and Emilee Rader. 2015. Too Much Knowledge? Security Beliefs and Protective Behaviors Among US Internet Users. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS).
[17]
Rick Wash, Emilee Rader, Ruthie Berman, and Zac Wellmer. 2016. Understanding Password Choices: How Frequently Entered Passwords are Re-used Across Websites. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS). Denver, CO.
[18]
Rick Wash, Emilee Rader, Kami Vaniea, and Michelle Rizor. 2014. Out of the Loop: How Automated Software Updates Cause Unintended Security Consequences. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS). 89--104.
[19]
Ryan West. 2008. The Psychology of Security. Commun. ACM 51, 4 (2008), 34--40.

Cited By

View all
  • (2024)Encouraging Users to Change Breached Passwords Using the Protection Motivation TheoryACM Transactions on Computer-Human Interaction10.1145/368943231:5(1-45)Online publication date: 30-Aug-2024
  • (2024)Know their Customers: An Empirical Study of Online Account Enumeration AttacksACM Transactions on the Web10.1145/366420118:3(1-36)Online publication date: 17-Jun-2024
  • (2024)E-Scooter Dynamics: Unveiling Rider Behaviours and Interactions with Road Users through Multi-Modal Data AnalysisProceedings of the Augmented Humans International Conference 202410.1145/3652920.3653056(307-310)Online publication date: 4-Apr-2024
  • Show More Cited By

Index Terms

  1. Can People Self-Report Security Accurately?: Agreement Between Self-Report and Behavioral Measures

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
    May 2017
    7138 pages
    ISBN:9781450346559
    DOI:10.1145/3025453
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 May 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. intentions
    2. security
    3. self-report

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CHI '17
    Sponsor:

    Acceptance Rates

    CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)260
    • Downloads (Last 6 weeks)24
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Encouraging Users to Change Breached Passwords Using the Protection Motivation TheoryACM Transactions on Computer-Human Interaction10.1145/368943231:5(1-45)Online publication date: 30-Aug-2024
    • (2024)Know their Customers: An Empirical Study of Online Account Enumeration AttacksACM Transactions on the Web10.1145/366420118:3(1-36)Online publication date: 17-Jun-2024
    • (2024)E-Scooter Dynamics: Unveiling Rider Behaviours and Interactions with Road Users through Multi-Modal Data AnalysisProceedings of the Augmented Humans International Conference 202410.1145/3652920.3653056(307-310)Online publication date: 4-Apr-2024
    • (2024)How We Browse: Measurement and Analysis of Browsing Behavior2024 IEEE 6th International Conference on Cognitive Machine Intelligence (CogMI)10.1109/CogMI62246.2024.00041(257-264)Online publication date: 28-Oct-2024
    • (2023)An Empirical Study on Secure Usage of Mobile Health Apps: The Attack Simulation ApproachInformation and Software Technology10.1016/j.infsof.2023.107285(107285)Online publication date: Jun-2023
    • (2023)Relationship Between ADHD-Like Traits and Emotion Dysregulation in the Adult General PopulationAdvances in Neurodevelopmental Disorders10.1007/s41252-023-00381-y8:4(627-637)Online publication date: 8-Dec-2023
    • (2022)On recruiting and retaining users for security-sensitive longitudinal measurement panelsProceedings of the Eighteenth USENIX Conference on Usable Privacy and Security10.5555/3563609.3563628(347-366)Online publication date: 8-Aug-2022
    • (2022)Am I Private and If So, how Many?Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security10.1145/3548606.3560693(1125-1139)Online publication date: 7-Nov-2022
    • (2022)Holding Your Hand on the Danger ButtonProceedings of the ACM on Human-Computer Interaction10.1145/35467306:MHCI(1-22)Online publication date: 20-Sep-2022
    • (2022)The self-assessed information security skills of the Finnish populationComputers and Security10.1016/j.cose.2022.102732118:COnline publication date: 1-Jul-2022
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media