Using phishing experiments and scenario-based surveys to understand security behaviours in practice

The purpose of the study was threefold: to understand security behaviours in practice by investigating factors that may cause an individual to comply with a request posed by a perpetrator; to investigate if adding information about the victim to an attack increases the probability of the attack being successful; and, finally, to investigate if there is a correlation between self-reported and observed behaviour.

Factors for investigation were identified based on a review of existing literature. Data were collected through a scenario-based survey, phishing experiments, journals and follow-up interviews in three organisations.

The results from the experiment revealed that the degree of target information in an attack increased the likelihood that an organisational employee falls victim to an actual attack. Further, an individual’s trust and risk behaviour significantly affected the actual behaviour during the phishing experiment. Computer experience at work, helpfulness and gender (females tend to be less susceptible to a generic attack than men), had a significant correlation with behaviour reported by respondents in the scenario-based survey. No correlation between the results from the scenario-based survey and the experiments was found.

One limitation is that the scenario-based survey may have been interpreted differently by the participants. Another is that controlling how the participants reacted when receiving the phishing mail, and what actually triggered each and every participant to click on the attached link, was not possible. Data were however collected to capture these aspects during and after the experiments. In conclusion, the results do not imply that one or the other method should be ruled out, as they have both advantages and disadvantages which should be considered in the context of collecting data in the critical domain of information security.

Two different methods to collect data to understand security behaviours have rarely been used in previous research. Studies that add target information to understand if such information could increase the probability of attack success is sparse. This paper includes both approaches.