Skip to main content
SpringerLink
Log in

Metacognitive Skills in Phishing Email Detection: A Study of Calibration and Resolution

  • Conference paper
  • First Online:
Secure Knowledge Management In The Artificial Intelligence Era (SKM 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1549))

  • 417 Accesses

Abstract

Metacognition plays important roles in human judgments. In this study, we study two types of metacognitive skills, namely calibration and resolution, in individuals’ judgments of phishing emails. Drawing upon the Probabilistic Mental Model (PMM) and past research on phishing detection, we examine individual- and task-related factors and their impacts on both skills. Results from an online survey experiment show that task-related factors (i.e., email familiarity, judgment time, variability of judgment time, and task easiness) influence calibration while both task- and individual-related factors (i.e., online transaction experience, victimization experience, email entity familiarity, and variability of judgment time) influence resolution. Interventions to improve individuals’ metacognition in phishing email detection are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abbasi, A., Zahedi, F.M., Zeng, D., Chen, Y., Chen, H., Nunamaker, J.F.: Enhancing predictive analytics for anti-phishing by exploiting website genre information. J. Manag. Inf. Syst. 31(4), 109–157 (2015)

    Google Scholar 

  2. Abbasi, A., Zhang, Z., Zimbra, D., Chen, H., Nunamaker, J.J.F.: Detecting fake websites: the contribution of statistical learning theory. MIS Q. 34(3), 435–461 (2010)

    Google Scholar 

  3. Alba, J.W., Hutchinson, J.W.: Knowledge calibration: what consumers know and what they think they know. J. Consum. Res. 27(2), 123–156 (2000)

    Google Scholar 

  4. Anderson, B.B., Vance, A., Kirwan, C.B., Jenkins, J.L., Eargle, D.: From warning to wallpaper: why the brain habituates to security warnings and what can be done about it. J. Manag. Inf. Syst. 33(3), 713–743 (2016)

    Google Scholar 

  5. Baranski, J.V., Petrusic, W.M.: The calibration and resolution of confidence in perceptual judgments. Percept. Psychophys. 55, 412–428 (1994)

    Google Scholar 

  6. Björkman, M.: Knowledge, calibration, and resolution: a linear model. Organ. Behav. Hum. Decis. Process. 51(1), 1–21 (1992)

    MathSciNet  Google Scholar 

  7. Björkman, M.: Internal cue theory: calibration and resolution of confidence in general knowledge. Organ. Behav. Hum. Decis. Process. 58(3), 386–405 (1994)

    Google Scholar 

  8. Brucks, M.: The effects of product class knowledge on information search behavior. J. Consum. Res. 12(1), 1–16 (1985)

    MathSciNet  Google Scholar 

  9. Canfield, C.I., Fischhoff, B., Davis, A.: Better beware: comparing metacognition for phishing and legitimate emails. Metacogn. Learn. 14(3), 343–362 (2019). https://doi.org/10.1007/s11409-019-09197-5

  10. Carlton, M., Levy, Y., Ramim, M., Terrell, S.: Development of the MyCyberSkills™ iPad app: a scenarios-based, hands-on measure of non-it professionals’ cybersecurity skills. Pre-ICIS Workshop on Information Security and Privacy (SIGSEC). Fort Worth, Texas (2015)

    Google Scholar 

  11. Chen, R., Wang, J., Herath, T., Rao, H.R.: An investigation of email processing from a risky decision making perspective. Decis. Support Syst. 52(1), 73–81 (2011)

    Google Scholar 

  12. Dhamija, R., Tygar, J.D., Hearst, M.: Why phishing works. Inf. Syst. Educ. J. 13(5), 581–590 (2006)

    Google Scholar 

  13. Downs, J.S., Holbrook, M.B., Cranor, L.F.: Decision Strategies and Susceptibility to Phishing, vol. 79 (2006). https://doi.org/10.1145/1143120.1143131

  14. Ferri, C., Hernández-Orallo, J., Modroiu, R.: An experimental comparison of performance measures for classification. Pattern Recogn. Lett. 30(1), 27–38 (2009)

    Google Scholar 

  15. George, J.F., et al.: The role of e-training in protecting information assets against deception attacks. MIS Q. Exec. 7(2), 85–97 (2008)

    Google Scholar 

  16. Gigerenzer, G.: How to make cognitive illusions disappear: beyond “heuristics and biases.” Eur. Rev. Soc. Psychol. 2, 83–115 (1991)

    Google Scholar 

  17. Gigerenzer, G., Hoffrage, U., Kleinbolting, H.: Probabilistic mental models: a Brunswikian theory of confidence. Psychol. Rev. 98, 506–528 (1991)

    Google Scholar 

  18. Hadar, L., Sood, S., Fox, C.R.: Subjective knowledge in consumer financial decisions. J. Mark. Res. 50(3), 303–316 (2013)

    Google Scholar 

  19. Heath, C., Tversky, A.: Preference and belief: ambiguity and competence in choice under uncertainty. J. Risk Uncertain. 4(1), 5–28 (1991)

    MATH  Google Scholar 

  20. Hong, K.W., Kelley, C.M., Tembe, R., Murphy-Hill, E., Mayhorn, C.B.: Keeping up with the Joneses: assessing phishing susceptibility in an email task. Proc. Hum. Fact. Ergon. Soc. Ann. Meet. 57(1), 1012–1016 (2013). https://doi.org/10.1177/1541931213571226

    Article  Google Scholar 

  21. Jensen, M.L., Dinger, M., Wright, R.T., Thatcher, J.B.: Training to mitigate phishing attacks using mindfulness techniques. J. Manag. Inf. Syst. 34(2), 597–626 (2017)

    Google Scholar 

  22. Juslin, P., Olsson, H., Mats, B.: Brunswikian and Thurstonian origins of bias in probability assessment: on the interpretation of stochastic components of judgment. J. Behav. Decis. Mak. 10(3), 189–209 (1997)

    Google Scholar 

  23. Keren, G.: On the calibration of probability judgments: some critical comments and alternative perspectives. J. Behav. Decis. Mak. 10(3), 269–278 (1997)

    Google Scholar 

  24. Klein, B.D., Goodhue, D.L., Davis, G.B.: Can humans detect errors in data? Impact of base rates, incentives, and goals. MIS Q. 21(2), 169–194 (1997)

    Google Scholar 

  25. Koriat, A.: Metacognition: Decision Making Processes in Self‐Monitoring and Self‐Regulation. Wiley, New Jersey (2015)

    Google Scholar 

  26. Krug, K.: The relationship between confidence and accuracy: current thoughts of the literature and a new area of research. Appl. Psychol. Crim. Just. 3(1), 7–41 (2007)

    Google Scholar 

  27. Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L.F., Hong, J.: Teaching Johnny not to fall for phish. ACM Trans. Internet Technol. 10(2), 1–31 (2010)

    Google Scholar 

  28. Li, Y.: The impact of disposition to privacy, website reputation and website familiarity on information privacy concerns. Decis. Support Syst. 57(6), 343–354 (2014)

    Google Scholar 

  29. Liberman, V., Tversky, A.: On the evaluation of probability judgments: calibration, resolution, and monotonicity. Psychol. Bull. 114(1), 162 (1993)

    Google Scholar 

  30. Moody, G., Galletta, D., Dunn, B.: Which phish get caught? An exploratory study of individuals’ susceptibility to phishing. Eur. J. Inf. Syst. 26(6), 564–584 (2017)

    Google Scholar 

  31. Palmer, M.A., Brewer, N., Weber, N., Nagesh, A.: the confidence-accuracy relationship for eyewitness identification decisions: effects of exposure duration, retention interval, and divided attention. J. Exp. Psychol. 19(1), 55–71 (2013)

    Google Scholar 

  32. Payne, J.W.: Task complexity and contingent processing in decision making: an information search and protocol analysis. Organ. Behav. Hum. Perform. 16(2), 366–387 (1976)

    Google Scholar 

  33. Payne, J.W.: Contingent decision behavior. Psychol. Bull. 92(2), 382–402 (1982)

    Google Scholar 

  34. Pillai, K.G., Hofacker, C.: Calibration of consumer knowledge of the web. Int. J. Res. Mark. 24(3), 254–267 (2007)

    Google Scholar 

  35. Rao, J.M., Reiley, D.H.: The economics of spam. J. Econ. Perspect. 26(3), 87–110 (2012)

    Google Scholar 

  36. Schmid, C.H., Griffith, J.L.: Multivariate Classification Rules: Calibration and Discrimination. Wiley, New Jersey (2005)

    Google Scholar 

  37. Schneider, S.L.: Item difficulty, discrimination, and the confidence-frequency effect in a categorical judgment task. Organ. Behav. Hum. Decis. Process. 61(2), 148–167 (1995)

    Google Scholar 

  38. Sheng, S., Magnien, B., Kumaraguru, P., Acquisti, A., Cranor, L.F.: Anti-Phishing Phil: the Design and Evaluation of a Game That Teaches People Not to Fall for Phish. Carnegie Mellon University, Pittsburgh (2007)

    Google Scholar 

  39. Stone, D.N.: Overconfidence in initial self-efficacy judgments: effects on decision processes and performance. Organ. Behav. Hum. Decis. Process. 59(3), 452–472 (1994)

    Google Scholar 

  40. Stone, E.R., Opel, R.B.: Training to improve calibration and discrimination: the effects of performance and environmental feedback. Organ. Behav. Hum. Decis. Process. 83(2), 282–309 (2000)

    Google Scholar 

  41. Tang, F., Hess, T.J., Valacich, J.S., Sweeney, J.T.: The Effects of visualization and interactivity on calibration in financial decision-making. Behav. Res. Account. 26(1), 25–58 (2014)

    Google Scholar 

  42. Tenney, E.R., Spellman, B.A., Maccoun, R.J.: The benefits of knowing what you know (and what you don’t): how calibration affects credibility. J. Exp. Soc. Psychol. 44(5), 1368–1375 (2008)

    Google Scholar 

  43. Vishwanath, A., Herath, T., Chen, R., Wang, J., Rao, H.R.: Why do people get phished? Testing individual differences in phishing vulnerability within an integrated, information processing model. Decis. Support Syst. 51(3), 576–586 (2011)

    Google Scholar 

  44. Wang, J., Herath, T., Chen, R., Vishwanath, A., Rao, H.R.: Phishing susceptibility: an investigation into the processing of a targeted spear phishing email. IEEE Trans. Prof. Commun. 55(4), 345–362 (2012)

    Google Scholar 

  45. Wang, J., Li, Y., Rao, H.R.: Overconfidence in phishing email detection. J. Assoc. Inf. Syst. 17(11), 759–783 (2016)

    Google Scholar 

  46. Wang, J., Li, Y., Rao, H.R.: Coping responses in phishing detection: an investigation of antecedents and consequences. Inf. Syst. Res. 28(2), 378–396 (2017)

    Google Scholar 

  47. Weber, N., Brewer, N.: Confidence-accuracy calibration in absolute and relative face recognition judgments. J. Exp. Psychol. 10(3), 156–172 (2004)

    Google Scholar 

  48. Wright, R.T., Jensen, M.L., Thatcher, J.B., Dinger, M., Marett, K.: Influence techniques in phishing attacks: an examination of vulnerability and resistance. Inf. Syst. Res. 25(2), 385–400 (2014)

    Google Scholar 

  49. Wright, R.T., Marett, K.: The influence of experiential and dispositional factors in phishing: an empirical investigation of the deceived. J. Manag. Inf. Syst. 27(1), 273–303 (2010)

    Google Scholar 

  50. Wu, M., Miller, R.C., Garfinkel, S.L.: Do security toolbars actually prevent phishing attacks? In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems ACM, pp. 601–610. Quebec, Canada (2006)

    Google Scholar 

  51. Xiao, B., Benbasat, I.: Designing warning messages for detecting biased online product recommendations: an empirical investigation. Inf. Syst. Res. 26(4), 793–811 (2015)

    Google Scholar 

  52. Yeung, N., Summerfield, C.: Metacognition in human decision-making: confidence and error monitoring. Philos. Trans. Royal Soc. Biol. Sci. 367(1594), 1310–1321 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Tennessee at Knoxville, Knoxville, TN, 37996, USA

    Yuan Li

  2. University of Texas at Arlington, Arlington, TX, 76019, USA

    Jingguo Wang

  3. University of Texas at San Antonio, San Antonio, TX, 78249, USA

    H. Raghav Rao

Authors
  1. Yuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingguo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. Raghav Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuan Li .

Editor information

Editors and Affiliations

  1. The University of Texas at San Antonio, San Antonio, TX, USA

    Ram Krishnan

  2. The University of Texas at San Antonio, San Antonio, TX, USA

    H. Raghav Rao

  3. Birla Institute of Technology and Science, Pilani, Rajasthan, India

    Sanjay K. Sahay

  4. Indiana University, Bloomington, IN, USA

    Sagar Samtani

  5. SUNY Buffalo, Buffalo, NY, USA

    Ziming Zhao

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, Y., Wang, J., Rao, H.R. (2022). Metacognitive Skills in Phishing Email Detection: A Study of Calibration and Resolution. In: Krishnan, R., Rao, H.R., Sahay, S.K., Samtani, S., Zhao, Z. (eds) Secure Knowledge Management In The Artificial Intelligence Era. SKM 2021. Communications in Computer and Information Science, vol 1549. Springer, Cham. https://doi.org/10.1007/978-3-030-97532-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-97532-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-97531-9

  • Online ISBN: 978-3-030-97532-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation