Skip to main content
SpringerLink
Log in

Lessons Learned and Suitability of Focus Groups in Security Information Workers Research

  • Conference paper
  • First Online:
HCI for Cybersecurity, Privacy and Trust (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13333))

Included in the following conference series:

  • 1117 Accesses

Abstract

Security information workers (SIW) are professionals who develop and use security-related data within their jobs. Qualitative methods – primarily interviews – are becoming increasingly popular in SIW research. However, focus groups are an under-utilized, but potentially valuable way to explore the work practices, needs, and challenges of these professionals. Based on our experience with virtual focus groups of security awareness professionals, this paper documents lessons learned and the suitability of using focus groups to study SIW. We also suggest ways to alleviate concerns SIW may have with focus group participation. These insights may be helpful to other researchers embarking on SIW research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The term “security information worker” does not describe a formalized cybersecurity work role (e.g., like those described in the National Initiative for Cybersecurity Education Workforce Framework for Cybersecurity [24]), but rather encompasses a range of professionals handling security information.

References

  1. 7th Workshop on Security Information Workers. https://security-information-workers.org/ (2021)

  2. Acar, Y., Stransky, C., Wermke, D., Mazurek, M.L., Fahl, S.: Security developer studies with Github users: Exploring a convenience sample. In: Proceedings of the 13th Symposium on Usable Privacy and Security (SOUPS 2017). pp. 81–95 (2017)

    Google Scholar 

  3. Bada, M., Sasse, A.M., Nurse, J.R.: Cyber security awareness campaigns: Why do they fail to change behaviour? (2019). https://arxiv.org/ftp/arxiv/papers/1901/1901.02672.pdf

  4. Bada, M., Solms, B.V., Agrafiotis, I.: Reviewing national cybersecurity awareness in Africa: An empirical study (2019)

    Google Scholar 

  5. Botta, D., Werlinger, R., Gagné, A., Beznosov, K., Iverson, L., Fels, S., Fisher, B.: Studying IT security professionals: Research design and lessons learned (2007)

    Google Scholar 

  6. Corbin, J., Strauss, A.: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, 4th edn. Sage Publications, Thousand Oaks (2015)

    Google Scholar 

  7. Cyr, J.: The unique utility of focus groups for mixed-methods research. Polit. Sci. Politics 50(4), 1038 (2017)

    Google Scholar 

  8. David, D.P., Keupp, M.M., Mermoud, A.: Knowledge absorption for cyber-security: The role of human beliefs. Comput. Hum. Behav. 106, 106255 (2020)

    Google Scholar 

  9. Dykstra, J., Paul, C.L.: Cyber operations stress survey (COSS): Studying fatigue, frustration, and cognitive workload in cybersecurity operations. In: 11th USENIX Workshop on Cyber Security Experimentation and Test (CSET 18) (2018)

    Google Scholar 

  10. Fujs, D., Mihelic̆, A., Vrhovec, S.L.: The power of interpretation: Qualitative methods in cybersecurity research. In: Proceedings of the 14th International Conference on Availability, Reliability and Security, pp. 1–10 (2019)

    Google Scholar 

  11. Galloway, K.L.: Focus groups in the virtual world: implications for the future of evaluation. New Dir. Eval. 131(2011), 47–51 (2011)

    Google Scholar 

  12. Goodall, J.R., Lutters, W.G., Komlodi, A.: I know my network: collaboration and expertise in intrusion detection. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, pp. 342–345 (2004)

    Google Scholar 

  13. Gorski, P., Leo, P., Acar, Y., Iacono, L.L., Fahl, S.: Listen to developers! A participatory design study on security warnings for cryptographic APIs. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2020)

    Google Scholar 

  14. Guest, G., Namey, E., McKenna, K.: How many focus groups are enough? Building an evidence base for nonprobability sample sizes. Field Methods 29(1), 3–22, 106255 (2017)

    Google Scholar 

  15. Krueger, R.A., Casey, M.A.: Focus Groups: A Practical Guide for Applied Research. Sage, Thousand Oaks (2015)

    Google Scholar 

  16. Kumar, P.C., Chetty, M., Clegg, T.L., Vitak, J.: Privacy and security considerations for digital technology use in elementary schools. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2019)

    Google Scholar 

  17. Malhotra, A., Majchrzak, A., Rosen, B.: Leading virtual teams. Acad. Manage. Perspect. 21(1), 60–70 (2007)

    Google Scholar 

  18. Mathew, A., Cheshire, C.: Risky business: Social trust and community in the practice of cybersecurity for internet infrastructure. In: Proceedings of the 50th Hawaii International Conference on System Sciences, pp. 2341–2350 (2017)

    Google Scholar 

  19. Mermoud, A., Keupp, M.M., Huguenin, K., Palmié, M., David, D.P.: To share or not to share: A behavioral perspective on human participation in security information sharing. J. Cybersecurity 5(1) (2019)

    Google Scholar 

  20. Nassar-McMillan, S.C., Borders, L.D.: Use of focus groups in survey item development. Qual. Rep. 7(1), 1–12, 106255 (2002)

    Google Scholar 

  21. National Institute of Standards and Technology: FISSEA - Federal Information Security Educators (2021). https://csrc.nist.gov/projects/fissea

  22. O’Brien, K.: Using focus groups to develop health surveys: An example from research on social relationships and AIDS-preventive behavior. Health Educ. Q. 20(3), 361–372, 106255 (1993)

    Google Scholar 

  23. Paul, C.L.: Human-centered study of a network operations center: Experience report and lessons learned. In: Proceedings of the 2014 ACM Workshop on Security Information Workers, pp. 39–42 (2014)

    Google Scholar 

  24. Petersen, R., Santos, D., Smith, M.C., Wetzel, K.A., Witte, G.: NIST Special Publication 800–181 Revision 1: Workforce Framework for Cybersecurity (NICE Framework) (2020). https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-181r1.pdf

  25. SANS: 2021 SANS security awareness report: Managing human cyber risk (2021). https://www.sans.org/security-awareness-training/resources/reports/sareport-2021/

  26. Schneier, B.: The security mindset (2008). https://www.schneier.com/blog/archives/2008/03/the_security_mi_1.html

  27. Sim, J.: Collecting and analysing qualitative data: Issues raised by the focus group. J. Adv. Nurs. 28(2), 345–352, 106255 (1998)

    Google Scholar 

  28. Smith, E., Loftin, R., Murphy-Hill, E., Bird, C., Zimmermann, T.: Improving developer participation rates in surveys. In: Proceedings of the 6th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), pp. 89–92 (2013)

    Google Scholar 

  29. Stewart, D.W., Shamdasani, P.N.: Focus Groups: Theory and Practice, vol. 20. Sage, Thousand Oaks (2014)

    Google Scholar 

  30. Sundaramurthy, S.C., McHugh, J., Ou, X.S., Rajagopalan, S.R., Wesch, M.: An anthropological approach to studying CSIRTs. IEEE Secur. Priv. 12(5), 52–60, 106255 (2014)

    Google Scholar 

  31. The State of Security: The security mindset: the key to success in the security field, November 2015. https://www.tripwire.com/state-of-security/off-topic/the-security-mindset-the-key-to-success-in-the-security-field/

  32. U.S. Bureau of Labor Statistics: Information security analysts (2021). https://www.bls.gov/ooh/computer-and-information-technology/information-security-analysts.htm

  33. U.S. Bureau of Labor Statistics: Software developers, quality assurance analysts, and testers (2021). https://www.bls.gov/ooh/computer-and-information-technology/software-developers.htm

  34. UX Alliance: Conducting remote online focus groups in times of COVID-19, April 2020. https://medium.com/@UXalliance/conducting-remote-online-focus-groups-in-times-of-covid-19-ee1c66644fdb

  35. Wilson, M., Hash, J.: NIST Special Publication 800–50 - Building an information technology security awareness program (2003). https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-50.pdf

  36. Witschey, J., Murphy-Hill, E., Xiao, S.: Conducting interview studies: Challenges, lessons learned, and open questions. In: Proceedings of the 1st International Workshop on Conducting Empirical Studies in Industry (CESI), pp. 51–54 (2013)

    Google Scholar 

  37. Woelk, B.: The successful security awareness professional: Foundational skills and continuing education strategies (2015). https://library.educause.edu/~/media/files/library/2016/8/erb1608.pdf

Download references

Author information

Authors and Affiliations

  1. National Institute of Standards and Technology, Gaithersburg, MD, 20899, USA

    Julie M. Haney, Jody L. Jacobs, Fernando Barrientos & Susanne M. Furman

Authors
  1. Julie M. Haney
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jody L. Jacobs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fernando Barrientos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Susanne M. Furman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julie M. Haney .

Editor information

Editors and Affiliations

  1. San Jose State University, San Jose, CA, USA

    Abbas Moallem

Appendix A Focus Group Script

Appendix A Focus Group Script

Moderator Introduction and Ground Rules

Welcome to our focus group! I’d like to start off by thanking each of you for taking time to participate today. We’ll be here for about [insert time] at most. It may be less than that, but we want to allow plenty of time for discussion.

I’m going to lead our discussion today. I will be asking you questions and then moderating our discussion. [Research team members] are part of the research team and will be assisting me by taking notes and jumping in with follow-up questions when appropriate.

I’d like to go over a few items that will allow our conversation to flow more freely. [Share PowerPoint presentation that summarizes ground rules.]

  1. 1.

    This is a confidential discussion without fear of reprisal or comments being taken out of context. We told you how we are going to protect your confidentiality, and we ask the same of you with respect to others in the group here today.

  2. 2.

    If you don’t understand a question or need clarification, please ask.

  3. 3.

    You don’t have to answer every question, but we’d like to hear from each of you today as the discussion progresses. There are no “wrong answers,” just different opinions and experiences.

  4. 4.

    We’ll do our best with turn-taking. Unmute and jump in or click the “raise hand” icon next to your name in the Participants panel.

  5. 5.

    When not talking, please mute yourself to cut down on background noise and feedback.

  6. 6.

    Turning on your camera is optional but can help with conversational cues, but there’s no pressure to turn it on.

  7. 7.

    Chat is available if you’d like to share a link or resource with the group or have any technical issues. But if you’d like to say something that contributes directly to the conversation, please say it out loud so that we can capture it on the recording.

Introduction of Participants

Opening question: First, we’ll do some introductions. These will NOT be recorded. I’ll go around to each of you. Please tell everyone your name, organization, and your role with respect to security awareness.

Focus Group Questions

I’m now going to start recording this session. [Advance through slides for each question.]

  1. 1.

    Introductory question: When I say “security awareness and training,” what does that mean to you? What comes to mind?

  2. 2.

    Transition question: Tell me about your organization’s approach to security awareness and training. This can include general security awareness for the workforce as well as awareness for specialized job roles.

  3. 3.

    Key question: How do you decide what topics and approaches to use for your security awareness program?

    1. (a)

      [Probe for sub-components] What kind of guidance/direction, if any, does your department provide? How much leeway do you have to tailor the training to your own organization?

    2. (b)

      [Probe for department-level agencies] What kind of guidance/direction, if any, do you push down to sub-components within your department?

  4. 4.

    Key question: What’s working well with your program?

  5. 5.

    Key question: What’s not working as well and why? What are your challenges and concerns with respect to security awareness in your organization?

  6. 6.

    Key question: How do you determine the effectiveness of your program, if at all?

  7. 7.

    Key question: If you could have anything or do anything for your security awareness program, what would that be?

    1. (a)

      [Probe] What would you do to solve the challenges you currently experience?

    2. (b)

      [Probe] What kinds and formats of resources and information sharing would be most beneficial?

  8. 8.

    Key question: What knowledge, skills, or competencies do you think are needed for those performing security awareness functions in your organization?

  9. 9.

    Ending question: If you had one or two pieces of advice for someone just starting a security awareness program in an agency like yours, what would that advice be?

  10. 10.

    Ending question: Recall that the purpose of our study is to better understand the needs, challenges, practices, and professional competencies of federal security awareness teams and programs. This understanding will lead to the creation of resources for federal security awareness professionals.

  11. 11.

    Ending question: Is there anything else that we should have talked about, but didn’t?

Closing

I will now end the recording. That concludes our focus group. Thanks for attending and talking about these issues. Your comments have been very insightful.

Just a few reminders. If you want something that you said removed from the research record, please let us know. Also, if you think of anything else you didn’t get a chance to talk about, feel free to email us.

We really appreciate your participation and thank you again for your time. Have a wonderful day!

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Haney, J.M., Jacobs, J.L., Barrientos, F., Furman, S.M. (2022). Lessons Learned and Suitability of Focus Groups in Security Information Workers Research. In: Moallem, A. (eds) HCI for Cybersecurity, Privacy and Trust. HCII 2022. Lecture Notes in Computer Science, vol 13333. Springer, Cham. https://doi.org/10.1007/978-3-031-05563-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05563-8_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05562-1

  • Online ISBN: 978-3-031-05563-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation