Skip to main content
SpringerLink
Log in

Studying Analysts’ Data Triage Operations in Cyber Defense Situational Analysis

  • Chapter
  • First Online:
Theory and Models for Cyber Situation Awareness

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10030))

Abstract

Cyber defense analysts are playing a critical role in Security Operations Centers (SOCs) to make sense of the immense amount of network monitoring data for detecting and responding to cyber attacks, including large-scale cyber attack campaigns involving advanced persistent threats. The network data continuously generated by multiple cyber defense systems, which may contain many false alerts, are overwhelming to the analysts. Analysts often need to make quick decisions/responses in a very short time based on their awareness of the situation at that moment. Data triage is the first and the most fundamental step performed routinely by the analysts — it filters a massive network monitoring data to identify known malicious events. Due to the high noise-to-signal ratio of network monitoring data, this steps accounts for a very significant portion of the time and attention of intrusion detection analysts. Therefore, a smart human-machine system that improves the performance of data triage operation in SOC is highly desirable. In this chapter, we describe a human-centered smart data triage system that leverages the cognitive trace of intrusion detection analysts. Our approach is based on a dynamic cyber-human system that integrates three dimensions: cyber defense analysts, network monitoring data, and attack activities. The approach leverages recorded analytic processes of intrusion detection analysts, which we refer to as “cognitive traces”. These traces of the analysts capture the examples of malicious events detected from the network monitoring data. Such traces from senior analysts provide a powerful opportunity for training junior analysts in performing data triage operations. To realize this potential, we also developed a smart retrieval framework that automatically retrieves traces of other senior analysts based on their similarity to the events already identified by a junior analyst. The traces from analysts, as demonstrated by a case study, also enable us to better understand their analytic processes in a systematic, yet minimum-reactive way. We summarize this chapter by discussing limitations of the proposed framework and the directions of future research regarding improving the data triage operations of cyber defense analysts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Security Operations: Building a Successful SOC, Hewlett-Packard Development Company, hp.com/go/sioc (2013)

  2. D’Amico, A., Whitley, K.: The real work of computer network defense analysts. In: Goodall, J.R., Conti, G., Ma, K.-L. (eds.) VizSEC 2007, pp. 19–37. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  3. D’Amico, A., Whitley, K., Tesone, D., O’Brien, B., Roth, E.: Achieving cyber defense situational awareness: a cognitive task analysis of information assurance analysts. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 49, no. 3, pp. 229–233. SAGE Publications (2005)

    Google Scholar 

  4. Erbacher, R.F., Frincke, D.A., Wong, P.C., Moody, S., Fink, G.: A multi-phase network situational awareness cognitive task analysis. Inf. Vis. 9(3), 204–219 (2010)

    Article  Google Scholar 

  5. Granåsen, M., Dennis, A.: Measuring team effectiveness in cyber-defense exercises: a cross-disciplinary case study. Cogn. Technol. Work 18(1), 1–23 (2015)

    Google Scholar 

  6. Yen, J., Erbacher, R.F., Zhong, C., Liu, P.: Cognitive process. In: Kott, A., Wang, C., Erbacher, R.F. (eds.) Cyber Defense and Situational Awareness. AIS, vol. 62, pp. 119–144. Springer, Cham (2014). doi:10.1007/978-3-319-11391-3_7

    Google Scholar 

  7. Etoty, R.E., Erbacher, R.F.: A survey of visualization tools assessed for anomaly-based intrusion detection analysis. No. ARL-TR-6891. Army Research Lab Adelphi MD Computational and Information Sciences Directorate (2014)

    Google Scholar 

  8. Barford, P., et al.: Cyber SA: situational awareness for cyber defense. In: Jajodia, S., Liu, P., Swarup, V., Wang, C. (eds.) Cyber Situational Awareness, vol. 46, pp. 3–13. Springer, US (2010)

    Chapter  Google Scholar 

  9. Dutt, V., Ahn, Y.-S., Gonzalez, C.: Cyber situation awareness: modeling the security analyst in a cyber-attack scenario through instance-based learning. In: Li, Y. (ed.) DBSec 2011. LNCS, vol. 6818, pp. 280–292. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22348-8_24

    Chapter  Google Scholar 

  10. Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Hum. Factors J. Hum. Factors Ergon. Soc. 37(1), 32–64 (1995)

    Article  Google Scholar 

  11. Boyd, J.R.: The Essence of Winning and Losing (1996). Unpublished lecture notes

    Google Scholar 

  12. Pirolli, P., Card, S.: The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proceedings of International Conference on Intelligence Analysis, vol. 5, pp. 2–4 (2005)

    Google Scholar 

  13. Bass, T.: Intrusion detection systems and multisensor data fusion. Commun. ACM 43(4), 99–105 (2000)

    Article  Google Scholar 

  14. Mahmood, T., Afzal, U.: Security analytics: Big Data analytics for cybersecurity: a review of trends, techniques and tools. In: 2nd National Conference on Information Assurance (NCIA), pp. 129–134. IEEE (2013)

    Google Scholar 

  15. Zuech, R., Khoshgoftaar, T.M., Wald, R.: Intrusion detection and big heterogeneous data: a survey. J. Big Data 2(1), 1–41 (2015)

    Article  Google Scholar 

  16. Biros, D.P., Eppich, T.: THEME: security-human element key to intrusion detection. Signal-Fairfax 55(12), 31–34 (2001)

    Google Scholar 

  17. Ericsson, K.A., Lehmann, A.C.: Expert and exceptional performance: evidence of maximal adaptation to task constraints. Annu. Rev. Psychol. 47(1), 273–305 (1996)

    Article  Google Scholar 

  18. Chen, P.C., Liu, P., Yen, J., Mullen, T.: Experience-based cyber situation recognition using relaxable logic patterns. In: IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 243–250. IEEE (2012)

    Google Scholar 

  19. Grance, T., Kent, K., Kim, B.: Computer security incident handling guide. NIST Spec. Publ. 800, 61 (2004)

    Google Scholar 

  20. Information Security: Agencies Need to Improve Cyber Incident Response Practices. GAO-14-354, 30 April 2014. Publicly Released: May 30, 2014

    Google Scholar 

  21. Freiling, F.C., Schwittay, B.: A common process model for incident response and computer forensics. IMF 7, 19–40 (2007)

    Google Scholar 

  22. Prosise, C., Mandia, K., Pepe, M.: Incident Response & Computer Forensics. McGraw-Hill/Osborne, New York (2003)

    Google Scholar 

  23. Dawkins, J., Hale, J.: A systematic approach to multi-stage network attack analysis. In: Second IEEE International Information Assurance Workshop, Proceedings, pp. 48–56. IEEE (2004)

    Google Scholar 

  24. Jha, S., Sheyner, O., Jeannette, M.W.: Minimization and reliability analyses of attack graphs. No. CMU-CS-02-109. Carnegie-Mellon Univ. Pittsburgh PA School of Computer Science (2002)

    Google Scholar 

  25. Thomas, J.J., Cook, K.A.: The science of analytical reasoning. In: Illuminating the Path: The Research and Development Agenda for Visual Analytics, pp. 32–68 (2005)

    Google Scholar 

  26. Mancuso, V.F., Minotra, D., Giacobe, N., McNeese, M., Tyworth, M.: idsNETS: an experimental platform to study situation awareness for intrusion detection analysts. In: IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 73–79. IEEE (2012)

    Google Scholar 

  27. Giacobe, N.A.: Measuring the effectiveness of visual analytics and data fusion techniques on situation awareness in cyber-security. PhD diss., The Pennsylvania State University (2013)

    Google Scholar 

  28. Poling, A., Methot, L.L., LeSage, M.G.: Fundamentals of Behavior Analytic Research. Springer Science & Business Media, US (2013)

    Google Scholar 

  29. Lee, F.J., Anderson, J.R.: Does learning a complex task have to be complex? A study in learning decomposition. Cogn. Psychol. 42(3), 267–316 (2001)

    Article  Google Scholar 

  30. Kukreja, U., Stevenson, W.E., Ritter, F.E.: RUI: recording user input from interfaces under Windows and Mac OS X. Behav. Res. Methods 38(4), 656–659 (2006)

    Article  Google Scholar 

  31. Allopenna, P.D., Magnuson, J.S., Tanenhaus, M.K.: Tracking the time course of spoken word recognition using eye movements: evidence for continuous mapping models. J. Mem. Lang. 38(4), 419–439 (1998)

    Article  Google Scholar 

  32. Rabinovich, M.I., Huerta, R., Varona, P., Afraimovich, V.S.: Transient cognitive dynamics, metastability, and decision making. PLoS Comput. Biol. 4(5), e1000072 (2008)

    Article  MathSciNet  Google Scholar 

  33. Tom, P., Santtila, P., Bosco, D.: The ability of human judges to link crimes using behavioral information: current knowledge and unresolved issues. In: Crime Linkage: Theory, Research, and Practice. CRC Press, p. 268 (2014)

    Google Scholar 

  34. Zhong, C., Samuel, D., Yen, J., Liu, P., Erbacher, R., Hutchinson, S., Etoty, R., Cam, H., Glodek, W.: RankAOH: context-driven similarity-based retrieval of experiences in cyber analysis. In: IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 230–236. IEEE (2014)

    Google Scholar 

  35. Zhong, C., Yen, J., Liu, P., Erbacher, R., Etoty, R., Garneau, C.: An integrated computer-aided cognitive task analysis method for tracing cyber-attack analysis processes. In: Proceedings of the 2015 Symposium and Bootcamp on the Science of Security, p. 9. ACM (2015)

    Google Scholar 

  36. Pirolli, P.: Information Foraging Theory: Adaptive Interaction with Information. Oxford University Press (2007)

    Google Scholar 

  37. Pirolli, P., Card, S.: Information foraging. Psychol. Rev. 106(4), 643 (1999)

    Article  Google Scholar 

  38. Zhong, C., Yen, J., Liu, P., Erbacher, R., Etoty, R., Garneau, C.: ARSCA: a computer tool for tracing the cognitive processes of cyber-attack analysis. In: IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 165–171. IEEE (2015)

    Google Scholar 

  39. “VAST Challenge 2012 Mini-Challenge 2”, Visual Analytics Community (2012)

    Google Scholar 

  40. Scholtz, J., Whiting, M.A., Plaisant, C., Grinstein, G.: A reflection on seven years of the VAST challenge. In: Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization, p. 13. ACM (2012)

    Google Scholar 

  41. Bass, T.: Multisensor data fusion for next generation distributed intrusion detection systems, pp. 24–27 (1999)

    Google Scholar 

  42. Lan, F., Chunlei, W., Guoqing, M.: A framework for network security situation awareness based on knowledge discovery. In: 2nd international conference on Computer Engineering and Technology (ICCET), vol. 1, pp. V1–226. IEEE (2010)

    Google Scholar 

  43. Fink, G.A., North, C.L., Endert, A., Rose, S.: Visualizing cyber security: usable workspaces. In: 6th International Workshop on Visualization for Cyber Security, VizSec 2009, pp. 45–56. IEEE (2009)

    Google Scholar 

  44. McClain, J., Silva, A., Emmanuel, G., Anderson, B., Nauer, K., Abbott, R., Forsythe, C.: Human Performance Factors in Cyber Security Forensic Analysis (2015)

    Google Scholar 

  45. Zhong, C., Kirubakaran, D.S., Yen, J., Liu, P., Hutchinson, S., Cam, H.: How to use experience in cyber analysis: an analytical reasoning support system. In: IEEE International Conference on Intelligence and Security Informatics (ISI), pp. 263–265. IEEE (2013)

    Google Scholar 

  46. Giacobe, N.A.: Application of the JDL data fusion process model for cyber security. In: SPIE Defense, Security, and Sensing, p. 77100R. International Society for Optics and Photonics (2010)

    Google Scholar 

  47. Yang, S.J., Stotz, A., Holsopple, J., Sudit, M., Kuhl, M.: High level information fusion for tracking and projection of multistage cyber attacks. Inf. Fusion 10(1), 107–121 (2009)

    Article  Google Scholar 

  48. Vandenberghe, G.: Visually assessing possible courses of action for a computer network incursion. In: SANS Institute, InfoSec Reading Room (2007)

    Google Scholar 

  49. Aamodt, A., Plaza, E.: Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Commun. 7(1), 39–59 (1994)

    Google Scholar 

  50. Cockburn, A., Karlson, A., Bederson, B.B.: A review of overview+detail, zooming, and focus+context interfaces. ACM Comput. Surv. (CSUR) 41(1), 2 (2009)

    Google Scholar 

Download references

Acknowledgements

This work was supported by ARO W911NF-09-1-0525 (MURI), ARO W911NF-15-1-0576, NSF CNS-1422594, and NIETP CAE Cybersecurity Grant (BAA-003-15).

Author information

Authors and Affiliations

  1. College of Information Sciences and Technology, Pennsylvania State University, State College, USA

    John Yen & Peng Liu

  2. Army Research Lab, Adelphi, USA

    Rob F. Erbacher & Christopher Garneau

  3. Indiana University Kokomo, Kokomo, USA

    Chen Zhong

  4. University of Memphis, Memphis, USA

    Bo Chen

Authors
  1. Chen Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Yen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rob F. Erbacher
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christopher Garneau
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bo Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chen Zhong .

Editor information

Editors and Affiliations

  1. Pennsylvania State University, University Park, Pennsylvania, USA

    Peng Liu

  2. George Mason University, Fairfax, Virginia, USA

    Sushil Jajodia

  3. Army Research Office, Research Triangle Park, North Carolina, USA

    Cliff Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Zhong, C., Yen, J., Liu, P., Erbacher, R.F., Garneau, C., Chen, B. (2017). Studying Analysts’ Data Triage Operations in Cyber Defense Situational Analysis. In: Liu, P., Jajodia, S., Wang, C. (eds) Theory and Models for Cyber Situation Awareness. Lecture Notes in Computer Science(), vol 10030. Springer, Cham. https://doi.org/10.1007/978-3-319-61152-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61152-5_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61151-8

  • Online ISBN: 978-3-319-61152-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation