Abstract
Cybersecurity Metrics and Quantification is a fundamental but notoriously hard problem and is undoubtedly one of the pillars underlying the emerging Science of Cybersecurity. In this paper, we present an novel approach to addressing this problem by unifying Security, Agility, Resilience and Risk (SARR) metrics into a single framework. The SARR approach and the resulting framework are unique because: (i) it is driven by the assumptions that are made when modeling, designing, implementing, operating, and defending systems, which are broadly defined to include infrastructures and enterprise networks; and (ii) it embraces the uncertainty inherent to the cybersecurity domain. We will review the status quo by looking into existing metrics and quantification research through the SARR lens and discuss a range of open problems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Charlton, J., Du, P., Cho, J., Xu, S.: Measuring relative accuracy of malware detectors in the absence of ground truth. In: Proceedings of IEEE MILCOM, pp. 450–455 (2018)
Charlton, J., Du, P., Xu, S.: A new method for inferring ground-truth labels. In: Proceedings of SciSec (2021)
Chen, H., Cho, J., Xu, S.: Quantifying the security effectiveness of firewalls and DMZs. In: Proceedings of HoTSoS 2018, pp. 9:1–9:11 (2018)
Chen, H., Cho, J., Xu, S.: Quantifying the security effectiveness of network diversity. In: Proceedings of HoTSoS 2018, p. 24:1 (2018)
Chen, Y., Huang, Z., Xu, S., Lai, Y.: Spatiotemporal patterns and predictability of cyberattacks. PLoS ONE 10(5), e0124472 (2015)
Cheng, Y., Deng, J., Li, J., DeLoach, S., Singhal, A., Ou, X.: Metrics of security. In: Cyber Defense and Situational Awareness, pp. 263–295 (2014)
Cho, J., Hurley, P., Xu, S.: Metrics and measurement of trustworthy systems. In: Proceedings IEEE MILCOM (2016)
Cho, J., Xu, S., Hurley, P., Mackay, M., Benjamin, T., Beaumont, M.: STRAM: measuring the trustworthiness of computer-based systems. ACM Comput. Surv. 51(6), 128:1–128:47 (2019)
National Research Council: Review of the Department of Homeland Security’s Approach to Risk Analysis. The National Academies Press (2010)
INFOSEC Research Council. Hard problem list. http://www.infosec-research.org/docs_public/20051130-IRC-HPL-FINAL.pdf (2007)
Da, G., Xu, M., Xu, S.: A new approach to modeling and analyzing security of networked systems. In: Proceedings HotSoS 2014, pp. 6:1–6:12 (2014)
Dai, W., Parker, P., Jin, H., Xu, S.: Enhancing data trustworthiness via assured digital signing. IEEE TDSC 9(6), 838–851 (2012)
Du, P., Sun, Z., Chen, H., Cho, J.H., Xu, S.: Statistical estimation of malware detection metrics in the absence of ground truth. IEEE T-IFS 13(12), 2965–2980 (2018)
Durumeric, Z., et al.: The matter of heartbleed. In: Proceedings IMC (2014)
Fang, Z., Xu, M., Xu, S., Hu, T.: A framework for predicting data breach risk: leveraging dependence to cope with sparsity. IEEE T-IFS 16, 2186–2201 (2021)
Goldreich, O.: The Foundations of Cryptography, vol. 1. Cambridge University Press (2001)
Haimes, Y.Y.: On the definition of resilience in systems. Risk Anal. 29(4), 498–501 (2009)
Han, Y., Lu, W., Xu, S.: Characterizing the power of moving target defense via cyber epidemic dynamics. In: HotSoS, pp. 1–12 (2014)
Han, Y., Lu, W., Xu, S.: Preventive and reactive cyber defense dynamics with ergodic time-dependent parameters is globally attractive. IEEE TNSE, accepted for publication (2021)
Harrison, K., Xu, S.: Protecting cryptographic keys from memory disclosures. In: IEEE/IFIP DSN 2007, pp. 137–143 (2007)
Homer, J., et al.: Aggregating vulnerability metrics in enterprise networks using attack graphs. J. Comput. Secur. 21(4), 561–597 (2013)
Jensen, U.: Probabilistic risk analysis: foundations and methods. J. Am. Stat. Assoc. 97(459), 925 (2002)
Kantchelian, A., et al.: Better malware ground truth: techniques for weighting anti-virus vendor labels. In: Proceedings AISec, pp. 45–56 (2015)
Li, D., Li, Q., Ye, Y., Xu, S.: SoK: arms race in adversarial malware detection. CoRR, abs/2005.11671 (2020)
Li, D., Li, Q., Ye, Y., Xu, S.: A framework for enhancing deep neural networks against adversarial malware. IEEE TNSE 8(1), 736–750 (2021)
Li, X., Parker, P., Xu, S.: A stochastic model for quantitative security analyses of networked systems. IEEE TDSC 8(1), 28–43 (2011)
Lin, Z., Lu, W., Xu, S.: Unified preventive and reactive cyber defense dynamics is still globally convergent. IEEE/ACM ToN 27(3), 1098–1111 (2019)
Lu, W., Xu, S., Yi, X.: Optimizing active cyber defense dynamics. In: Proceedings GameSec 2013, pp. 206–225 (2013)
Lynch, N.: Distributed Algorithms. Morgan Kaufmann (1996)
Mireles, J., Ficke, E., Cho, J., Hurley, P., Xu, S.: Metrics towards measuring cyber agility. IEEE T-IFS 14(12), 3217–3232 (2019)
Morales, J., Xu, S., Sandhu, R.: Analyzing malware detection efficiency with multiple anti-malware programs. In: Proceedings CyberSecurity (2012)
Nicol, D., et al.: The science of security 5 hard problems, August 2015. http://cps-vo.org/node/21590
Noel, S., Jajodia, S.: A suite of metrics for network attack graph analytics. In: Network Security Metrics, pp. 141–176. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66505-4_7
Park, J., Seager, T.P., Rao, P.S.C., Convertino, M., Linkov, I.: Integrating risk and resilience approaches to catastrophe management in engineering systems. Risk Anal. 33(3), 356–367 (2013)
Pendleton, M., Garcia-Lebron, R., Cho, J., Xu, S.: A survey on systems security metrics. ACM Comput. Surv. 49(4), 62:1–62:35 (2016)
Pfleeger, S.L., Cunningham, R.K.: Why measuring security is hard. IEEE Secur. Priv. 8(4), 46–54 (2010)
Ramos, A., Lazar, M., Filho, R.H., Rodrigues, J.J.P.C.: Model-based quantitative network security metrics: a survey. IEEE Commun. Surv. Tutor. 19(4), 2704–2734 (2017)
National Science and Technology Council: Trustworthy cyberspace: strategic plan for the federal cybersecurity research and development program (2011). https://www.nitrd.gov/SUBCOMMITTEE/csia/Fed_Cybersecurity_RD_Strategic_Plan_2011.pdf
Wang, L., Jajodia, S., Singhal, A.: Network Security Metrics. Network Security Metrics, Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66505-4
Wang, L., Jajodia, S., Singhal, A., Cheng, P., Noel, S.: k-zero day safety: a network security metric for measuring the risk of unknown vulnerabilities. IEEE TDSC 11(1), 30–44 (2014)
Xu, L., et al.: KCRS: a blockchain-based key compromise resilient signature system. In: Proceedings BlockSys, pp. 226–239 (2019)
Xu, M., Da, G., Xu, S.: Cyber epidemic models with dependences. Internet Math. 11(1), 62–92 (2015)
Xu, M., Hua, L., Xu, S.: A vine copula model for predicting the effectiveness of cyber defense early-warning. Technometrics 59(4), 508–520 (2017)
Xu, M., Schweitzer, K.M., Bateman, R.M., Xu, S.: Modeling and predicting cyber hacking breaches. IEEE T-IFS 13(11), 2856–2871 (2018)
Xu, M., Xu, S.: An extended stochastic model for quantitative security analysis of networked systems. Internet Math. 8(3), 288–320 (2012)
Xu, S.: Emergent behavior in cybersecurity. In: Proceedings HotSoS, pp. 13:1–13:2 (2014)
Xu, S.: Cybersecurity dynamics: a foundation for the science of cybersecurity. In: Proactive and Dynamic Network Defense, pp. 1–31 (2019)
Xu, S.: The cybersecurity dynamics way of thinking and landscape (invited paper). In: ACM Workshop on Moving Target Defense (2020)
Xu, S., Lu, W., Xu, L.: Push- and pull-based epidemic spreading in networks: thresholds and deeper insights. ACM TAAS 7(3), 1–26 (2012)
Xu, S., Lu, W., Xu, L., Zhan, Z.: Adaptive epidemic dynamics in networks: thresholds and control. ACM TAAS 8(4), 1–19 (2014)
Xu, S., Lu, W., Zhan, Z.: A stochastic model of multivirus dynamics. IEEE Trans. Dependable Secure Comput. 9(1), 30–45 (2012)
Xu, S., Yung, M.: Expecting the unexpected: towards robust credential infrastructure. In: Financial Crypto, pp. 201–221 (2009)
Xu, S.: Cybersecurity dynamics. In: Proceedings HotSoS 2014, pp. 14:1–14:2 (2014)
Shouhuai, X., Wenlian, L., Li, H.: A stochastic model of active cyber defense dynamics. Internet Math. 11(1), 23–61 (2015)
Xu, S., Trivedi, K.: Report of the 2019 SATC pi meeting break-out session on “cybersecurity metrics: Why is it so hard?” (2019)
Shouhuai, X., Yung, M., Wang, J.: Seeking foundations for the science of cyber security. Inf. Syst. Front. 23, 263–267 (2021)
Zhan, Z., Xu, M., Xu, S.: Characterizing honeypot-captured cyber attacks: statistical framework and case study. IEEE T-IFS 8(11), 1775–1789 (2013)
Zhan, Z., Maochao, X., Shouhuai, X.: Predicting cyber attack rates with extreme values. IEEE T-IFS 10(8), 1666–1677 (2015)
Zhang, M., Wang, L., Jajodia, S., Singhal, A., Albanese, M.: Network diversity: a security metric for evaluating the resilience of networks against zero-day attacks. IEEE Trans. Inf. Forensics Secur. 11(5), 1071–1086 (2016)
Zheng, R., Lu, W., Xu, S.: Active cyber defense dynamics exhibiting rich phenomena. In: Proceedings HotSoS (2015)
Zheng, R., Lu, W., Xu, S.: Preventive and reactive cyber defense dynamics is globally stable. IEEE TNSE 5(2), 156–170 (2018)
Acknowledgement
We thank Moti Yung for illuminating discussions and Eric Ficke for proofreading the paper. This work was supported in part by ARO Grant #W911NF-17-1-0566, NSF Grants #2115134 and #2122631 (#1814825), and by a Grant from the State of Colorado.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, S. (2021). SARR: A Cybersecurity Metrics and Quantification Framework (Keynote). In: Lu, W., Sun, K., Yung, M., Liu, F. (eds) Science of Cyber Security. SciSec 2021. Lecture Notes in Computer Science(), vol 13005. Springer, Cham. https://doi.org/10.1007/978-3-030-89137-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-89137-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89136-7
Online ISBN: 978-3-030-89137-4
eBook Packages: Computer ScienceComputer Science (R0)