Abstract
[Context] The CVSS framework provides several dimensions to score vulnerabilities. The environmental metrics allow security analysts to downgrade or upgrade vulnerability scores based on a company’s computing environments and security requirements. [Question] How difficult is for a human assessor to change the CVSS environmental score due to changes in security requirements (let alone technical configurations) for PCI-DSS compliance for networks and systems vulnerabilities of different type? [Results] A controlled experiment with 29 MSc students shows that given a segmented network it is significantly more difficult to apply the CVSS scoring guidelines on security requirements with respect to a flat network layout, both before and after the network has been changed to meet the PCI-DSS security requirements. The network configuration also impact the correctness of vulnerabilities assessment at system level but not at application level. [Contribution] This paper is the first attempt to empirically investigate the guidelines for the CVSS environmental metrics. We discuss theoretical and practical key aspects needed to move forward vulnerability assessments for large scale systems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Removed for anonymity.
References
Allodi, L., Massacci, F.: Comparing vulnerability severity and exploits using case-control studies. ACM Trans. Inf. Syst. Secur. 17(1), 1:1–1:20 (2014)
Beck, A., Rass, S.: Decision-support by aggregation and flexible visualization of risk situations. In: Proceedings of ECCWS 2016, p. 313. Academic Conferences and Publishing Limited (2016)
CVSS-SIG. Common vulnerability scoring system v3.0: Specification document. Technical report (2015). First.org
Frei, S., May, M., Fiedler, U., Plattner, B.: Large-scale vulnerability analysis. In: Proceedings of LSAD 2006, pp. 131–138. ACM (2006)
Gallon, L., Bascou, J.J.: Using cvss in attack graphs. In: Proceedings of ARES 2011, pp. 59–66. IEEE (2011)
Giacalone, M., Mammoliti, R., Massacci, F., Paci, F., Perugino, R., Selli, C.: Security triage: a report of a lean security requirements methodology for cost-effective security analysis. In: Proceedings of ACM/IEE ESEM 2014, pp. 25–27 (2014)
Hamid, T., MacDermott, Á.: A methodology to develop dynamic cost-centric risk impact metrics. In: Proceedings of DeSE 2015, pp. 53–59. IEEE (2015)
Holm, H., Afridi, K.K.: An expert-based investigation of the common vulnerability scoring system. Comput. Secur. 53, 18–30 (2015)
Holm, H., Ekstedt, M., Andersson, D.: Empirical analysis of system-level vulnerability metrics through actual attacks. IEEE Trans. Dependable Secur. Comput. 9(6), 825–837 (2012)
Höst, M., Regnell, B., Wohlin, C.: Using students as subjects-a comparative study of students and professionals in lead-time impact assessment. Empir. Soft. Eng. 5(3), 201–214 (2000)
Houmb, S.H., Franqueira, V.N., Engum, E.A.: Quantifying security risk level from cvss estimates of frequency and impact. J. Sys. Soft. 83(9), 1622–1634 (2010)
Liu, Q., Zhang, Y., Kong, Y., Wu, Q.: Improving VRSS-based vulnerability prioritization using analytic hierarchy process. J. Sys. Soft. 85(8), 1699–1708 (2012)
PCI. PCI (2010)
Pennington, R., Tuttle, B.: The effects of information overload on software project risk assessment. Decision Sci. 38(3), 489–526 (2007)
Quinn, S.D., Scarfone, K.A., Barrett, M., Johnson, C.S.: SP 800–117: Guide to adopting and using the security content automation protocol (SCAP) version 1.0. Technical report, NIST (2010)
Runeson, P.: Using students as experiment subjects-an analysis on graduate and freshmen student data. In: Proceedings of EASE 2003, pp. 95–102 (2003)
Singh, U.K., Joshi, C.: Quantitative security risk evaluation using CVSS metrics by estimation of frequency and maturity of exploit. In: Proceedings of the WCECS 2016, vol. 1, pp. 19–21 (2016)
Verizon. PCI compliance report. Technical report, Verizon Enterprise (2015)
Wang, L., Zhang, M., Jajodia, S., Singhal, A., Albanese, M.: Modeling network diversity for evaluating the robustness of networks against zero-day attacks. In: Kutyłowski, M., Vaidya, J. (eds.) ESORICS 2014. LNCS, vol. 8713, pp. 494–511. Springer, Cham (2014). doi:10.1007/978-3-319-11212-1_28
Wang, R., Gao, L., Sun, Q., Sun, D.: An improved CVSS-based vulnerability scoring mechanism. In: Proceedings of MINES 2011, pp. 352–355. IEEE (2011)
Wen, T., Zhang, Y., Dong, Y., Yang, G.: A novel automatic severity vulnerability assessment framework. J. Commun. 10(5) (2015)
Williams, B.R., Chuvakin, A.: PCI compliance: understand and implement effective PCI data security standard compliance. Syngress (2014)
Younis, A.A., Malaiya, Y.K.: Comparing and evaluating CVSS-based base metrics and microsoft rating system. In: Proceedings of QRS 2015, pp. 252–261. IEEE (2015)
Zhang, M., Wang, L., Jajodia, S., Singhal, A., Albanese, M.: Network diversity: a security metric for evaluating the resilience of networks against zero-day attacks. IEEE Trans. Inf. Forensics Secur. 11(5), 1071–1086 (2016)
Zhuang, H., Aberer, K.: A non-intrusive and context-based vulnerability scoring framework for cloud services. arXiv preprint arXiv:1611.07383 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Allodi, L., Biagioni, S., Crispo, B., Labunets, K., Massacci, F., Santos, W. (2017). Estimating the Assessment Difficulty of CVSS Environmental Metrics: An Experiment. In: Dang, T., Wagner, R., Küng, J., Thoai, N., Takizawa, M., Neuhold, E. (eds) Future Data and Security Engineering. FDSE 2017. Lecture Notes in Computer Science(), vol 10646. Springer, Cham. https://doi.org/10.1007/978-3-319-70004-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-70004-5_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70003-8
Online ISBN: 978-3-319-70004-5
eBook Packages: Computer ScienceComputer Science (R0)