short-paper

The (un)reliability of NVD vulnerable versions data: an empirical experiment on Google Chrome vulnerabilities

Authors:

Viet Hung Nguyen,

Fabio MassacciAuthors Info & Claims

ASIA CCS '13: Proceedings of the 8th ACM SIGSAC symposium on Information, computer and communications security

Pages 493 - 498

https://doi.org/10.1145/2484313.2484377

Published: 08 May 2013 Publication History

Abstract

NVD is one of the most popular databases used by researchers to conduct empirical research on data sets of vulnerabilities. Our recent analysis on Chrome vulnerability data reported by NVD has revealed an abnormally phenomenon in the data where almost vulnerabilities were originated from the first versions. This inspires our experiment to validate the reliability of the NVD vulnerable version data. In this experiment, we verify for each version of Chrome that NVD claims vulnerable is actually vulnerable. The experiment revealed several errors in the vulnerability data of Chrome. Furthermore, we have also analyzed how these errors might impact the conclusions of an empirical study on foundational vulnerability. Our results show that different conclusions could be obtained due to the data errors.

References

[1]

G. Antoniol, K. Ayari, M. D. Penta, F. Khomh, and Y. Guhneuc. Is it a bug or an enhancement? a text-based approach to classify change requests. In Proc. of CASCON'08, pages 304--318, 2008.

Digital Library

[2]

C. Bird, A. Bachmann, E. Aune, J. Duffy, A. Bernstein, V. Filkov, and P. Devanbu. Fair and balanced? bias in bug-fix datasets. pages 121--130. ACM, 2009.

Digital Library

[3]

IBM. Internet Security System/X-Force, 2012. http://xforce.iss.net/.

[4]

F. Massacci, S. Neuhaus, and V. H. Nguyen. After-life vulnerabilities: A study on firefox evolution, its vulnerabilities and fixes. In Proc. of ESSoS'11, 2011.

Digital Library

[5]

National Institute of Standards and Technology. National Vulnerability Database, August 2012. http://web.nvd.nist.gov/.

[6]

B. H. A. Nguyen, T.H.D.; Adams. A case study of bias in bug-fix datasets. In Proc. of WCRE'10, 2010.

Digital Library

[7]

V. H. Nguyen and F. Massacci. The (un)reliability of nvd vulnerable versions data: an empirical experiment on google chrome vulnerabilities. CoRR, 2013. http://arxiv.org/abs/1302.4133.

Digital Library

[8]

V. H. Nguyen and F. Massacci. An independent validation of vulnerability discovery models. In Proc. of ASIACCS'12, May 2012.

Digital Library

[9]

OSVDB. The Open Source Vulnerability Database. http://www.osvdb.org.

[10]

A. Ozment and S. E. Schechter. Milk or wine: Does software security improve with age? In Proc. of USENIX'06, 2006.

Digital Library

[11]

E. Rescorla. Is finding security holes a good idea? IEEE Sec. and Privacy, 3(1):14--19, 2005.

Digital Library

[12]

Security Focus. Bug Traq, 2012. http://www.securityfocus.com.

[13]

J. Sliwerski, T. Zimmermann, and A. Zeller. When do changes induce fixes? In Proc. of MSR'05, pages 24--28, 2005.

Digital Library

[14]

A. Younis, H. Joh, and Y. Malaiya. Modeling learningless vulnerability discovery using a folded distribution. In Proc. of SAM'11, pages 617--623, 2011.

[15]

T. Zimmermann, R. Premraj, and A. Zeller. Predicting defects for eclipse. pages 9--15. IEEE Computer Society, 2007.

Digital Library

Cited By

Wu SWang RHuang KCao YSong WZhou ZHuang YChen BPeng XFilkov VRay BZhou M(2024)Vision: Identifying Affected Library Versions for Open Source Software VulnerabilitiesProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695516(1447-1459)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695516
Wunder JCorona AHammer ABenenson Z(2024)On NVD Users’ Attitudes, Experiences, Hopes, and HurdlesDigital Threats: Research and Practice10.1145/36888065:3(1-19)Online publication date: 21-Aug-2024
https://dl.acm.org/doi/10.1145/3688806
Dunlap TLin EEnck WReaves BQuek TGao DZhou JCardenas A(2024)VFCFinder: Pairing Security Advisories and PatchesProceedings of the 19th ACM Asia Conference on Computer and Communications Security10.1145/3634737.3657007(1128-1142)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3634737.3657007
Show More Cited By

Recommendations

Browser Security: Lessons from Google Chrome: Google Chrome developers focused on three key problems to shield the browser from attacks.
Distributed Computing

The Web has become one of the primary ways people interact with their computers, connecting people with a diverse landscape of content, services, and applications. Users can find new and interesting content on the Web easily, but this presents a ...
CVEfixes: automated collection of vulnerabilities and their fixes from open-source software
PROMISE 2021: Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering

Data-driven research on the automated discovery and repair of security vulnerabilities in source code requires comprehensive datasets of real-life vulnerable code and their fixes. To assist in such research, we propose a method to automatically collect ...
A Large-Scale Empirical Study of Security Patches
CCS '17: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security

Given how the "patching treadmill" plays a central role for enabling sites to counter emergent security concerns, it behooves the security community to understand the patch development process and characteristics of the resulting fixes. Illumination of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASIA CCS '13: Proceedings of the 8th ACM SIGSAC symposium on Information, computer and communications security

May 2013

574 pages

ISBN:9781450317672

DOI:10.1145/2484313

General Chairs:
Kefei Chen
Shanghai Jiao Tong University, China
,
Qi Xie
Hangzhou Normal University, China
,
Weidong Qiu
Shanghai Jiao Tong University, China
,
Program Chairs:
Ninghui Li
Purdue University, USA
,
Wen-Guey Tzeng
National Chiao Tung University, Taiwan

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ASIA CCS '13

Sponsor:

SIGSAC

ASIA CCS '13: 8th ACM Symposium on Information, Computer and Communications Security

May 8 - 10, 2013

Hangzhou, China

Acceptance Rates

ASIA CCS '13 Paper Acceptance Rate 35 of 216 submissions, 16%;

Overall Acceptance Rate 418 of 2,322 submissions, 18%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

44
Total Citations
View Citations
463
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)5

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu SWang RHuang KCao YSong WZhou ZHuang YChen BPeng XFilkov VRay BZhou M(2024)Vision: Identifying Affected Library Versions for Open Source Software VulnerabilitiesProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695516(1447-1459)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695516
Wunder JCorona AHammer ABenenson Z(2024)On NVD Users’ Attitudes, Experiences, Hopes, and HurdlesDigital Threats: Research and Practice10.1145/36888065:3(1-19)Online publication date: 21-Aug-2024
https://dl.acm.org/doi/10.1145/3688806
Dunlap TLin EEnck WReaves BQuek TGao DZhou JCardenas A(2024)VFCFinder: Pairing Security Advisories and PatchesProceedings of the 19th ACM Asia Conference on Computer and Communications Security10.1145/3634737.3657007(1128-1142)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3634737.3657007
Wu SSong WHuang KChen BPeng XRoychoudhury APaiva AAbreu RStorey M(2024)Identifying Affected Libraries and Their Ecosystems for Open Source Software VulnerabilitiesProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639582(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639582
He YWang YZhu SWang WZhang YLi QYu A(2024)Automatically Identifying CVE Affected Versions With Patches and Developer LogsIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.326456721:2(905-919)Online publication date: Mar-2024
https://doi.org/10.1109/TDSC.2023.3264567
Reid DRahkema KWalden JMcIntosh SChoi EHerbold S(2023)Large Scale Study of Orphan Vulnerabilities in the Software Supply ChainProceedings of the 19th International Conference on Predictive Models and Data Analytics in Software Engineering10.1145/3617555.3617872(22-32)Online publication date: 8-Dec-2023
https://dl.acm.org/doi/10.1145/3617555.3617872
Song JWan SHuang MLiu JSun LLi Q(2023)Toward Automatically Connecting IoT Devices with Vulnerabilities in the WildACM Transactions on Sensor Networks10.1145/360895120:1(1-26)Online publication date: 17-Jul-2023
https://dl.acm.org/doi/10.1145/3608951
Okutan AMell PMirakhorli MKhokhlov ISantos JGonzalez DSimmons S(2023)Empirical Validation of Automated Vulnerability Curation and CharacterizationIEEE Transactions on Software Engineering10.1109/TSE.2023.325047949:5(3241-3260)Online publication date: 1-May-2023
https://doi.org/10.1109/TSE.2023.3250479
Malik ATosh D(2023)Dynamic Vulnerability Classification for Enhanced Cyber Situational Awareness2023 IEEE International Systems Conference (SysCon)10.1109/SysCon53073.2023.10131235(1-8)Online publication date: 17-Apr-2023
https://doi.org/10.1109/SysCon53073.2023.10131235
Xu CChen BLu CHuang KPeng XLiu YRoychoudhury ACadar CKim M(2022)Tracking patches for open source software vulnerabilitiesProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3540250.3549125(860-871)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3540250.3549125
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten