research-article

DeepXplore: Automated Whitebox Testing of Deep Learning Systems

Authors:

Kexin Pei,

Yinzhi Cao,

Junfeng Yang,

Suman JanaAuthors Info & Claims

GetMobile: Mobile Computing and Communications, Volume 22, Issue 3

Pages 36 - 38

https://doi.org/10.1145/3308755.3308767

Published: 17 January 2019 Publication History

Get Access

Abstract

Over the past few years, Deep Learning (DL) has made tremendous progress, achieving or surpassing human-level performance for a diverse set of tasks, including image classification, speech recognition, and playing games like Go. These advances have led to widespread adoption and deployment of DL in security- and safety-critical systems, such as selfdriving cars, malware detection, and aircraft collision avoidance systems.

References

[1]

Google-accident 2016. "A Google self-driving car caused a crash for the first time." http://www. theverge.com/2016/2/29/11134344/google-selfdriving- car-crash-report. (2016).

Google Scholar

[2]

Tesla-accident 2016. "Understanding the fatal Tesla accident on Autopilot and the NHTSA probe." https://electrek.co/2016/07/01/understanding-fataltesla- accident-autopilot-nhtsa-probe/. (2016).

Google Scholar

[3]

Fei-Fei, Li. "ImageNet: Crowdsourcing, benchmarking & other cool things." Carnegie Mellon University VASC Seminar. Vol. 16. 2010.

Google Scholar

[4]

2016. Report on autonomous mode disengagements for waymo self-driving vehicles in california. https:// www.dmv.ca.gov/portal/wcm/connect/42aff875- 7ab1--4115-a72a-97f6f24b23cc/Waymofull. pdf?MOD=AJPERES&CVID=

Google Scholar

[5]

2017. "Inside Waymo's secret world for training self-driving cars." https://www.theatlantic.com/ technology/archive/2017/08/inside-waymos-secrettesting- and-simulation-facilities/537648/. (2017).

Google Scholar

[6]

Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. "Explaining and harnessing adversarial examples." In Proceedings of the 3rd International Conference on Learning Representations. https://arxiv.org/abs/1412.6572

Google Scholar

[7]

Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. "Deep neural networks are easily fooled: High confidence predictions for unrecognizable images." In Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition.

Google Scholar

[8]

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. "Intriguing properties of neural networks." In Proceedings of the 2nd International Conference on Learning Representations.

Google Scholar

[9]

William M McKeeman. 1998. "Differential testing for software." Digital Technical Journal (1998).

Google Scholar

Cited By

View all

Jiang ZLiu JSang MLi HPan Y(2024)Critical Test Cases Generalization for Autonomous Driving Object Detection Algorithms2024 IEEE Intelligent Vehicles Symposium (IV)10.1109/IV55156.2024.10588519(1149-1156)Online publication date: 2-Jun-2024
https://doi.org/10.1109/IV55156.2024.10588519
Xia CZheng CXia YKong XSong X(2024)Survey on Simulation Testing of Autonomous Driving System2024 11th International Conference on Dependable Systems and Their Applications (DSA)10.1109/DSA63982.2024.00038(221-229)Online publication date: 2-Nov-2024
https://doi.org/10.1109/DSA63982.2024.00038
Wang BWang RSong H(2023)Toward the Trustworthiness of Industrial Robotics Using Differential Fuzz TestingIEEE Transactions on Industrial Informatics10.1109/TII.2022.321188819:3(2782-2791)Online publication date: Mar-2023
https://doi.org/10.1109/TII.2022.3211888
Show More Cited By

Recommendations

DeepXplore: automated whitebox testing of deep learning systems

Deep learning (DL) systems are increasingly deployed in safety- and security-critical domains such as self-driving cars and malware detection, where the correctness and predictability of a system's behavior for corner case inputs are of great ...
DeepXplore: Automated Whitebox Testing of Deep Learning Systems
SOSP '17: Proceedings of the 26th Symposium on Operating Systems Principles

Deep learning (DL) systems are increasingly deployed in safety- and security-critical domains including self-driving cars and malware detection, where the correctness and predictability of a system's behavior for corner case inputs are of great ...
Deep-Learning Approach with DeepXplore for Software Defect Severity Level Prediction
Computational Science and Its Applications – ICCSA 2021
Abstract
Fixing the defects of earlier releases and working on fast and efficient fixing of those software defects is detrimental for the release of further versions. Bug tracking systems like Bugzilla get thousands of software defect reports every day. ...

Comments

Information & Contributors

Information

Published In

cover image GetMobile: Mobile Computing and Communications

GetMobile: Mobile Computing and Communications Volume 22, Issue 3

September 2018

34 pages

ISSN:2375-0529

EISSN:2375-0537

DOI:10.1145/3308755

Editor:
Eyal de Lara
University of Toronto

Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 January 2019

Published in SIGMOBILE-GETMOBILE Volume 22, Issue 3

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
301
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)10

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Jiang ZLiu JSang MLi HPan Y(2024)Critical Test Cases Generalization for Autonomous Driving Object Detection Algorithms2024 IEEE Intelligent Vehicles Symposium (IV)10.1109/IV55156.2024.10588519(1149-1156)Online publication date: 2-Jun-2024
https://doi.org/10.1109/IV55156.2024.10588519
Xia CZheng CXia YKong XSong X(2024)Survey on Simulation Testing of Autonomous Driving System2024 11th International Conference on Dependable Systems and Their Applications (DSA)10.1109/DSA63982.2024.00038(221-229)Online publication date: 2-Nov-2024
https://doi.org/10.1109/DSA63982.2024.00038
Wang BWang RSong H(2023)Toward the Trustworthiness of Industrial Robotics Using Differential Fuzz TestingIEEE Transactions on Industrial Informatics10.1109/TII.2022.321188819:3(2782-2791)Online publication date: Mar-2023
https://doi.org/10.1109/TII.2022.3211888
Mao CSong YChen J(2023)A lightweight adaptive random testing method for deep learning systemsSoftware: Practice and Experience10.1002/spe.325653:11(2271-2295)Online publication date: 30-Aug-2023
https://doi.org/10.1002/spe.3256
Zhu WCheng KGuo YChen Y(2022)Comprehensive Evaluation of the Tendency of Vertical Collusion in Construction Bidding Based on Deep Neural NetworkComputational Intelligence and Neuroscience10.1155/2022/28976722022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/2897672
Li YPei HHuang LYin B(2022)A Distance-Based Dynamic Random Testing Strategy for Natural Language Processing DNN Models2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS57517.2022.00089(842-853)Online publication date: Dec-2022
https://doi.org/10.1109/QRS57517.2022.00089
Yu SAn CYu TZhao ZLi TWang J(2022)Phishing Detection Based on Multi-Feature Neural Network2022 IEEE International Performance, Computing, and Communications Conference (IPCCC)10.1109/IPCCC55026.2022.9894337(73-79)Online publication date: 11-Nov-2022
https://doi.org/10.1109/IPCCC55026.2022.9894337
Yan RChen YGao HYan J(2022)Test case prioritization with neuron valuation based patternScience of Computer Programming10.1016/j.scico.2021.102761215:COnline publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.scico.2021.102761
Haq FShin DBriand LStifter TWang JCadar CZhang X(2021)Automatic test suite generation for key-points detection DNNs using many-objective search (experience paper)Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3460319.3464802(91-102)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3460319.3464802
Zhao YZhu HChen KZhang SKim YKim JVigna GShi E(2021)AI-Lancet: Locating Error-inducing Neurons to Optimize Neural NetworksProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security10.1145/3460120.3484818(141-158)Online publication date: 12-Nov-2021
https://dl.acm.org/doi/10.1145/3460120.3484818
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations