skip to main content
10.1145/1882362.1882449acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article

Entropy and software systems: towards an information-theoretic foundation of software testing

Published: 07 November 2010 Publication History

Abstract

We integrate information theory into software testing. In particular, we use entropy in information theory to measure the amount of uncertainty in a software system before it is fully tested, and we show how the amount decreases when we test the system. Moreover, we introduce behaviorial complexity as a novel complexity metric for labeled graphs (which can be interpreted as control flow graphs, design specifications, etc.), which is also based on information theory. We seek practical approaches in testing real systems using the above theories, and we apply our novel approaches in testing model-based embedded systems and network intrusion detection systems. Our information-theoretic approach is syntax-independent, which is a desired property in software testing.

References

[1]
http://www.uml.org/.
[2]
R. Alur, T. Dang, J. Esposito, Y. Hur, F. Ivancic, V. Kumar, P. Mishra, GJ Pappas, and O. Sokolsky. Hierarchical modeling and analysis of embedded systems. Proceedings of the IEEE, 91(1):11--28, 2003.
[3]
P. Ammann and J. Offutt. Introduction to Software Testing. Cambridge University Press, 2008.
[4]
E. M. Clarke, O. Grumberg, and D. A. Peled. Model Checking. The MIT Press, 1999.
[5]
SC-167 committee. Software considerations in airborne systems and equipment certification. Technical report, Radio Technical Commission for Aeronautics, 1992.
[6]
T. M. Cover and J. A. Thomas. Elements of information theory. Wiley-Interscience, second edition, 2006.
[7]
D. Harel. Statecharts: A visual formalism for complex systems. Sci. Comput. Program., 8(3):231--274, 1987.
[8]
MathWorks. Simulink and stateflow. http://www.mathworks.com.
[9]
T.J. McCabe. A complexity measure. IEEE Transactions on Software Engineering, 2:308--320, 1976.
[10]
C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal, 27:379--423, 623--656, 1948.
[11]
L. Tan, O. Sokolsky, and I. Lee. Specification-based testing with linear temporal logic. In IRI'04: proceedings of IEEE Internation Conference on Information Reuse and Integration, pages 483--498. IEEE Computer Society, 2004.
[12]
L. Yang, Z. Dang, and T. R. Fischer. Information gain of black-box testing. Submitted.
[13]
L. Yang, Z. Dang, and T. R. Fischer. A syntax-independent complexity metric. In proceedings of TMFCS'10, pages 127--134, 2010.

Cited By

View all
  • (2018)From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis2018 IEEE 18th International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM.2018.00009(1-23)Online publication date: Sep-2018
  • (2018)Recommending Refactoring Solutions Based on Traceability and Code MetricsIEEE Access10.1109/ACCESS.2018.28689906(49460-49475)Online publication date: 2018
  • (2018)Deploying Search Based Software Engineering with Sapienz at FacebookSearch-Based Software Engineering10.1007/978-3-319-99241-9_1(3-45)Online publication date: 22-Aug-2018
  • Show More Cited By

Recommendations

Reviews

Partha Pratim Das

Considering the density of bugs in software systems to be the equivalent of the amount of uncertainty in them, this paper introduces a model for using entropy in information theory to measure it. It shows that, as we test a software system, the amount of entropy actually decreases as uncertainty decreases in the system. Two basic issues arise from adopting the notion of entropy for software systems. First, information theory is based on probability, and there are no probabilities in software systems. Second, in information theory, entropy is defined on a random variable with no internal structure, but software systems are graphs that are inherently structural. Working around these issues, the paper develops the formalism for applying entropy to software testing, using two approaches to modeling software systems: as random variables, and as random processes. To model software systems as random variables, the authors use the trace specifications of the reactive black box (the software under test). Representing the trace specification on a tree, and associating probabilities with its edges, they define the notion of entropy. As they conduct more tests, edges on the trace-specification tree are either confirmed or removed, and in the process, the entropy decreases. This, then, is used as a testing coverage criterion. To model software systems as random processes, the specifications of software systems are modeled as labeled graphs instead of trees. We can interpret the "behavior set"? of such a graph as the set of desired behaviors of the software system. With this, the paper introduces the "behavioral complexity"? of a software system. It measures a system from the perspective of software testing: it intends to asymptotically measure the cost of exhaustive testing of the system. The paper is well presented. For illustration, it discusses the core theoretical results and applications of the proposed approaches on embedded systems and intrusion detection systems (IDSs). It is quite concise, however, and the reader may need to refer to the authors' earlier work in order to grasp the concepts in depth. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
FoSER '10: Proceedings of the FSE/SDP workshop on Future of software engineering research
November 2010
460 pages
ISBN:9781450304276
DOI:10.1145/1882362
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. information theory
  2. software testing
  3. syntax-independent

Qualifiers

  • Research-article

Conference

SIGSOFT/FSE'10
Sponsor:

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)6
  • Downloads (Last 6 weeks)0
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2018)From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis2018 IEEE 18th International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM.2018.00009(1-23)Online publication date: Sep-2018
  • (2018)Recommending Refactoring Solutions Based on Traceability and Code MetricsIEEE Access10.1109/ACCESS.2018.28689906(49460-49475)Online publication date: 2018
  • (2018)Deploying Search Based Software Engineering with Sapienz at FacebookSearch-Based Software Engineering10.1007/978-3-319-99241-9_1(3-45)Online publication date: 22-Aug-2018
  • (2016)Software Testing and Information TheorySmart Trends in Information Technology and Computer Communications10.1007/978-981-10-3433-6_39(323-330)Online publication date: 27-Dec-2016

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media