research-article

Test adequacy criterion based on coincidental correctness probability

Authors:

Jie Chen,

Qian Li,

Jianhua Zhao,

Xuandong LiAuthors Info & Claims

Internetware '10: Proceedings of the Second Asia-Pacific Symposium on Internetware

Article No.: 20, Pages 1 - 4

https://doi.org/10.1145/2020723.2020743

Published: 03 November 2010 Publication History

Get Access

Abstract

In recent years, coincidental correctness attracts many interests in testing area. Studies prove that it can adversely affect the effectiveness of testing. But current test adequacy criteria don't consider the effect of coincidental correctness. To address the problem, we design an approach to calculate the probability that an error in a given statement of the program under test is hidden by coincidental correctness. Such probability is used to measure the test adequacy on the statement. The test adequacy of a test suit is computed based on the adequacy on all the statement. A mutation analysis on five C programs is designed to compare the effectiveness of our adequacy criterion. The experimental results consistently approve our conjecture that such a test approach effectively quantizes the ability of test suite to find errors.

References

[1]

E. Miller, "Coverage measure definitions reviewed," Testing Tech.Newslett., vol. 3, p. 6, Nov. 1980.

Google Scholar

[2]

J. B. Goodenough and S. L. Gerhart, "Toward a theory of testing:Data selection criteria," in Current Trends in Programming Methodology, vol. 2, R. T. Yeh, Ed. Englewood Cliffs, NJ: Prentice-Hall, 1977, pp. 44--79.

Google Scholar

[3]

W. E. Howden, "Methodology for the generation of test data," IEEE Trans. Comput., vol. C-24, pp. 554--560, May 1975.

Digital Library

Google Scholar

[4]

S. Rapps and E. J. Weyuker, "Data Flow Analysis Techniques for Test Data Selection", Proc. Sixth Int. Conf. Software Engineering, Tokyo, (September 1982).

Digital Library

Google Scholar

[5]

E. J. Weyuker, "The complexity of data flow criteria for test data selection," Inform. Processing Left., vol. 19, no. 2, pp. 103--109, Aug. 1984.

Digital Library

Google Scholar

[6]

B. Baudry, F. Fleurey, and Y. Le Traon, Improving test suites for efficient fault localization, In Proc.of ICSE'06, pages 82--91, May, 2006.

Digital Library

Google Scholar

[7]

John Mitchell and Monty Zukowski, 2003, "A complete GNU C parser and translator", http://www.antlr.org/grammar/list

Google Scholar

[8]

E. J. Weyuker, "On testing non-testable programs," Comput. J., vol. 15, no. 4, pp. 465--470, 1982.

Crossref

Google Scholar

[9]

E. J. Weyuker and T. J. Ostrand, "Theories of ProgramTesting and the Application of Revealing Subdomains,"IEEE Trans. Software Eng., Vol. SE-6, May 1986, pp. 236--246.

Digital Library

Google Scholar

[10]

J. H. Andrews, L. C. Briand,"Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria," IEEE Transactions on Software Engineering, vol. 32 (8), pp. 608--624, 2006.

Digital Library

Google Scholar

Cited By

View all

Feyzi F(2020)CGT-FL: using cooperative game theory to effective fault localization in presence of coincidental correctnessEmpirical Software Engineering10.1007/s10664-020-09859-yOnline publication date: 17-Aug-2020
https://doi.org/10.1007/s10664-020-09859-y
Feyzi FParsa S(2018)A program slicing-based method for effective detection of coincidentally correct test casesComputing10.1007/s00607-018-0591-z100:9(927-969)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1007/s00607-018-0591-z
Zhou XWang HZhao J(2015)A Fault-Localization Approach Based on the Coincidental Correctness ProbabilityProceedings of the 2015 IEEE International Conference on Software Quality, Reliability and Security10.1109/QRS.2015.48(292-297)Online publication date: 3-Aug-2015
https://dl.acm.org/doi/10.1109/QRS.2015.48
Show More Cited By

Index Terms

Test adequacy criterion based on coincidental correctness probability
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Process validation
        Traceability
      2. Software defect analysis
        Software testing and debugging

Recommendations

Prevalence of coincidental correctness and mitigation of its impact on fault localization

Researchers have argued that for failure to be observed the following three conditions must be met: C_R = the defect was reached; C_I = the program has transitioned into an infectious state; and C_P = the infection has propagated to the output. ...
Cleansing Test Suites from Coincidental Correctness to Enhance Fault-Localization
ICST '10: Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation

Researchers have argued that for failure to be observed the following three conditions must be met: 1) the defect is executed, 2) the program has transitioned into an infectious state, and 3) the infection has propagated to the output. Coincidental ...
Mitigating the Effect of Coincidental Correctness in Spectrum Based Fault Localization
ICST '12: Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation

Coincidentally correct test cases are those that execute faulty statements but do not cause failures. Such test cases reduce the effectiveness of spectrum-based fault localization techniques, such as Ochiai. These techniques calculate a suspiciousness ...

Comments

Information & Contributors

Information

Published In

Internetware '10: Proceedings of the Second Asia-Pacific Symposium on Internetware

November 2010

159 pages

ISBN:9781450306942

DOI:10.1145/2020723

Conference Chairs:
Hong Mei,
Jian Lv,
Program Chairs:
Xuandong Li,
Zhi Jin

In-Cooperation

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Internetware 2010

Sponsor:

Internetware 2010: The Second Asia-Pacific Symposium on Internetware

November 3 - 4, 2010

Suzhou, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
113
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Feyzi F(2020)CGT-FL: using cooperative game theory to effective fault localization in presence of coincidental correctnessEmpirical Software Engineering10.1007/s10664-020-09859-yOnline publication date: 17-Aug-2020
https://doi.org/10.1007/s10664-020-09859-y
Feyzi FParsa S(2018)A program slicing-based method for effective detection of coincidentally correct test casesComputing10.1007/s00607-018-0591-z100:9(927-969)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1007/s00607-018-0591-z
Zhou XWang HZhao J(2015)A Fault-Localization Approach Based on the Coincidental Correctness ProbabilityProceedings of the 2015 IEEE International Conference on Software Quality, Reliability and Security10.1109/QRS.2015.48(292-297)Online publication date: 3-Aug-2015
https://dl.acm.org/doi/10.1109/QRS.2015.48
Androutsopoulos KClark DDan HHierons RHarman MJalote PBriand LHoek A(2014)An analysis of the relationship between conditional entropy and failed error propagation in software testingProceedings of the 36th International Conference on Software Engineering10.1145/2568225.2568314(573-583)Online publication date: 31-May-2014
https://dl.acm.org/doi/10.1145/2568225.2568314

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Prevalence of coincidental correctness and mitigation of its impact on fault localization

Cleansing Test Suites from Coincidental Correctness to Enhance Fault-Localization

Mitigating the Effect of Coincidental Correctness in Spectrum Based Fault Localization

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations