research-article

Enhanced lattice-based adaptive random testing

Authors:

Johannes MayerAuthors Info & Claims

SAC '09: Proceedings of the 2009 ACM symposium on Applied Computing

Pages 422 - 429

https://doi.org/10.1145/1529282.1529376

Published: 08 March 2009 Publication History

Abstract

Adaptive Random Testing (ART) has been proposed to improve the fault-detection capability of Random Testing (RT). Lattice-based ART (L-ART) is a distinctive ART method which generates test cases by systematically placing and then randomly shifting lattice nodes in the input domain. Previous studies showed that L-ART has a better fault-detection capability than RT, at the same generation cost. Test cases of L-ART however may be highly concentrated on certain parts of the input domain - a "skewed distribution of test cases". Because of this skewed distribution, when failure regions coincidentally reside in the area where L-ART selects a high density of test cases, L-ART can have a better fault-detection capability than when failure regions are in the low density area. Since failure regions can be in any part of the input domain, this dependency of fault-detection capability on the failure region location is undesirable. We have investigated the cause of such skewed test case distributions using L-ART. Based on our observations, we propose an enhancement to L-ART, which not only has a less-skewed test case distribution, but also demonstrates better and more consistent fault-detection capability than the original L-ART.

References

[1]

P. E. Ammann and J. C. Knight. Data diversity: an approach to software fault tolerance. IEEE Transactions on Computers, 37(4): 418--425, 1988.

Digital Library

[2]

K. P. Chan, T. Y. Chen, and D. Towey. Normalized restricted random testing. In Proceedings of the 8th Ada-Europe International Conference on Reliable Software Technologies (Ada-Europe 2003), volume 2655 of Lecture Notes in Computer Science, pages 368--381, Toulouse, France, 2003. Springer-Verlag.

Digital Library

[3]

K. P. Chan, Chen T. Y., and D. Towey. Probabilistic adaptive random testing. In Proceedings of the 6th International Conference on Quality Software (QSIC 2006), pages 274--280, Beijing, China, 2006. IEEE Computer Society.

Digital Library

[4]

T. Y. Chen, Kuo F.-C., and H. Liu. Enhancing adaptive random testing through partitioning by edge and centre. In Proceedings of the 18th Australian Software Engineering Conference (ASWEC 2007), pages 265--273, Melbourne, Australia, 2007. IEEE Computer Society.

Digital Library

[5]

T. Y. Chen and D. H. Huang. Adaptive random testing by localization. In Proceedings of the 11th Asia-Pacific Software Engineering Conference (APSEC, 04), pages 292--298, Busan, Korea, 2004. IEEE Computer Society.

Digital Library

[6]

T. Y. Chen, D. H. Huang, T. H. Tse, and Zongyuan Yang. An innovative approach to tackling the boundary effect in adaptive random testing. In Proceedings of the 40th Hawaii International Conference on System Sciences (HICSS 2007), pages 1--10, Big Island, HI, USA, 2007. IEEE Press.

Digital Library

[7]

T. Y. Chen, D. H. Huang, and Zhi Quan Zhou. Adaptive random testing through iterative partitioning. In Proceedings of the 11th Ada-Europe International Conference on Reliable Software Technologies (Ada-Europe 2006), volume 4006 of Lecture Notes in Computer Science, pages 155--166, Porto, Portugal, 2006. Springer-Verlag.

Digital Library

[8]

T. Y. Chen, H. Leung, and I. K. Mak. Adaptive random testing. In Proceedings of the 9th Asian Computing Science Conference (ASIAN 2004), volume 3321 of Lecture Notes in Computer Science, pages 320--329, Chiang Mai, Thailand, 2004. Springer-Verlag.

Digital Library

[9]

T. Y. Chen and R. Merkel. An upper bound on software testing effectiveness. ACM Transaction on Software Engineering Methodologies, 17(3): 16: 1--16: 27, 2008.

Digital Library

[10]

T. Y. Chen, T. H. Tse, and Y. T. Yu. Proportional sampling strategy: a compendium and some insights. Journal of Systems and Software, 58(1): 65--81, 2001.

Digital Library

[11]

R. Cobb and H. D. Mills. Engineering software under statistical quality control. IEEE Software, 7(6): 45--54, 1990.

Digital Library

[12]

T. Dabóczi, I. Kollr, G. Simon, and T. Megyeri. Automatic testing of graphical user interfaces. In Proceedings of the 20th IEEE Instrumentation and Measurement Technology Conference 2003 (IMTC 2003), pages 441--445, Vail, Colorado, USA, 2003. IEEE Computer Society.

[13]

B. S. Everitt. The Cambridge Dictionary of Statistics. Cambridge University Press, 1998.

[14]

J. E. Forrester and B. P. Miller. An empirical study of the robustness of windows nt applications using random testing. In Proceedings of the 4th USENIX Windows Systems Symposium, pages 59--68, Seattle, Washington, USA, 2000.

Digital Library

[15]

R. Hamlet. Random testing. In J. Marciniak, editor, Encyclopedia of Software Engineering, pages 970--978. John Wiley & Sons, second edition, 2002.

[16]

F.-C. Kuo, T. Y. Chen, H. Liu, and W. K. Chan. Enhancing adaptive random testing in high dimensional input domains. In Proceedings of the 2007 ACM symposium on Applied computing, pages 265--273, Seoul, Korea, 2007. ACM press.

Digital Library

[17]

P. S. Loo and W. K. Tsai. Random testing revisited. Information and Software Technology, 30(7): 402--417, 1988.

[18]

Johannes Mayer. Lattice-based adaptive random testing. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (ASE 2005), pages 333--336, New York, NY, USA, 2005. ACM Press.

Digital Library

[19]

B. P. Miller, L. Fredriksen, and B. So. An empirical study of the reliability of unix utilities. Communications of the ACM, 33(12): 32--44, 1990.

Digital Library

[20]

E. Miller. Website testing. Software Research, Inc. http://www.soft.com/eValid/Technology/White.Papers/website.testing.html, 2005.

[21]

N. Nyman. In defense of monkey testing: Random testing can find bugs, even in well engineered software. Microsoft Corporation http://www.softtest.org/sigs/material/nnyman2.htm.

[22]

D. Slutz. Massive stochastic testing of sql. In Proceedings of the 24th International Conference on Very Large Data Bases (VLDB98), pages 618--622, New York, NY, USA, 1998.

Digital Library

[23]

L. J. White and E. I. Cohen. A domain strategy for computer program testing. IEEE Transactions on Software Engineering, 6(3): 247--257, 1980.

Digital Library

[24]

T. Yoshikawa, K. Shimura, and T. Ozawa. Random program generator for java jit compiler test system. In Proceedings of the 3rd International Conference on Quality Software (QSIC 2003), pages 20--24, Dallas, TX, USA, 2003. IEEE Computer.

Digital Library

Cited By

Huang RSun WXu YChen HTowey DXia X(2021)A Survey on Adaptive Random TestingIEEE Transactions on Software Engineering10.1109/TSE.2019.294292147:10(2052-2083)Online publication date: 1-Oct-2021
https://doi.org/10.1109/TSE.2019.2942921
Chen YSu ZDi Nitto EHarman MHeymans P(2015)Guided differential testing of certificate validation in SSL/TLS implementationsProceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering10.1145/2786805.2786835(793-804)Online publication date: 30-Aug-2015
https://dl.acm.org/doi/10.1145/2786805.2786835
Patrick MJia Y(2015)Kernel Density Adaptive Random Testing2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW.2015.7107451(1-10)Online publication date: Apr-2015
https://doi.org/10.1109/ICSTW.2015.7107451
Show More Cited By

Enhanced lattice-based adaptive random testing
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

Lattice-based adaptive random testing
ASE '05: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering

Adaptive Random Testing (ART) denotes a family of testing algorithms that have a better performance compared to pure random testing with respect to the number of test cases necessary to detect the first failure. Many of these algorithms, however, are ...
Adaptive Random Testing
QSIC '08: Proceedings of the 2008 The Eighth International Conference on Quality Software

Random testing is a basic testing technique. Motivated by the observation that neighboring inputs normally exhibit similar failure behavior, the approach of Adaptive Random Testing has recently been proposed to enhance the fault detection capability of ...
Does Adaptive Random Testing Deliver a Higher Confidence than Random Testing?
QSIC '08: Proceedings of the 2008 The Eighth International Conference on Quality Software

Random testing (RT) is a fundamental software testing technique. Motivated by the rationale that neighbouring test cases tend to cause similar execution behaviours, adaptive random testing (ART) was proposed as an enhancement of RT, which enforces ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAC '09: Proceedings of the 2009 ACM symposium on Applied Computing

March 2009

2347 pages

ISBN:9781605581668

DOI:10.1145/1529282

Conference Chairs:
Sung Y. Shin
South Dakota State University, United States
,
Sascha Ossowski
University Rey Juan Carlos, Spain

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Australian Research Council

Conference

SAC09

Sponsor:

SIGAPP

SAC09: The 2009 ACM Symposium on Applied Computing

March 8, 2009 - March 12, 2008

Hawaii, Honolulu

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25

Sponsor:
sigapp

The 40th ACM/SIGAPP Symposium on Applied Computing

March 31 - April 4, 2025

Catania , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
232
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Huang RSun WXu YChen HTowey DXia X(2021)A Survey on Adaptive Random TestingIEEE Transactions on Software Engineering10.1109/TSE.2019.294292147:10(2052-2083)Online publication date: 1-Oct-2021
https://doi.org/10.1109/TSE.2019.2942921
Chen YSu ZDi Nitto EHarman MHeymans P(2015)Guided differential testing of certificate validation in SSL/TLS implementationsProceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering10.1145/2786805.2786835(793-804)Online publication date: 30-Aug-2015
https://dl.acm.org/doi/10.1145/2786805.2786835
Patrick MJia Y(2015)Kernel Density Adaptive Random Testing2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW.2015.7107451(1-10)Online publication date: Apr-2015
https://doi.org/10.1109/ICSTW.2015.7107451
Arcuri ABriand LDwyer MTip F(2011)Adaptive random testingProceedings of the 2011 International Symposium on Software Testing and Analysis10.1145/2001420.2001452(265-275)Online publication date: 17-Jul-2011
https://dl.acm.org/doi/10.1145/2001420.2001452
Parizi RGhani AAbdullah RAtan R(2011)Empirical evaluation of the fault detection effectiveness and test effort efficiency of the automated AOP testing approachesInformation and Software Technology10.1016/j.infsof.2011.05.00453:10(1062-1083)Online publication date: Oct-2011
https://doi.org/10.1016/j.infsof.2011.05.004

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten