skip to main content
10.1145/1143997.1144319acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
Article

The state problem for test generation in Simulink

Published: 08 July 2006 Publication History

Abstract

Search based test-data generation has proved successful for code-level testing. In this paper we investigate the application of such approaches at the higher levels of abstraction offered by Matlab-Simulink models. The presence of persistent state has been shown to be problematic at the code level and such difficulties remain when Matlab-Simulink models are to be tested. In such cases, sequences of inputs that can put the model under test into particular states are needed to enable the underlying test goals to be achieved. Simple search guidance appears to be insufficient and results in a 'flat' cost function landscape. To address this problem, we introduce a technique called tracing and deducing, which helps provide better guidance to the search, allowing our developed tools to home in on the targeted test-data.

References

[1]
B. Korel. Automated Software Test Data Generation. IEEE Trans. on Softw. Engineering, 16(8): 870--879, 1990.
[2]
N. Tracey, J. Clark, K. Mander, and J. McDermid. An Automated Framework for Structural Test-Data Generation. Int'l Conf. on Auto. Softw. Eng., pp 285--288, 1998.
[3]
J. Wegener, K. Buhr, and H. Pohlheim. Automatic Test Data Generation for Structural Testing of Embedded Software Systems by Evolutionary Testing. GECCO 2002, pp 1233--1240.
[4]
S. Xanthakis, C. Ellis, C. Skourlas, A. Le Gal, S. Katsikas and K. Karapoulios. Application of Genetic Algorithms to Software Testing. In Int'l Conf. on Softw. Engineering and its Applications, pp 625--636, 1992.
[5]
B. Jones, H. Sthamer, and D. Eyres. Automatic Structural Testing Using Genetic Algorithms. Software Engineering Journal, 11(5): 299--306, 1996.
[6]
N. Tracey, J. Clark, and K. Mander. Automated Program Flaw Finding Using Simulated Annealing. Symposium on Software Testing and Analysis (ISSTA), pp 73--81. 1998.
[7]
N. Tracey, J. Clark, K. Mander, and J. McDermid. Automated Test Data Generation for Exception Conditions. Software - Practice and Experience, 30(1): 61--79, 2000.
[8]
O. Buehler and J. Wegener. Evolutionary Functional Testing of an Automated Parking System. In Int'l Conf. on Computer, Communication and Control Technologies (CCCT'03) and The 9th Int'l Conf. on Information Systems Analysis and Synthesis, (ISAS'03), 2003.
[9]
A. Baresel, H. Pohlheim, and S. Sadeghipour. Structural and Functional Sequence Test of Dynamic and State-Based Software with Evolutionary Algorithms. GECCO 2003, pp 2428--2441.
[10]
J. Wegener, K. Grimm, M. Grochtmann, H. Sthamer and B. Jones. Systematic Testing of Real-Time Systems. Proc. of the 4th European Conference on Software Testing, Analysis & Review (EuroSTAR '1996), Dec. 1996.
[11]
P. Puschner and R. Nossal. Testing the Results of Static Worst-Case Execution-Time Analysis. Proc. of the 19th IEEE Real-Time Systems Symposium, pp 134--143, 1998.
[12]
B. Jones, H. Sthamer, X. Yang, and D. Eyres. The Automatic Generation of Software Test Data Sets Using Adaptive Search Techniques. The 3rd Int'l Conf. on Software Quality Management, pp 435--444, 1995.
[13]
Y. Zhan, and J. Clark. Search-Based Automatic Test-Data Generation at an Architectural Level. GECCO 2004, pp 1413--1426.
[14]
Leonardo Bottaci. Predicate Expression Cost Functions to Guide Evolutionary Search for Test Data. GECCO 2003, pp 2455--2464.
[15]
Hong Zhu, Patrick A. V. Hall and John H. R. May. Software Unit Test Coverage and Adequacy. ACM Computing Surveys, Vol. 29(4): 366--427. December 1997.
[16]
C. R. Reeves (Ed.). Modern Heuristic Techniques for Combinatorial Problems. Blackwell, Oxford, 1993.
[17]
R. Kirner, R. Lang, G. Freiberger and P. Puschner. Fully Automatic Worst-Case Execution Time Analysis for Matlab/Simulink Models. Euromicro Conference on Real-Time Systems, 2002.
[18]
S. Kirkpatrick, C. Gelatt, and M. Vecchi. Optimization by Simulated Annealing. Science, 220(4598): 671--680, 1983.
[19]
P. McMinn. Search-based Software Test Data Generation: A Survey. Software Testing, Verification and Reliability, 14(2), pp 105-156, June 2004.
[20]
Eugenia Díaz, Javier Tuya, Raquel Blanco. Automated Software Testing Using a Metaheuristic Technique Based on Tabu Search. In 18th IEEE Int'l. Conf. on Automated Software Engineering. Montreal, Canada, Oct. 2003.
[21]
The MathWorks. http://www.mathworks.com/products/simulink.
[22]
Y. Zhan, and J. Clark. Search-Based Mutation Testing for Simulink Models. GECCO 2005, pp 1061--1068.
[23]
M. Harman, L. Hu, R. Hierons, A. Baresel, and H. Sthamer. Improving Evolutionary Testing by Flag Removal. GECCO 2002, pp 1359--1366.
[24]
P. McMinn, and M. Holcombe. The State Problem for Evolutionary Testing. GECCO 2003, pp 2488--2500.
[25]
P. McMinn. Evolutionary Search for Test Data in the Presence of State Behaviour. PhD Thesis, University of Sheffield, January 2005.
[26]
Reactive Systems Inc. Http://www.reactive-systems.com/.
[27]
L. Clarke. A System to Generate Test Data and Symbolically Execute Programs. IEEE Transactions on Software Engineering, 2(3): 215--222. 1976.
[28]
R. A. DeMillo and A. J. Offutt. Constraint-Based Automatic Test Data Generation. IEEE Transactions on Software Engineering, 17(9): 900--909. 1991.
[29]
A. J. Offutt, Z. Jin and J. Pan. The Dynamic Domain Reduction Procedure for Test Data Generation. Software - Practice and Experience, 29(2): 167--193. 1999.
[30]
Y. Zhan. A Search-Based Framework for Automatic Test-Set Generation for MATLAB/Simulink Models. PhD thesis, University of York. Dec 2005.

Cited By

View all
  • (2018)Mapping the Effectiveness of Automated Test Suite Generation TechniquesIEEE Transactions on Reliability10.1109/TR.2018.283207267:3(771-785)Online publication date: Sep-2018
  • (2016)Testing advanced driver assistance systems using multi-objective search and neural networksProceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering10.1145/2970276.2970311(63-74)Online publication date: 25-Aug-2016
  • (2016)The experimental applications of search-based techniques for model-based testingApplied Soft Computing10.1016/j.asoc.2016.08.03049:C(1094-1117)Online publication date: 1-Dec-2016
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '06: Proceedings of the 8th annual conference on Genetic and evolutionary computation
July 2006
2004 pages
ISBN:1595931864
DOI:10.1145/1143997
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Matlab-Simulink
  2. automation
  3. state problem
  4. structural coverage
  5. test-data generation
  6. tracing and deducing

Qualifiers

  • Article

Conference

GECCO06
Sponsor:
GECCO06: Genetic and Evolutionary Computation Conference
July 8 - 12, 2006
Washington, Seattle, USA

Acceptance Rates

GECCO '06 Paper Acceptance Rate 205 of 446 submissions, 46%;
Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2018)Mapping the Effectiveness of Automated Test Suite Generation TechniquesIEEE Transactions on Reliability10.1109/TR.2018.283207267:3(771-785)Online publication date: Sep-2018
  • (2016)Testing advanced driver assistance systems using multi-objective search and neural networksProceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering10.1145/2970276.2970311(63-74)Online publication date: 25-Aug-2016
  • (2016)The experimental applications of search-based techniques for model-based testingApplied Soft Computing10.1016/j.asoc.2016.08.03049:C(1094-1117)Online publication date: 1-Dec-2016
  • (2014)An Integrated Analysis and Testing Methodology to Support Model-Based Quality AssuranceSoftware Quality. Model-Based Approaches for Advanced Software and Systems Engineering10.1007/978-3-319-03602-1_9(135-154)Online publication date: 2014
  • (2013)Analysis and testing of matlab simulink models: a systematic mapping studyProceedings of the 2013 International Workshop on Joining AcadeMiA and Industry Contributions to testing Automation10.1145/2489280.2489285(29-34)Online publication date: 15-Jul-2013
  • (2012)Search-based system testing: high coverage, no false alarmsProceedings of the 2012 International Symposium on Software Testing and Analysis10.1145/2338965.2336762(67-77)Online publication date: 15-Jul-2012
  • (2012)Multi-objective coevolutionary automated software correctionProceedings of the 14th annual conference on Genetic and evolutionary computation10.1145/2330163.2330333(1229-1236)Online publication date: 7-Jul-2012
  • (2012)Evolutionary algorithm for prioritized pairwise test data generationProceedings of the 14th annual conference on Genetic and evolutionary computation10.1145/2330163.2330331(1213-1220)Online publication date: 7-Jul-2012
  • (2012)Evolutionary algorithms for the multi-objective test data generation problemSoftware—Practice & Experience10.1002/spe.113542:11(1331-1362)Online publication date: 1-Nov-2012
  • (2011)Transition coverage testing for simulink/stateflow models using messy genetic algorithmsProceedings of the 13th annual conference on Genetic and evolutionary computation10.1145/2001576.2001825(1851-1858)Online publication date: 12-Jul-2011
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media