skip to main content
10.1145/1450058.1450088acmconferencesArticle/Chapter ViewAbstractPublication PagesesweekConference Proceedingsconference-collections
research-article

Randomized directed testing (REDIRECT) for Simulink/Stateflow models

Published: 19 October 2008 Publication History

Abstract

The Simulink/Stateflow (SL/SF) environment from Math-works is becoming the de facto standard in industry for model based development of embedded control systems. Many commercial tools are available in the market for test case generation from SL/SF designs; however, we have observed that these tools do not achieve satisfactory coverage in cases when designs involve nonlinear blocks and Stateflow blocks occur deep inside the Simulink blocks.
The recent past has seen the emergence of several novel techniques for testing large C, C++ and Java programs; prominent among them are directed automated random testing (DART), hybrid concolic testing and feedback-directed random testing. We believe that some of these techniques could be lifted to testing of SL/SF based designs; REDIRECT (RandomizEd DIRECted Testing), the proposed testing method of this paper, is an attempt towards this direction. Specifically, REDIRECT uses a careful combination of the above techniques, and in addition, the method uses a set of pattern-guided heuristics for tackling nonlinear blocks. A prototype tool has been developed and the tool has been applied to many industrial strength case studies. Our experiments indicate that a careful choice of heuristics and certain combinations of random and directed testing achieve better coverages as compared to the existing commercial tools. 1

References

[1]
Applied Dynamics International. BEACON for Simulink/Stateflow, http://www.adi.com
[2]
R. Alur, A.Kanade, S.Ramesh, and K.C. Shashidhar. Symbolic Analysis for Improving Coverage of Simulink/Stateflow Models, In International Conference on Embedded Software, 2008, To appear.
[3]
R. Cleaveland, S.A. Smolka, and S.T. Sims. An Instrumentation-Based Approach to Controller Model Validation, Available at http://aswsd.ucsd.edu/2006/pdfs/Smolka-vm-slides.pdf
[4]
ETSI. ETS 300 406: Methods for Testing and Specification (MTS); Protocol and profile conformance testing specifications; Standardization Methodology, European Telecommunication Standard, 1995.
[5]
A. Gadkari, A. Yeolekar, J. Suresh, S. Ramesh, S. Mohalik, and K.C. Shashidhar. AutoMOTGen: Automatic Model Oriented Test Generator for Embedded Control Systems, In Proc. of the CAV'08, LNCS Volume 5123, pages 204--208, 2008.
[6]
P. Godefroid, N. Klarlund, and K. DART: Directed Automated Random Testing, In Proc. of the PLDI'05, Chicago, pp. 213--223, 2005.
[7]
G. Hamon, L. de Moura, and J. Rushby. Automated Test Generation with SAL, CSL Technical Note, January 2005.
[8]
ISO. OSI Conformance Testing Methodology and Framework- ISO 9646, 1998.
[9]
A. Agung Julius, G. Fainekos, M. Anand, I. Lee, and G.J. Pappas. Robust test generation and coverage for hybrid systems, In LNCS Volume 4416, Springer, pages 329--342, 2007.
[10]
R. Majumdar, and K. Sen. Hybrid Concolic Testing, In Proc. of the ICSE, Minneaplois, pages 416--426, 2007.
[11]
The Mathworks, http://www.mathworks.com
[12]
The Mathworks, Simulink Design verifier, http://www.mathworks.com
[13]
S.P Miller, E.A. Anderson, L.G. Wagner, M.W. Whalen, and M.P.E. Heimdahl. Formal Verification of Flight Control Software, In Proc. of the AIAA Guidance, Navigation and Control Conference and Exhibit, San Francisco, pages 1--16, August 2005.
[14]
J. Offutt, S. Liu, A. Abdurazik, and P. Ammann. Generating Test Data from State Based Specifications, Software Testing, Verification and Reliability, John Wiley, 13(1): 25--53, March 2003.
[15]
C. Pacheco, S.K. Lahiri, M. Ernst, T. Ball. Feedback-directed Random Test Generation, In Proc. of the ICSE, Minneapolis, pages 75--84, May 2007.
[16]
Reactis: http://www.reactive-systems.com
[17]
Reactive Systems. Model Based Testing and Validation with Reactis, Reactive Systems Inc., http://www.reactive-systems.com
[18]
D.J. Richardson, A. Leif Aha, and T.O. O'Malley. Specification-based Test Oracles for Reactive Systems, In Proc. of ICSE, Melbourne, pp. 105--118, 1992.
[19]
SRI International. SAL home page http://sal.csl.sri.com
[20]
S. Sims, and D.C DuVarney. Experience Report: The Reactis Validation Tool, In Proc. of the ACM International Conference on Functional Programming, Freiburg, pages 137--139, October 2007.
[21]
STB. Safety Test Builder, Automatic Test Generation for Simulink/Stateflow, TIN Software.
[22]
T-Vec. T-Vec Tester for Simulink, http://www.t-vec.com
[23]
H. Zhu, P.A.V. Hall, and J.H.R. May. Software Unit Test Coverage and Adequacy, ACM Computing Surveys, 29(4):366--427, 1997.

Cited By

View all
  • (2024)Test Case Generation for Simulink Models using Model Fuzzing and State SolvingProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3694991(117-128)Online publication date: 27-Oct-2024
  • (2024)CFTCG: Test Case Generation for Simulink Model through Code Based FuzzingProceedings of the 61st ACM/IEEE Design Automation Conference10.1145/3649329.3656216(1-6)Online publication date: 23-Jun-2024
  • (2024)HSTCG: State-Aware Simulink Model Test Case Generation with Heuristic StrategyIEEE Transactions on Software Engineering10.1109/TSE.2024.3428528(1-17)Online publication date: 2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
EMSOFT '08: Proceedings of the 8th ACM international conference on Embedded software
October 2008
284 pages
ISBN:9781605584683
DOI:10.1145/1450058
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2008

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. coverage
  2. hybrid systems
  3. model based testing
  4. simulink/stateflow models

Qualifiers

  • Research-article

Conference

ESWEEK 08
ESWEEK 08: Fourth Embedded Systems Week
October 19 - 24, 2008
GA, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 60 of 203 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)12
  • Downloads (Last 6 weeks)3
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Test Case Generation for Simulink Models using Model Fuzzing and State SolvingProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3694991(117-128)Online publication date: 27-Oct-2024
  • (2024)CFTCG: Test Case Generation for Simulink Model through Code Based FuzzingProceedings of the 61st ACM/IEEE Design Automation Conference10.1145/3649329.3656216(1-6)Online publication date: 23-Jun-2024
  • (2024)HSTCG: State-Aware Simulink Model Test Case Generation with Heuristic StrategyIEEE Transactions on Software Engineering10.1109/TSE.2024.3428528(1-17)Online publication date: 2024
  • (2023)Improve Model Testing by Integrating Bounded Model Checking and Coverage Guided FuzzingElectronics10.3390/electronics1207157312:7(1573)Online publication date: 27-Mar-2023
  • (2023)STCG: State-Aware Test Case Generation for Simulink Models2023 60th ACM/IEEE Design Automation Conference (DAC)10.1109/DAC56929.2023.10247787(1-6)Online publication date: 9-Jul-2023
  • (2022)Coverage Testing of Industrial Simulink Models using Monte-Carlo and SMT-Based Methods2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS57517.2022.00050(422-433)Online publication date: Dec-2022
  • (2021)Model-based test case generation and prioritization: a systematic literature reviewSoftware and Systems Modeling10.1007/s10270-021-00924-821:2(717-753)Online publication date: 7-Sep-2021
  • (2020)Template-Based Monte-Carlo Test-Suite Generation for Large and Complex Simulink ModelsIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences10.1587/transfun.2019MAP0010E103.A:2(451-461)Online publication date: 1-Feb-2020
  • (2019)Concolic testing for models of state-based systemsProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3338906.3338908(4-15)Online publication date: 12-Aug-2019
  • (2019)Model-based, Mutation-driven Test-case Generation Via Heuristic-guided Branching SearchACM Transactions on Embedded Computing Systems10.1145/328925618:1(1-28)Online publication date: 25-Jan-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media