Graph based test case generation for TLM functional verification

https://doi.org/10.1016/j.micpro.2008.03.010Get rights and content

Abstract

Describing complex systems at a high level of abstraction provides designers with the possibility of exploring multiple SoC design architectures before committing to the low level-details of a complete implementation. Transaction level modeling understandably expedites the design simulation and verification. During the verification process, generating good test cases plays a significant role in determining the quality of the design. Inadequate test cases may cause bugs to remain. In this paper, first, in order to generate test cases for a TL model, we present a Control-Transaction Graph (CTG) which describes the behavior of a TL Model. A Control Graph is a control flow graph of a module in the design and transactions represent the interactions such as synchronization between modules. Second, we define dependent paths (DePaths) on the CTG as test cases for a transaction level model, which can find communication errors in simulation. We also give coverage metrics for a TL model to measure the quality of the generated test cases. Finally, we apply our method on the SystemC model of AMBA–AHB bus and JPEG encoder and generate test cases based on the CTG of these models.

Introduction

Conventional RTL simulation is no longer fast enough for verification of today’s complex system-on-a-chip (SoC) designs. Comprehensive verification of such sophisticated designs needs not only verification of the hardware implementation, but also verification of the embedded software. To achieve the necessary simulation performance, we must move to a higher, more abstract level of modeling. Transaction level modeling (TLM) [2], [8] in SystemC [1] achieves 100x improvements in the simulation speed and enables architectural analysis and hardware/software co-verification.

Highly dependable construction of system level designs and proving their correctness are particularly important and arduous. Simulation is a practical and attractive method to achieve this goal. Although TLM expedites the verification of a hardware design [10], [11], [15], the problem of having high coverage test cases remains unsettled at this level of abstraction.

Admittedly, testbench development for SoC designs [7] requires even more effort than the design itself. The complexity of the code requires the use of a sophisticated software development environment.

To prove the correctness of a design by simulation, we must utilize all cases in the design input domain exhaustively. However, as this is not practical, we rather execute a test target program with test-data which are elements selected from a subset, according to some conditions on the input domain. These conditions are called test cases. Test cases play an important role in determining the quality of a design. They must be written carefully and if their number is not adequate, it is likely that some bugs will remain undetected in the design. In addition, test case overlaps would increase the verification cost.

To perform the verification in a minimum period of time, we need some metrics to detail the conditions. These metrics are called coverage metrics. At the evaluation step, by examining the coverage, we can determine how much of the design has been verified. By clearly setting up the coverage metrics before we start verifying a design, we can generate test cases that will satisfy these metrics.

Taylor and Kelly proposed the concept of structural testing of concurrent programs [3] in software domain. They defined the concurrent state graph as a model of a concurrent program in software. Some EDA tools such as Specman elite [9] of Cadence provide test generation with assertion coverage. They offer a user defined constrained random simulation for RTL designs. Jindal and Jain in [5] describe a methodology for verification of TL models by using RTL Testbenches. Habibi et al. in [4], [12] proposed a method based on FSM generation for TLM verification. However, they did not offer a methodology for generating test cases for it. Bombieri et al. [6] proposed a technique for reusing TLM testbenches at RT level with respect to both fault coverage and assertion coverage. Nevertheless, they used already written TLM testbenches and did not explain how these testbenches are obtained. To the best of our knowledge, our previous work [15] is the first in generating test cases for functional verification of TL models. However, in that work we assumed that the TL model does not have any wait statement or synchronization statement. In this paper, we extend our graph to cover these kinds of statements and apply our method to a second case study.

The motivation behind this work is to investigate some techniques to provide high-quality test cases for the verification of an SoC in TLM domain. To show that these test cases are satisfactory, we define some coverage metrics. We introduce the Control-Transaction Graph (CTG) which is a directed graph to generate the appropriate test cases and coverage metrics.

In this paper we propose a technique for generating test cases, which satisfies some defined coverage metrics, based on the SystemC code. We use the Control-Transaction Graph (CTG) as a model for Transaction Level designs. The CTG describes design modules as Control Graphs and Transactions between modules. We generate test cases based on the CTG. Some coverage metrics are defined based on this graph (i.e. CTG) to check the quality of test cases and to control the verification flow.

The rest of this paper is organized as follows: the next section represents the Control-Transaction Graph (CTG) as a model for transaction level designs. Section 3 describes our technique for generating test cases by using CTG. Section 4 defines coverage metrics for Transaction Level Models. Section 5 describes the algorithms which generate test cases on the CTG. Section 6 gives an outline of our test case generation tool. In Section 7 we apply our method on two real SystemC models as case studies. Finally, the last Section concludes the paper.

Section snippets

Representing a TL model by CTG

In this section we introduce the CTG describing the behavior of a transaction level model. A CTG includes two parts: Control Graphs and Transactions. Each Control Graph represents a module independently and transactions show the relations between modules.

Test cases for transaction level models

To generate test cases from a CTG, we initially consider test cases for a single module on a Control Graph.

Coverage metrics

In Section 3 we defined DePaths as test cases for a TL model. To generate paths, a trunk path in a CG and an exchangeable subpath in the trunk path are used. In presence of a loop in a CG, there will be an infinitive number of generated paths. Coverage metrics specify not only termination conditions of test case generation but also the reliability of simulation. If we verify a design based on a specific coverage metric, we expect to detect some types of errors in the design. We adopted simple

Algorithms

We introduce two algorithms to generate test cases from a CTG. The first one describes path generation and the second one DePath generation.

TLM test case generation tool

We have developed the TLM test case generation tool to implement the algorithms of Section 5. The tool generates DePaths from the source code written in SystemC. The tool consists of five steps as shown in Fig. 2. These steps are as follows:

  • 1.

    Generating Control Graphs: all Control Graphs of all sc_modules in SystemC as well as transaction table are generated.

  • 2.

    Making Control-Transaction Graph: the whole CTG consisting of Control Graphs and transaction table is produced. The Control Graphs made in

Case studies

Two case studies from real AMBA and JPEG designs are used to show the effectiveness of our approach. The Advanced Microcontroller Bus Architecture (AMBA) specification [13] defines an on-chip communications standard for designing high-performance embedded microcontrollers. To evaluate our test case generation methodology, we consider the Advanced High-performance Bus (AHB), one of the buses defined within the AMBA specification and JPEG encoder which is a method of compression commonly used for

Conclusion

We introduced a Control-Transaction Graph to describe the principle of test case generation for simulation of a Transaction Level Model. A Control Graph is the control flow graph of a module in a design and transactions represent the interactions such as synchronizations between the modules. Based on the concept of Control Graph and Transactions we set up a TL model with a CTG to generate test cases. The number of overlapped or insufficient test cases decreased since the test cases generated by

Acknowledgements

We would like to thank the TLM group within the CAD research center at the school of ECE, University of Tehran for the concept of “event rate” and “number of events” used in Table 1, Table 2.

References (15)

  • Open SystemC Initiative:...
  • L. Cai et al.

    Transaction level modeling: an overview

    Proc. of CODES+ISSS

    (2003)
  • R.N. Taylor et al.

    Structural testing of concurrent programs

    Proc. of Workshop on Software Testing

    (1986)
  • A. Habibi

    Efficient assertion based verification using TLM

    Proc. of DATE Conference

    (2006)
  • R. Jindal et al.

    Verification of transaction-level SystemC models using RTL Testbenches

    Proc. of MEMOCODE Conference

    (2003)
  • N. Bombieri et al.

    On the evaluation of transactor-based verification for reusing TLM assertions and testbenches at RTL

    Proc. of DATE Conference

    (2006)
  • S. Paricha, Transaction level modeling of SoC with SystemC 2.0, Synopsys User Group Conference,...
There are more references available in the full text version of this article.

Cited by (2)

  • A systematic approach to configurable functional verification of HW IP blocks at transaction level

    2012, Computers and Electrical Engineering
    Citation Excerpt :

    TBs are generated using the SystemC verification library while the coverage metric is not specified. Kakoee et al. ([13]) propose a transaction-level verification of the communication between a system of IPs using ABV. A new coverage metric is proposed accompanied by a TB generation proposal.

  • Modeling of transaction level systemc modules and transactional channels with petri nets

    2008, 2008 2nd International Conference on Signals, Circuits and Systems, SCS 2008
View full text