A framework for assume-guarantee regression verification of evolving software

Highlights

• This paper presents a method for generating local weakest assumptions using a backtracking algorithm.

• The backtracking algorithm is based on CDNF algorithm and a variant of membership query answering technique.

• The correctness of the backtracking algorithm is presented in the paper

• The backtracking algorithm is then integrated into a framework for effectively rechecking evolving software.

• The paper presents experimental results for some common systems in the researcher community.

Abstract

This paper presents a framework for verifying evolving component-based software using assume-guarantee logic. The goal is to improve CDNF-based assumption generation method by having local weakest assumptions that can be used more effectively when verifying component-based software in the context of software evolution. For this purpose, we improve the technique for responding to membership queries when generating candidate assumptions. This technique is then integrated into a proposed backtracking algorithm to generate local weakest assumptions. These assumptions are effectively used in rechecking the evolving software by reducing time required for assumption regeneration within the proposed framework. The proposed framework can be applied to verify software that is continually evolving. An implemented tool and experimental results are presented to demonstrate the effectiveness and usefulness of the framework.

Introduction

In the last three decades, component-based software engineering (CBSE) has emerged as one of the important approaches in software engineering. This approach has shown a number of advantages such as increasing effectiveness and efficiency, lowering cost, shortening product time-to-market, improving maintainability [52]. As a result, component-based software (CBS) quality assurance plays a critical role in software production life cycles due to the increasing demand for high-quality products. Due to the high-quality standard test procedure in software industry, the verification process in CBSs ensures that certain properties are not violated at all times.

There are two approaches to the verification of modern software: theorem proving which is semi-automatic, requires the interaction of domain experts [21], [20], [30], [37], [38], [51], and costs a lot of effort [5]; model checking which is automatic and does not require the interaction of domain experts [7], [18]. Although the model checking has gained considerable attention due to its fully automatic characteristic, the approach suffers from the problem of state space explosion [15], [18], [48], [16]. The assume-guarantee framework [17], [19], [24], [46], which performs modular verification of CBS, has been considered a promising solution for dealing with the state space explosion problem during model checking. The framework uses the “divide-and-conquer” strategy to verify whether a given system satisfies a predefined property. Therefore, it can potentially be applied to large-scale systems in practice. The key problem of the framework is to generate assumptions that satisfy the assume-guarantee rules [19], [29], [33]. If such an assumption exists, the given system satisfies the required property. Although the framework can be applied to large-scale systems effectively, it does not consider the system under check in the context of software evolution.

Modern software applications are continually evolving, and any verification has to be revisited repeatedly. A reduction in the cost of this repeated verification would offer significant benefits for industry: improving the quality of software through application of verification techniques in situations where this is currently infeasible. Progress has been made using approaches such as labeled transition systems [12], [19], [31], [33], [34], [35], implicit representation of transition systems [13], [27], timed transition systems [3], [28], [40], [41], [42]. The following two solutions have been used in reducing the verification costs for evolving software.

The first solution is to generate a new assumption each time software evolves at a lower cost. For software modeled by exploiting labeled transition systems, assumptions with small sizes (i.e., assumptions with small numbers of states) can be used effectively to recheck modified software leading to reduced verification cost. In a series of papers, Hung et al. proposed a method to generate minimized assumptions for CBS verification [31], [34], [35] and a framework to perform modular verification of evolving CBS [33]. However, the cost for generating minimal assumptions can be high [34]. The reason is that the investigated assumption generation problem [19], [33], [34], [35], [31] is formulated as an automata learning problem using the $L^{⁎}$ algorithm [4]. As a result, it is difficult to apply this approach to large-scale systems. On the other hand, for the faster assumption generation speed, another verification method, which uses CDNF (Conjunction of Disjunctive Normal Form) algorithm [10] and implicit representation of software, was proposed in 2010 by Chen et al. [13]. Later, in 2016, this method was improved by He et al. and applied in CBS regression verification [26] by introducing a fine–grained learning technique. However, with modified software, some of the subpredicates of the new version of components can be different, which requires the regression verification progress to regenerate the assumptions for every small change in the software component.

The second solution to reduce the verification cost for modified software is to increase assumption reuse as much as possible. This is because the software development cycle involves daily change. Therefore, the less time required to regenerate assumptions, the greater the cost savings when verifying modified software. Moreover, from the analysis in Section 5 below, weak assumptions (i.e., assumptions with large languages) can help to achieve this purpose and play a key role in the verification of modified software. On the other hand, to our knowledge, no research has been conducted on generating assumptions that have the weakest languages and use implicit specification. As a result, this research focuses on improving the learning algorithm proposed by Chen [13] to generate local weakest assumptions that can be used more efficiently to reduce the cost of software regression verification during software evolution.

To achieve the above goal, we first improve the technique to answer membership queries for the two ι (i.e., the initial predicate) and τ (i.e., transition relation) CDNF learning instances. Based on this improved answering technique, we can generate weaker assumptions than those generated by the algorithm proposed by Chen et al. [13] (hereafter, we refer to as CBAG algorithm) using a proposed backtracking learning algorithm (referred to as LWAG algorithm). This leads to an important result in the context of software evolution: LWAG algorithm can reduce the number of times assumptions must be regenerated when verifying modified software. The improved answering technique and LWAG algorithm are integrated into a framework to effectively reduce the number of times assumption regeneration is required for evolving software.

Using assumption generation algorithms which employ the implicit representation, we can not only benefit from the fast learning process but we can also obtain several advantages of implicit software representation over explicit representation. First, the contextual assumptions represented implicitly using Boolean functions have fewer states than do assumptions modeled using deterministic finite automata because implicit representations are equivalent to nondeterministic finite automata, which are exponentially more succinct than deterministic ones. As a result, our generated assumptions can have an exponentially smaller number of states than do assumptions generated from explicit representations. The second advantage is the scalability of the verification method using implicit representations, which occurs because the $L^{⁎}$ algorithm requires a polynomial number of queries in the number of states of the target finite automaton [4], [49]. In contrast, the CDNF algorithm requires a polynomial number of queries in the number of Boolean variables of the target Boolean function [10]. Because implicit assumptions can be exponentially more succinct than explicit ones, the learning algorithms for implicit assumptions can be exponentially better than automata-theoretic ones.

To our knowledge, the first paper that proposed using the $L^{⁎}$ algorithm to learn assumptions for the assume-guarantee reasoning algorithm was Cobleigh et al. [19]. Following this paper, several studies improved the method, including adoption of the assume-guarantee rules [6], [26], [39], [45], symbolic implementation for assume-guarantee rules [8], [9], [45], several improvements proposed in [1], [2], [12], [14], [25], [50], [53], and an extension to support liveness properties [22]. However, these papers all use the $L^{⁎}$ algorithm to learn an automaton as the required contextual assumption. Hence, they all have the same disadvantages as described above compared to the algorithm proposed in Chen's paper [13]. Hence, we based our paper on Chen's algorithm [13] to verify modified software.

The remainder of this paper is organized as follows. Section 2 presents the background for this paper. We review CBAG algorithm for generating assumptions using the CDNF algorithm in Section 3, followed by the proposed algorithms to improve the answers to membership queries and generate assumptions in Section 4. Section 5 presents a framework for verifying modified CBSs using assumptions generated by the proposed learning algorithm. Section 6 shows the preliminary experimental results. Related papers are presented in Section 7. Finally, we conclude the paper in Section 8.