Systematic Assessment of Formal Methods Based Models Quality Criteria

Abstract

When presented with two fully proved formal methods-based specifications, how can a System Engineer decide which is superior when both models specify the same requirements, but in two different ways? This paper investigates and propose a methodology by which formal methods (using the specific example of the Event-B notation) can be differentiated in terms of their quality, using criteria that may be highly subjective in nature. Established complexity functions applied to software are not applicable to formal methods, thus the paper proposes a new function which quantifies the “quality” of a given model. Complexity is not the only factor involved in determining the quality of formal methods, the quality of system thinking involved also play an impactful role. We propose a quality function which uses the well-established properties of axiomatic systems in theoretical mathematics with the addition of a specifically formulated complexity function. The distinction criteria are based on evaluating how four main properties have been achieved: “Consistency”, “Completeness”, “Independence” and “Complexity”. We base our approach according to the paradigm of; “if the formal specification looks visually complicated for a set-theory novice, then it is a poorly modeled specification”. Furthermore, we explore the notion of Miller’s rule (magic No. 7) to define what “good” should look like. We conclude that we need more than Miller’s 7, we need 1, 2 and 3 to help us with defining what good quality looks like, by taking human cognitive capacity as a benchmark. This novel approach implies considerable further research, described in future work section.