Leveraging Abstraction to Establish Out-of-Nominal Safety Properties

Abstract

Digital systems in an out-of-nominal environment (e.g., one causing hardware bit flips) may not be expected to function correctly in all respects but may be required to fail safely. We present an approach for understanding and verifying a system’s out-of-nominal behavior as an abstraction of nominal behavior that preserves designated critical safety requirements. Because abstraction and refinement are already widely used for improved tractability in formal design and proof techniques, this additional way of viewing an abstraction can potentially verify a system’s out-of-nominal safety with little additional work. We illustrate the approach with a simple model of a turnstile controller with possible logic faults (formalized in the temporal logic of actions and NuSMV), noting how design choices can be guided by the desired out-of-nominal abstraction. Principles of robustness in complex systems (specifically, Boolean networks) are found to be compatible with the formal abstraction approach. This work indicates a direction for broader use of formal methods in safety-critical systems.

Appendices

A High-Level Model for Turnstile in TLA

As discussed in Sect. 3 and Sect. 4.1, TLA+ is used to specify and verify both nominal and out-of-nominal models for the turnstile example. The nominal model is shown in Fig. 4 and the out-of-nominal model in Fig. 5. Both models have three variables $lock$, $coin$, and $push$, corresponding to the variables L, C, and P described in Sect. 3. The specifications are given in the idiomatic TLA+ style: $Init$ constrains the initial conditions, $Next$ describes the “next step” relation, and $Spec$ expresses the complete temporal logic specification [11].

The relation $Next$ is defined by existential quantification over parameters c and p, representing new values of $coin$ and $push$ in the relation $Step$. This somewhat contorted idiom is used because a step must completely describe the evolution of each variable. The existential expresses that $coin$ and $push$ may each evolve nondeterministically at each step.

The property $TypeInvariant$ states that each variable is limited to Boolean values, while $Safety$ expresses the set of safety properties drawn from S1 through S4 that apply to each model. In the nominal model, $OutOfNominalSpec$ imports the out-of-nominal specification for use in proving refinement (see Sect. 4.1). The type invariant, safety, and refinement properties were checked for correctness using the TLC model checker.

Fig. 4.

TLA+ specification for the nominal turnstile.

Fig. 5.

TLA+ specification for the out-of-nominal turnstile.

B Boolean Network Model for Turnstile in NuSMV

As described in Sect. 4.2, the NuSMV model checker is used to verify the robustness of the tiered combinational logic implementing the safety-critical term $\lnot C \wedge L$, along the lines of previous work [12]. The inputs are taken as node 0 ($\lnot C$) and node 1 (L), and the output is taken as node 18. The Boolean network (BN) is checked for conformance to the abstraction in the presence of any single internal bit flip in one of the first $n_{\text {max}}$ tiers, where $n_{\text {max}} \in \{1,\dots ,20\}$. Figure 6 shows an extract from the model in the case where node 2 can be flipped and $n_{\text {max}} = 14$. It is found that the quiescent BN is immune to any single bit flip up to $n_{\text {max}} = 15$, whereas the chaotic BN can be corrupted by a single bit flip for any value of $n_{\text {max}}$.

Fig. 6.

Extract from a NuSMV model that is programmatically generated so that all tiers and all nodes can be checked for susceptibility to bit flips. The linear temporal logic (LTL) property at the end expresses conformance of the out-of-nominal output to the abstraction $\lnot C \wedge L$.