Designing and Testing Cyber-Physical Products 4^th Generation Product Management Based on AHP and QFD

Abstract

Key element of 4^th generation cyber-physical products is the dominant impact of software on product features. Another characteristic of these products is its high complexity level as a system of system, and their ability to physically harm humans or the environment.

Today, manufacturers of highly complex systems, such as airplanes or trainsets, face problems both in designing and in testing their products. The system components do not interact smoothly and flawlessly. Testing a system whose software functionality size exceeds that of traditional software-only products is hardly possible with today’s methods. Good product design is even harder.

For solving the problem, the software build into the product must be measured, equally with physical measurements in mechanics for addressing issues of reliability, safety, and delighting users. Based on these measurements, the methods explained in the series of standards ISO 16355 lay the fundamentals by using AHP and QFD for 4^th generation product management.

Appendix

1.1 The Double-Tiddlemutzz Example

The Double-Decker Tilting Long-Distance Multiple Unit Trainset (D²TLDMUTS) serves as an example to explain the new concepts. For the details of the example, see [16, p. 103]. It refers to a large multiple unit trainset, able to run on international railway traffic as a double-decker with restaurant, with children’s corner, offering space for people with disabilities, featuring roll compensation for faster driving around a curve, comfortable enough for three to six hours of daytime train riding. Thus, it should respond to many different – and conflicting – needs.

However, when commissioning the D²TLDMUTS, the number of bugs found, and problems encountered, piled high. The reason for the problems is basic: it was virtually impossible to create complete test suites for such a complex, software-intense system. Consequently, the necessary functional testing was done during commissioning, for many more years than originally planned (Fig. 7).

Fig. 7.

The complete analytic hierarchy process for the D2TLDMUTS

Figure 7 shows The Complete Analytic Hierarchy Process for the D2TLDMUTS. The operator’s needs regarding the new D²TLDMUTS are hierarchically grouped and analyzed using the AHP, with one level of hierarchy. The hierarchy reflects those subsystems of the D²TLDMUTS that had been tested thoroughly. For testing, each subsystem will need its own operator’s needs for defining the goals of tests and test coverage.

1.2 Using the AHP Hierarchy for Combining Test Stories

Let ${{\varvec{A}}}_{1}, {{\varvec{A}}}_{2}, \dots ,{{\varvec{A}}}_{k}$ be a sequence of AHP pairwise decision matrices with solution profiles ${\mathrm{y}}_{1}, {\mathrm{y}}_{2}, \dots , {\mathrm{y}}_{\mathrm{k}}$ respectively; $\mathrm{k}\in \mathrm{N}$; $\mathrm{k}>0$. Thus, up to the convergence gap, ${{\varvec{A}}}_{{\varvec{i}}}{{\varvec{y}}}_{{\varvec{i}}}\cong {{\varvec{y}}}_{{\varvec{I}}}$, for $i=1,\dots ,k$. because there are no algebraic solutions for eigenvectors.

Let $\overline{{\varvec{A}} }$ be the Comparison Hierarchy with the solution profile $\overline{{\varvec{y}} }$. $\overline{{\varvec{A}} }$ is a $k\times k$ square matrix; with $\overline{\mathbf{y} }=\langle {\overline{y} }_{1},{\overline{y} }_{2},\dots ,{\overline{y} }_{k},\rangle$ being the solution profile for $\overline{{\varvec{A}} }$.

The combined solution profile for the full AHP is shown in Eq. (4):

$${\varvec{v}}=\sum_{\mathrm{i}=1}^{\mathrm{k}}{\overline{y} }_{i}{{\varvec{y}}}_{i}, i=1,\dots ,k;k\in \mathrm{N};k>0$$

(4)

Full AHP Solution Profile = $\frac{{\varvec{v}} }{\Vert {\varvec{v}}\Vert }$

Equation (4) denotes a sum of profile vectors, divided by its Euclidean length; thus, the normalized vector $\boldsymbol{ }{\varvec{v}}/\| {\varvec{v}}\|$ becomes yet another profile.

1.3 The Initial Sparse Matrix Testing Component Parts

Each of the ${{\varvec{A}}}_{{\varvec{i}}}$ pairwise decision matrices describe the needs of the customer with respect to its part, be it ETCS, door control, communications. For the user stories, a profile results that describes the importance of the functionality described by the user story to the customer in view of the stated needs ${{\varvec{y}}}_{i}$. Let ${{\varvec{u}}}_{\mathrm{i}}$ describe this profile. For profile ${{\varvec{u}}}_{\mathrm{i}}$, an initial set of test stories is needed to cover the user stories with tests, together with an initial sample starting set of test cases.

Fig. 8.

The Sparse Test Coverage Matrix combining Component Parts’ Tests

The resulting test coverage matrices ${{\varvec{F}}}_{i}$ map test stories onto user stories, again based on the data movements executed in the respective test cases.

The combined goal profile for the combined test coverage matrix is

$$\overline{{\varvec{y}} }=\langle {v}_{1}{{\varvec{u}}}_{1},{v}_{2}{{\varvec{u}}}_{2},\cdots ,{v}_{k},{{\varvec{u}}}_{k}\rangle $$

(5)

Full Test Coverage Goal Profile = $\frac{{\varvec{y}}}{\Vert \overline{{\varvec{y}}}\Vert }$

The combined SSTF matrices usually achieve an inadequate convergence gap only, even for the three sample component parts selected (Fig. 8). This reflects the fact that interacting components are not well tested.

1.4 Combining the Parts’ Tests

Applying the test case combination routine of ART for the empty parts of the full matrix yields additional test cases that can be selected for relevance (Fig. 9).

Fig. 9.

Filling up the test coverage matrix with combined test cases

For this ART run, we kept the diagonal parts’ matrices fixed. This is not necessarily so. The additional tests cover test stories with user stories form other components, and thus are specifically targeted towards detecting defects in parts’ cooperation.

The creation of new test cases can be repeated until test density is good enough to satisfy reliability, safety and security needs for the new product. Long commissioning can be avoided. How many tests are needed, and what test density is appropriate, depends from the product and is part of 4^th generation product management.