Embedded System Paranoia: a tool for testing embedded system arithmetic

https://doi.org/10.1016/j.infsof.2004.10.005Get rights and content

Abstract

The quality of arithmetic implementation is of concern to all who work with or depend on the results of numerical computations. Embedded systems have become enormously complicated and widespread in most if not all consumer devices in recent years so there is a clear need to measure the quality of the arithmetic in the same way that conventional systems have been measured for some time using programs such as the well-known paranoia. A new version of paranoia has been introduced specifically to extend the domain of testable systems to embedded control systems. This paper describes the development of ESP (Embedded System Paranoia) and gives example outputs and free download sites. The example outputs indicate that even today, the quality of arithmetic implementations cannot be taken for granted with numerous implementation problems being reported in those embedded environments tried so far.

Introduction

The reliable implementation of a numerical algorithm depends fundamentally on the underlying quality of the arithmetic implementation. Unfortunately, there are many examples of significant failures in implementation over the years. For example, a version of the CDC Fortran compiler reported 1.0—1.0 as being less than, equal to and greater than zero simultaneously [12]. Naturally such deviations make life hard for the algorithmic programmer and the problem has been addressed successfully by a number of authors over the years [8], [2], [10] as a result of which effective standardised approaches have appeared [5], [6]. These together with tools for diagnosing arithmetic problems have led to a gradual improvement in the quality of implementation of arithmetic such that today in general purpose systems, arithmetic quality is usually quite good, although there still remain significant concerns [9].

Perhaps the greatest concerns today are, however, associated with embedded control systems. Such tools as have appeared with the goal of diagnosing arithmetic implementation problems, have not in general been available for such systems and the average quality of arithmetic implementations for this environment therefore remains unknown.

Embedded control systems are of course at the heart of modern electronic system development. Twenty years ago, an embedded control system might have contained 2K of ROM, a simple 4 bit CPU such as the 74181 and be entirely coded in machine code. In general they controlled very simple devices and few demands were placed on them to implement high quality arithmetic. Today, things are completely different. Embedded control systems are in just about every consumer product from an electric toaster to an automobile. Not only that but the systems are as sophisticated as general purpose systems with in some cases, many megabytes of RAM, IDE discs, high end 32 bit microprocessors and are required to solve complex algorithms in real time such as coupled differential equations. Such systems are commonly programmed in C and can constitute millions of lines of code. Consequently, the demands on the arithmetic system are as high as in general purpose systems and the distinction between the two types of system becomes increasingly more blurred each year.

After a review of previous tools designed to measure arithmetic quality, the steps necessary to re-structure paranoia will be described, a sample output shown and the results of running the re-structured program on a number of different systems will be tabulated. A discussion of the role of extended precision computation will follow and some conclusions noted.

Section snippets

Tools for measuring arithmetic quality

A number of tools of greater or lesser sophistication have emerged over the years with the object of measuring arithmetic quality in some way. These vary from simply diagnosing important properties of the implementation such as the radix to tools capable of diagnosing a much wider class of problems.

ES Paranoia

For all kinds of reasons, optimisation capabilities, small footprint, generally lightweight compiler and environment and plentiful available skill, C has been the dominant language of embedded system control in the last 10 years. However, embedded control systems often do not implement the full ISO C standard, nor do they have to. Aware of the heavy use in the embedded system world where originally at least, memory and processing resources were at a premium, the ISO C committee had deliberately

Running ESP on a general purpose machine

ESP is now a batch program with no user intervention. As an example, it was built to assume the presence of signal.h, setjmp.h, and stdio.h and allowed to try to divide by zero. It then produced the following output on a SuSE 9.0 Linux machine running gcc version 3.3.1.

The size of the generated object module is around 68K with the GNU compiler excluding libraries.

Results of running ESP on real systems

The following results show what happens with real systems. They are shown in tabular form along with an explanation of the environment under which they were run. In one case, ESP could only be run on the simulator for space reasons.

Some comments on extra precision calculations

Adrian Capel and Simon Dalley of UKAEA Culham provided the author with important feedbacks about the treatment of machines which calculate results with higher internal precision than requested such as the x86 architecture which uses extended precision (80 active bits) for the actual calculations and floating-point registers. This is because although the actual test variables are sized float or double according to the compile option, the intermediate subexpressions are calculated to 80 bits.

Downloading ESP

ESP is available for free download from the author's personal site, http://www.leshatton.org/ as a zipped file containg a README document, the source esparanoia.c and a sample output. The author welcomes results from different systems and will endeavour to collate them for easy access on the above site.

Conclusions

A long overdue and enhanced version of paranoia has been made freely available to allow embedded control systems to be tested as well as conventional systems. Amongst other things, the re-structuring removes the former interactivity and makes various facilities of the C language optionally elidable depending on local support. In addition, a number of new functions have been added to increase the repertoire of tests including hyperbolic and transcendental tests.

So far results suggest that it is

Acknowledgements

The author would like to acknowledge the help of Adrian Capel and Simon Dalley of UKAEA Culham, Sivasankaran Krishnan, Sukumar Ranjeethkumar and Vibin Viswanbharan (Visteon India), Jurg Sturli (WORX) and Chris Tapp (Keylevel Consultants) for kindly taking the time to adapt, compile and run ESP on the systems shown.

Finally, the author would like to acknowledge the pioneering work of the original authors. The continuing existence of compiler/chip combinations which fail this test bears mute

References (13)

  • W.J. Cody et al.

    Software Manual for the Elementary Functions

    (1980)
  • W.J. Cody

    MACHAR: a subroutine to dynamically determine machine parameters

    ACM Transactions on Mathematical Software

    (1988)
  • L. Hatton

    Safer C: Developing Software for High Integrity and Safety Critical Systems

    (1995)
  • L. Hatton, EC—a measurement based safer subset of ISO C suitable for embedded system development, Accepted by IST,...
  • IEEE
    (1985)
  • IEEE
    (1987)
There are more references available in the full text version of this article.

Cited by (3)

View full text