Embedded System Paranoia: a tool for testing embedded system arithmetic
Introduction
The reliable implementation of a numerical algorithm depends fundamentally on the underlying quality of the arithmetic implementation. Unfortunately, there are many examples of significant failures in implementation over the years. For example, a version of the CDC Fortran compiler reported 1.0—1.0 as being less than, equal to and greater than zero simultaneously [12]. Naturally such deviations make life hard for the algorithmic programmer and the problem has been addressed successfully by a number of authors over the years [8], [2], [10] as a result of which effective standardised approaches have appeared [5], [6]. These together with tools for diagnosing arithmetic problems have led to a gradual improvement in the quality of implementation of arithmetic such that today in general purpose systems, arithmetic quality is usually quite good, although there still remain significant concerns [9].
Perhaps the greatest concerns today are, however, associated with embedded control systems. Such tools as have appeared with the goal of diagnosing arithmetic implementation problems, have not in general been available for such systems and the average quality of arithmetic implementations for this environment therefore remains unknown.
Embedded control systems are of course at the heart of modern electronic system development. Twenty years ago, an embedded control system might have contained 2K of ROM, a simple 4 bit CPU such as the 74181 and be entirely coded in machine code. In general they controlled very simple devices and few demands were placed on them to implement high quality arithmetic. Today, things are completely different. Embedded control systems are in just about every consumer product from an electric toaster to an automobile. Not only that but the systems are as sophisticated as general purpose systems with in some cases, many megabytes of RAM, IDE discs, high end 32 bit microprocessors and are required to solve complex algorithms in real time such as coupled differential equations. Such systems are commonly programmed in C and can constitute millions of lines of code. Consequently, the demands on the arithmetic system are as high as in general purpose systems and the distinction between the two types of system becomes increasingly more blurred each year.
After a review of previous tools designed to measure arithmetic quality, the steps necessary to re-structure paranoia will be described, a sample output shown and the results of running the re-structured program on a number of different systems will be tabulated. A discussion of the role of extended precision computation will follow and some conclusions noted.
Section snippets
Tools for measuring arithmetic quality
A number of tools of greater or lesser sophistication have emerged over the years with the object of measuring arithmetic quality in some way. These vary from simply diagnosing important properties of the implementation such as the radix to tools capable of diagnosing a much wider class of problems.
ES Paranoia
For all kinds of reasons, optimisation capabilities, small footprint, generally lightweight compiler and environment and plentiful available skill, C has been the dominant language of embedded system control in the last 10 years. However, embedded control systems often do not implement the full ISO C standard, nor do they have to. Aware of the heavy use in the embedded system world where originally at least, memory and processing resources were at a premium, the ISO C committee had deliberately
Running ESP on a general purpose machine
ESP is now a batch program with no user intervention. As an example, it was built to assume the presence of signal.h, setjmp.h, and stdio.h and allowed to try to divide by zero. It then produced the following output on a SuSE 9.0 Linux machine running gcc version 3.3.1.
The size of the generated object module is around 68K with the GNU compiler excluding libraries.
Results of running ESP on real systems
The following results show what happens with real systems. They are shown in tabular form along with an explanation of the environment under which they were run. In one case, ESP could only be run on the simulator for space reasons.
Some comments on extra precision calculations
Adrian Capel and Simon Dalley of UKAEA Culham provided the author with important feedbacks about the treatment of machines which calculate results with higher internal precision than requested such as the x86 architecture which uses extended precision (80 active bits) for the actual calculations and floating-point registers. This is because although the actual test variables are sized float or double according to the compile option, the intermediate subexpressions are calculated to 80 bits.
Downloading ESP
ESP is available for free download from the author's personal site, http://www.leshatton.org/ as a zipped file containg a README document, the source esparanoia.c and a sample output. The author welcomes results from different systems and will endeavour to collate them for easy access on the above site.
Conclusions
A long overdue and enhanced version of paranoia has been made freely available to allow embedded control systems to be tested as well as conventional systems. Amongst other things, the re-structuring removes the former interactivity and makes various facilities of the C language optionally elidable depending on local support. In addition, a number of new functions have been added to increase the repertoire of tests including hyperbolic and transcendental tests.
So far results suggest that it is
Acknowledgements
The author would like to acknowledge the help of Adrian Capel and Simon Dalley of UKAEA Culham, Sivasankaran Krishnan, Sukumar Ranjeethkumar and Vibin Viswanbharan (Visteon India), Jurg Sturli (WORX) and Chris Tapp (Keylevel Consultants) for kindly taking the time to adapt, compile and run ESP on the systems shown.
Finally, the author would like to acknowledge the pioneering work of the original authors. The continuing existence of compiler/chip combinations which fail this test bears mute
References (13)
- et al.
Software Manual for the Elementary Functions
(1980) MACHAR: a subroutine to dynamically determine machine parameters
ACM Transactions on Mathematical Software
(1988)Safer C: Developing Software for High Integrity and Safety Critical Systems
(1995)- L. Hatton, EC—a measurement based safer subset of ISO C suitable for embedded system development, Accepted by IST,...
- (1985)
- (1987)
Cited by (3)
The quality attribute for embedded system and application
2013, Applied Mechanics and MaterialsConstraint-based test-scheduling of embedded microprocessors
2008, Proceedings of the Argentine School of Micro-Nanoelectronics, Technology and Applications 2008, EAMTADynamic scheduling of test routines for efficient online self-testing of embedded microprocessors
2008, Proceedings - 14th IEEE International On-Line Testing Symposium, IOLTS 2008