Alex Sanchez-Stern
Pavel Panchekha
Zachary Tatlock
ABSTRACT
Floating-point arithmetic plays a central role in science, engineering, and finance by enabling developers to approximate real arithmetic. To address numerical issues in large floating-point applications, developers must identify root causes, which is difficult because floating-point errors are generally non-local, non-compositional, and non-uniform.
This paper presents Herbgrind, a tool to help developers identify and address root causes in numerical code written in low-level languages like C/C++ and Fortran. Herbgrind dynamically tracks dependencies between operations and program outputs to avoid false positives and abstracts erroneous computations to simplified program fragments whose improvement can reduce output error. We perform several case studies applying Herbgrind to large, expert-crafted numerical programs and show that it scales to applications spanning hundreds of thousands of lines, correctly handling the low-level details of modern floating point hardware and mathematical libraries and tracking error across function boundaries and through the heap.
Supplemental Material
- Micah Altman, Jeff Gill, and Michael P. McDonald. 2003. Numerical Issues in Statistical Computing for the Social Scientist. Springer-Verlag. 1–11 pages. Google Scholar
Digital Library
- Micah Altman and Michael P. McDonald. 2003. Replication with attention to numerical accuracy. Political Analysis 11, 3 (2003), 302– 307. http://pan.oxfordjournals.org/content/11/3/302.abstractGoogle ScholarCross Ref
- Tao Bao and Xiangyu Zhang. 2013. On-the-fly Detection of Instability Problems in Floating-point Program Execution. SIGPLAN Not. 48, 10 (Oct. 2013), 817–832. Google ScholarDigital Library
- Florian Benz, Andreas Hildebrandt, and Sebastian Hack. 2012. A Dynamic Program Analysis to Find Floating-point Accuracy Problems (PLDI ’12). ACM, New York, NY, USA, 453–462. Google ScholarDigital Library
- Hans-J. Boehm. 2004. The constructive reals as a Java Library. J. Log. Algebr. Program 64 (2004), 3–11.Google ScholarCross Ref
- Wei-Fan Chiang, Mark Baranowski, Ian Briggs, Alexey Solovyev, Ganesh Gopalakrishnan, and Zvonimir Rakamarić. 2017. Rigorous Floating-point Mixed-precision Tuning. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL 2017). ACM, New York, NY, USA, 300–315. Google ScholarDigital Library
- Wei-Fan Chiang, Ganesh Gopalakrishnan, Zvonimir Rakamarić, and Alexey Solovyev. 2014. Efficient Search for Inputs Causing High Floating-point Errors. ACM, 43–52. Google ScholarDigital Library
- Nasrine Damouche, Matthieu Martel, and Alexandre Chapoutot. 2015. Formal Methods for Industrial Critical Systems: 20th International Workshop, FMICS 2015 Oslo, Norway, June 22-23, 2015 Proceedings. (2015), 31–46.Google Scholar
- N. Damouche, M. Martel, and A. Chapoutot. 2015. Transformation of a {PID} Controller for Numerical Accuracy. Electronic Notes in Theoretical Computer Science 317 (2015), 47 – 54. Google ScholarDigital Library
- Nasrine Damouche, Matthieu Martel, Pavel Panchekha, Jason Qiu, Alex Sanchez-Stern, and Zachary Tatlock. 2016. Toward a Standard Benchmark Format and Suite for Floating-Point Analysis. (July 2016).Google Scholar
- Eva Darulova and Viktor Kuncak. 2014. Sound Compilation of Reals (POPL ’14). ACM, New York, NY, USA, 235–248. Google ScholarDigital Library
- François Févotte and Bruno Lathuilière. 2016. VERROU: Assessing Floating-Point Accuracy Without Recompiling. (Oct. 2016). https: //hal.archives-ouvertes.fr/hal-01383417 working paper or preprint.Google Scholar
- Laurent Fousse, Guillaume Hanrot, Vincent Lefèvre, Patrick Pélissier, and Paul Zimmermann. 2007. MPFR: A Multiple-Precision Binary Floating-Point Library with Correct Rounding. ACM Trans. Math. Software 33, 2 (June 2007), 13:1–13:15. Google ScholarDigital Library
- Eric Goubault and Sylvie Putot. 2011. Static Analysis of Finite Precision Computations (VMCAI’11). Springer-Verlag, Berlin, Heidelberg, 232– 247. http://dl.acm.org/citation.cfm?id=1946284.1946301Google Scholar
- Nicholas J. Higham. 2002. Accuracy and Stability of Numerical Algorithms (2nd ed.). Society for Industrial and Applied Mathematics, Philadelphia, PA, USA. Google Scholar
- Andreas Jaeger. 2016. OpenLibm. http://openlibm.org/Google Scholar
- W. Kahan. 1965. Pracniques: Further Remarks on Reducing Truncation Errors. Commun. ACM 8, 1 (Jan. 1965), 40–. Google ScholarDigital Library
- William Kahan. 1971. A Survey of Error Analysis.. In IFIP Congress (2). 1214–1239. http://dblp.uni-trier.de/db/conf/ifip/ifip71-2.html# Kahan71Google Scholar
- W. Kahan. 1987. Branch Cuts for Complex Elementary Functions or Much Ado About Nothing’s Sign Bit. In The State of the Art in Numerical Analysis (Birmingham, 1986), A. Iserles and M. J. D. Powell (Eds.). Inst. Math. Appl. Conf. Ser. New Ser., Vol. 9. Oxford Univ. Press, New York, 165âĂŞ211.Google Scholar
- W. Kahan. 1998. The Improbability of Probabilistic Error Analyses for Numerical Computations. Technical Report. 34 pages. http://www.cs. berkeley.edu/~wkahan/improber.pdfGoogle Scholar
- William Kahan. 2005. Floating-Point Arithmetic Besieged by “Business Decisions”. World-Wide Web lecture notes.., 28 pages. http://www.cs. berkeley.edu/~wkahan/ARITH_17.pdfGoogle Scholar
- C. L. Lawson, R. J. Hanson, D. R. Kincaid, and F. T. Krogh. 1979. Basic Linear Algebra Subprograms for Fortran Usage. ACM Trans. Math. Softw. 5, 3 (Sept. 1979), 308–323. Google ScholarDigital Library
- Vernon A. Lee, Jr. and Hans-J. Boehm. 1990. Optimizing Programs over the Constructive Reals. In Proceedings of the ACM SIGPLAN 1990 Conference on Programming Language Design and Implementation (PLDI ’90). ACM, New York, NY, USA, 102–111. Google ScholarDigital Library
- Matthieu Martel. 2009. Program Transformation for Numerical Precision (PEPM ’09). ACM, New York, NY, USA, 101–110. Google ScholarDigital Library
- B. D. McCullough and H. D. Vinod. 1999. The Numerical Reliability of Econometric Software. Journal of Economic Literature 37, 2 (1999), 633–665.Google ScholarCross Ref
- Marvin L. Minsky. 1967. Computation: Finite and Infinite Machines. Prentice-Hall, Inc., Upper Saddle River, NJ, USA. Google ScholarDigital Library
- Nethercote and Seward. 2007. Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation. (June 2007).Google Scholar
- Dr. K-C Ng. 1993. FDLIBM. http://www.netlib.org/fdlibm/readmeGoogle Scholar
- Pavel Panchekha, Alex Sanchez-Stern, James R. Wilcox, and Zachary Tatlock. 2015. Automatically Improving Accuracy for Floating Point Expressions. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). ACM. Google ScholarDigital Library
- Gordon D. Plotkin. 1970. A note on inductive generalization. Machine Intelligence 5 (1970), 153–163.Google Scholar
- Kevin Quinn. 1983. Ever Had Problems Rounding Off Figures? This Stock Exchange Has. The Wall Street Journal (November 8, 1983), 37.Google Scholar
- Jonathan Richard Shewchuk. 1996. Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator. In Applied Computational Geometry: Towards Geometric Engineering, Ming C. Lin and Dinesh Manocha (Eds.). Lecture Notes in Computer Science, Vol. 1148. Springer-Verlag, 203–222. From the First ACM Workshop on Applied Computational Geometry. Google ScholarDigital Library
- Hang Si. 2015. TetGen, a Delaunay-Based Quality Tetrahedral Mesh Generator. ACM Trans. Math. Softw. 41, 2, Article 11 (Feb. 2015), 36 pages. Google ScholarDigital Library
- Alexey Solovyev, Charlie Jacobsen, Zvonimir Rakamaric, and Ganesh Gopalakrishnan. 2015. Rigorous Estimation of Floating-Point Roundoff Errors with Symbolic Taylor Expansions (FM’15). Springer.Google Scholar
- O. Tange. 2011. GNU Parallel - The Command-Line Power Tool. ;login: The USENIX Magazine 36, 1 (Feb 2011), 42–47. http://www.gnu.org/s/ parallelGoogle Scholar
- U.S. General Accounting Office. 1992. Patriot Missile Defense: Software Problem Led to System Failure at Dhahran, Saudi Arabia. http://www. gao.gov/products/IMTEC-92-26Google Scholar
- Ran Wang, Daming Zou, Xinrui He, Yingfei Xiong, Lu Zhang, and Gang Huang. 2015. Detecting and Fixing Precision-Specific Operations for Measuring Floating-Point Errors (FSE’15). Google ScholarDigital Library
- Debora Weber-Wulff. 1992. Rounding error changes Parliament makeup. http://catless.ncl.ac.uk/Risks/13.37.html#subj4Google Scholar
Index Terms
- Finding root causes of floating point error
Recommendations
Finding root causes of floating point error
PLDI '18Floating-point arithmetic plays a central role in science, engineering, and finance by enabling developers to approximate real arithmetic. To address numerical issues in large floating-point applications, developers must identify root causes, which is ...
Decimal Floating-Point Multiplication
Decimal multiplication is important in many commercial applications including financial analysis, banking, tax calculation, currency conversion, insurance, and accounting. This paper presents the design of two decimal floating-point multipliers: one ...
Low-Cost Binary128 Floating-Point FMA Unit Design with SIMD Support
Binary64 arithmetic is rapidly becoming inadequate to cope with today's large-scale computations due to an accumulation of errors. Therefore, binary128 arithmetic is now required to increase the accuracy and reliability of these computations. At the ...
Comments