Anxiang Xiao
ABSTRACT
Floating point programs are widely used in science and engineering. These programs approximate real arithmetic with floating point arithmetic, which introduces rounding errors and leads to inaccurate results. In life-critical software systems, inaccuracy can lead to catastrophes. To avoid these tragedies, program rewriting techniques have been proposed to automatically improve accuracy for floating point programs. Unfortunately, efficiency can be the bottleneck of such techniques due to the lack of effective pruning strategy.
We introduce a heuristic pruning strategy called Memory Based Pruning to accelerate these rewrite-based approaches. The heuristic comes from an observation that floating point programs with similar computation steps usually suffer from similar inaccuracy problems, therefore they can be improved by the same or similar rewriting rules. With Memory Based Pruning, rewriting rules used in accuracy improvement are memorized in a knowledge memory. While improving accuracy for similar programs, these rules can help to guide the search and prune away probably useless rewriting rules. Consequently, a large amount of rule examinations can be avoided and the improving efficiency is significantly promoted. Based on the above approach, we have implemented a tool called HMBP and evaluated it on FPBench. In the evaluation, we compare the efficiency of HMBP and the well-known accuracy-improving tool Herbie. The results show that HMBP is 2.32x faster than Herbie in average. For the best case, the speedup is up to 29.64x.
- Christophe Andrieu, Nando de Freitas, Arnaud Doucet, and Michael I. Jordan. 2003. An Introduction to MCMC for Machine Learning. In Machine Learning. 5–43.Google Scholar
- Tao Bao and Xiangyu Zhang. 2013. On-the-fly Detection of Instability Problems in Floating-point Program Execution. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA 2013. 817–832.Google ScholarDigital Library
- Florian Benz, Andreas Hildebrandt, and Sebastian Hack. 2012. A Dynamic Program Analysis to Find Floating-point Accuracy Problems. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2012. 453–462.Google ScholarDigital Library
- Wei-Fan Chiang, Ganesh Gopalakrishnan, Zvonimir Rakamaric, and Alexey Solovyev. 2014. Efficient Search for Inputs Causing High Floating-point Errors. In ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2014. 43–52.Google Scholar
- Rémi Coulom. 2006. Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In Computers and Games, 5th International Conference, CG 2006. 72–83.Google Scholar
- Nasrine Damouche, Matthieu Martel, Pavel Panchekha, Chen Qiu, Alexander Sanchez-Stern, and Zachary Tatlock. 2016. Toward a Standard Benchmark Format and Suite for Floating-Point Analysis. In Numerical Software Verification - 9th International Workshop, NSV 2016. 63–77.Google Scholar
- Laurent Fousse, Guillaume Hanrot, Vincent Lefèvre, Patrick Pélissier, and Paul Zimmermann. 2007. MPFR: A Multiple-precision Binary Floating-point Library with Correct Rounding. In ACM Transactions on Mathematical Software, TOMS 2007. 13.Google Scholar
- David Goldberg. 1991. What Every Computer Scientist Should Know About Floating-Point Arithmetic. In ACM Computing Surveys. 5–48.Google Scholar
- Richard Hamming. 2012. Numerical methods for scientists and engineers. Courier Corporation.Google Scholar
- Nicholas J. Higham. 2002. Accuracy and stability of numerical algorithms, Second Edition. SIAM.Google Scholar
- Donald E. Knuth and Ronald W. Moore. 1975. An Analysis of Alpha-Beta Pruning. In Artificial intelligence. 293–326.Google Scholar
- Levente Kocsis and Csaba Szepesvári. 2006. Bandit Based Monte-Carlo Planning. In Proceedings of the 17th European Conference on Machine Learning, ECML 2006. 282–293.Google ScholarDigital Library
- Peter Larsson. 2013. Exploring Quadruple Precision Floating Point Numbers in GCC and ICC. https://www.nsc.liu.se/~pla/blog/2013/10/17/quadruple-precision/Google Scholar
- Jacques-Louis Lions, Lennart Luebeck, Jean-Luc Fauquembergue, Kahn Gilles, Wolfgang Kubbat, Stefan Levedag, Leonardo Mazzini, Didier Merle, and Colin O’Halloran. 1996. Ariane 5 flight 501 failure report by the inquiry board. (1996).Google Scholar
- Peter W. Markstein. 2008. The New IEEE-754 Standard for Floating Point Arithmetic. In Numerical Validation in Current Hardware Architectures.Google Scholar
- Bruce D. McCullough and Hrishikesh D. Vinod. 1999. The Numerical Reliability of Econometric Software. In Journal of Economic Literature. 633–665.Google Scholar
- Nicholas Nethercote and Julian Seward. 2007. Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2007. 89–100.Google ScholarDigital Library
- Pavel Panchekha, Alex Sanchez-Stern, James R. Wilcox, and Zachary Tatlock. 2015. Automatically Improving Accuracy for Floating Point Expressions. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2015. 1–11.Google ScholarDigital Library
- Kevin Quinn. 1983. Ever had problems rounding off figures? This stock exchange has. In The Wall Street Journal. 37.Google Scholar
- Alex Sanchez-Stern, Pavel Panchekha, Sorin Lerner, and Zachary Tatlock. 2018. Finding Root Causes of Floating Point Error. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2018. 256–269.Google ScholarDigital Library
- Robert Skeel. 1992. Roundoff error and the Patriot missile. In SIAM News. 11.Google Scholar
- Rainer Storn and Kenneth V. Price. 1997. Differential Evolution - A Simple and Efficient Heuristic for global Optimization over Continuous Spaces. In Journal of Global Optimization. 341–359.Google Scholar
- Enyi Tang, Xiangyu Zhang, Norbert Th. Müller, Zhenyu Chen, and Xuandong Li. 2017. Software Numerical Instability Detection and Diagnosis by Combining Stochastic and Infinite-Precision Testing. In IEEE Transactions on Software Engineering, TSE 2017. 975–994.Google Scholar
- Ran Wang, Daming Zou, Xinrui He, Yingfei Xiong, Lu Zhang, and Gang Huang. 2016. Detecting and Fixing Precision-specific Operations for Measuring Floating-point Errors. In Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016. 619–630.Google ScholarDigital Library
- Xie Wang, Huaijin Wang, Zhendong Su, Enyi Tang, Xin Chen, Weijun Shen, Zhenyu Chen, Linzhang Wang, Xianpei Zhang, and Xuandong Li. 2019. Global Optimization of Numerical Programs via Prioritized Stochastic Algebraic Transformations. In Proceedings of the 41st International Conference on Software Engineering, ICSE 2019. 1131–1141.Google ScholarDigital Library
- Debora Weber-Wulff. 1992. Rounding error changes Parliament makeup. In The Risks Digest.Google Scholar
- Xin Yi, Liqian Chen, Xiaoguang Mao, and Tao Ji. 2017. Efficient Global Search for Inputs Triggering High Floating-Point Inaccuracies. In 24th Asia-Pacific Software Engineering Conference, APSEC 2017. 11–20.Google Scholar
- Xin Yi, Liqian Chen, Xiaoguang Mao, and Tao Ji. 2019. Efficient Automated Repair of High Floating-point Errors in Numerical Libraries. In Proceedings of the ACM on Programming Languages, POPL 2019. 1–29.Google Scholar
- Daming Zou, Muhan Zeng, Yingfei Xiong, Zhoulai Fu, Lu Zhang, and Zhendong Su. 2020. Detecting Floating-point Errors via Atomic Conditions. In Proceedings of the ACM on Programming Languages, POPL 2019. 1–27.Google Scholar
Index Terms
- Accelerating Accuracy Improvement for Floating Point Programs via Memory Based Pruning
Recommendations
Automatically improving accuracy for floating point expressions
PLDI '15Scientific and engineering applications depend on floating point arithmetic to approximate real arithmetic. This approximation introduces rounding error, which can accumulate to produce unacceptable results. While the numerical methods literature ...
Automatically improving accuracy for floating point expressions
PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and ImplementationScientific and engineering applications depend on floating point arithmetic to approximate real arithmetic. This approximation introduces rounding error, which can accumulate to produce unacceptable results. While the numerical methods literature ...
Decimal Floating-Point Multiplication
Decimal multiplication is important in many commercial applications including financial analysis, banking, tax calculation, currency conversion, insurance, and accounting. This paper presents the design of two decimal floating-point multipliers: one ...
Comments