Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Automated Technology for Verification and Analysis

ATVA 2019: Automated Technology for Verification and Analysis pp 294–313Cite as

Synthesizing Efficient Low-Precision Kernels

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 11781))

Abstract

In this paper, we present a fully automated approach for synthesizing fast numerical kernels with guaranteed error bounds. The kernels we target contain elementary functions such as sine and logarithm, which are widely used in scientific computing, embedded as well as machine-learning programs. However, standard library implementations of these functions are often overly accurate and therefore unnecessarily expensive. Our approach trades superfluous accuracy against performance by approximating elementary function calls by polynomials and by implementing arithmetic operations in low-precision fixed-point arithmetic. Our algorithm soundly distributes and guarantees an overall error budget specified by the user. The evaluation on benchmarks from different domains shows significant performance improvements of 2.23\(\times \) on average compared to state-of-the-art implementations of such kernel functions.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    These techniques are applied to underlying roundoff error analysis, and our approach can be combined with any sound roundoff error analysis. Therefore, the application domain for our technique is only limited by what a roundoff error analysis can handle. For programs with discrete decisions – like machine-learning classifiers – the effect of the approximations on decision errors can be obtained experimentally, or to a limited extent via static analysis [31].

  2. 2.

    We optimize for running time, but our algorithm is also applicable to other objectives such as energy, with an appropriate cost function. We note that running time often correlates with energy.

  3. 3.

    All cost functions are available in the source code in repo/opt/CostFunctions.scala.

References

  1. Project CORPIN. https://www-sop.inria.fr/corpin/logiciels/ALIAS/Benches/

  2. Python sklearn - multi-layer perceptron regressor (2019). https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html

  3. Alur, R., et al.: Syntax-guided synthesis. In: FMCAD, pp. 1–8. IEEE (2013)

    Google Scholar 

  4. Alur, R., Radhakrishna, A., Udupa, A.: Scaling enumerative program synthesis via divide and conquer. In: Legay, A., Margaria, T. (eds.) TACAS 2017. LNCS, vol. 10205, pp. 319–336. Springer, Heidelberg (2017). https://doi.org/10.1007/978-3-662-54577-5_18

    Chapter  Google Scholar 

  5. Ansel, J., Wong, Y.L., Chan, C., Olszewski, M., Edelman, A., Amarasinghe, S.: Language and compiler support for auto-tuning variable-accuracy algorithms. In: CGO (2011)

    Google Scholar 

  6. Bodik, R., Jobstmann, B.: Algorithmic program synthesis: introduction. STTT 15(5), 397–411 (2013)

    Article  Google Scholar 

  7. Bornholt, J., Torlak, E., Grossman, D., Ceze, L.: Optimizing Synthesis with Metasketches. In: POPL (2016)

    Article  Google Scholar 

  8. Brunie, N., De Dinechin, F., Kupriianova, O., Lauter, C.: Code generators for mathematical functions. In: ARITH (2015)

    Google Scholar 

  9. Chiang, W.F., Baranowski, M., Briggs, I., Solovyev, A., Gopalakrishnan, G., Rakamarić, Z.: Rigorous floating-point mixed-precision tuning. In: POPL (2017)

    Google Scholar 

  10. Damouche, N., Martel, M.: Mixed precision tuning with salsa. In: PECCS, pp. 185–194. SciTePress (2018)

    Google Scholar 

  11. Damouche, N., Martel, M., Chapoutot, A.: Improving the numerical accuracy of programs by automatic transformation. STTT 19(4), 427–448 (2017)

    Article  Google Scholar 

  12. D’Antoni, L., Samanta, R., Singh, R.: Qlose: program repair with quantitative objectives. In: Chaudhuri, S., Farzan, A. (eds.) CAV 2016. LNCS, vol. 9780, pp. 383–401. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41540-6_21

    Chapter  Google Scholar 

  13. Darulova, E., Volkova, A.: Sound approximation of programs with elementary functions. In: Dillig, I., Tasiran, S. (eds.) CAV 2019. LNCS, vol. 11562, pp. 174–183. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-25543-5_11

    Chapter  Google Scholar 

  14. Darulova, E., Izycheva, A., Nasir, F., Ritter, F., Becker, H., Bastian, R.: Daisy - framework for analysis and optimization of numerical programs (tool paper). In: Beyer, D., Huisman, M. (eds.) TACAS 2018. LNCS, vol. 10805, pp. 270–287. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-89960-2_15

    Chapter  Google Scholar 

  15. Darulova, E., Kuncak, V.: Towards a compiler for reals. TOPLAS 39(2), 8 (2017)

    Article  Google Scholar 

  16. Darulova, E., Sharma, S., Horn, E.: Sound mixed-precision optimization with rewriting. In: ICCPS (2018)

    Google Scholar 

  17. De Dinechin, F., Lauter, C.Q., Melquiond, G.: Assisted verification of elementary functions using Gappa. In: ACM Symposium on Applied Computing (2006)

    Google Scholar 

  18. Esmaeilzadeh, H., Sampson, A., Ceze, L., Burger, D.: Neural acceleration for general-purpose approximate programs. In: MICRO (2012)

    Google Scholar 

  19. Feng, Y., Martins, R., Wang, Y., Dillig, I., Reps, T.W.: Component-based synthesis for complex APIs. In: POPL (2017)

    Google Scholar 

  20. de Figueiredo, L.H., Stolfi, J.: Affine arithmetic: concepts and applications. Numer. Algorithms 37(1–4), 147–158 (2004)

    Article  MathSciNet  Google Scholar 

  21. Goubault, E., Putot, S.: Static analysis of finite precision computations. In: Jhala, R., Schmidt, D. (eds.) VMCAI 2011. LNCS, vol. 6538, pp. 232–247. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-18275-4_17

    Chapter  Google Scholar 

  22. Gulwani, S.: Automating string processing in spreadsheets using input-output examples. In: POPL (2011)

    Article  Google Scholar 

  23. IEEE: IEEE Standard for Floating-Point Arithmetic. IEEE Std 754–2008 (2008)

    Google Scholar 

  24. Kneuss, E., Kuraj, I., Kuncak, V., Suter, P.: Synthesis modulo recursive functions. In: OOPSLA (2013)

    Google Scholar 

  25. Kuncak, V., Mayer, M., Piskac, R., Suter, P.: Complete functional synthesis. In: PLDI (2010)

    Google Scholar 

  26. Kupriianova, O., Lauter, C.: A domain splitting algorithm for the mathematical functions code generator. In: Asilomar (2014)

    Google Scholar 

  27. Kupriianova, O., Lauter, C.: Metalibm: a mathematical functions code generator. In: Hong, H., Yap, C. (eds.) ICMS 2014. LNCS, vol. 8592, pp. 713–717. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44199-2_106

    Chapter  MATH  Google Scholar 

  28. Lee, D.U., Gaffar, A.A., Cheung, R.C., Mencer, O., Luk, W., Constantinides, G.A.: Accuracy-guaranteed bit-width optimization. TCAD 25(10), 1990–2000 (2006)

    Google Scholar 

  29. Lee, S., John, L.K., Gerstlauer, A.: High-level synthesis of approximate hardware under joint precision and voltage scaling. In: DATE, pp. 187–192 (2017)

    Google Scholar 

  30. Lee, W., Sharma, R., Aiken, A.: On automatically proving the correctness of math.h implementations. Proc. ACM Program. Lang. 2(POPL), 47 (2018)

    Google Scholar 

  31. Lohar, D., Darulova, E., Putot, S., Goubault, E.: Discrete choice in the presence of numerical uncertainties. IEEE TCAD 37, 2381–2392 (2018)

    Google Scholar 

  32. Loncaric, C., Torlak, E., Ernst, M.D.: Fast synthesis of fast collections. ACM SIGPLAN Not. 51(6), 355–368 (2016)

    Article  Google Scholar 

  33. Magron, V., Constantinides, G., Donaldson, A.: Certified roundoff error bounds using semidefinite programming. ACM Trans. Math. Softw. 43(4), 34 (2017)

    Article  MathSciNet  Google Scholar 

  34. Misailovic, S., Carbin, M., Achour, S., Qi, Z., Rinard, M.C.: Chisel: reliability- and accuracy-aware optimization of approximate computational kernels. In: OOPSLA (2014)

    Google Scholar 

  35. Moore, R.: Interval Analysis. Prentice-Hall, Upper Saddle River (1966)

    MATH  Google Scholar 

  36. Moscato, M., Titolo, L., Dutle, A., Muñoz, C.A.: Automatic estimation of verified floating-point round-off errors via static analysis. In: Tonetta, S., Schoitsch, E., Bitsch, F. (eds.) SAFECOMP 2017. LNCS, vol. 10488, pp. 213–229. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66266-4_14

    Chapter  Google Scholar 

  37. Briesbarre, N., Chevillard, S.: Efficient polynomial l-approximations. In: ARITH (2007)

    Google Scholar 

  38. Neider, D., Saha, S., Madhusudan, P.: Compositional synthesis of piece-wise functions by learning classifiers. ACM Trans. Comput. Logic 19(2), 10:1–10:23 (2018)

    Article  MathSciNet  Google Scholar 

  39. Nori, A.V., Ozair, S., Rajamani, S.K., Vijaykeerthy, D.: Efficient synthesis of probabilistic programs. In: PLDI. ACM (2015)

    Article  Google Scholar 

  40. Braojos, R., Ansaloni, G., Atienza, D.: A methodology for embedded classification of heartbeats using random projections. In: DATE. EPFL (2013)

    Google Scholar 

  41. Renganarayana, L., Srinivasan, V., Nair, R., Prener, D.: Programming with relaxed synchronization. In: RACES, pp. 41–50 (2012)

    Google Scholar 

  42. Schkufza, E., Sharma, R., Aiken, A.: Stochastic optimization of floating-point programs with tunable precision. In: PLDI (2014)

    Google Scholar 

  43. Sidiroglou-Douskos, S., Misailovic, S., Hoffmann, H., Rinard, M.: Managing performance vs. accuracy trade-offs with loop perforation. In: ESEC/FSE (2011)

    Google Scholar 

  44. Solovyev, A., Jacobsen, C., Rakamaric, Z., Gopalakrishnan, G.: Rigorous estimation of floating-point round-off errors with symbolic taylor expansions. In: FM (2015)

    Google Scholar 

  45. Xilinx: Vivado design suite (2018). https://www.xilinx.com/products/design-tools/vivado.html

  46. Xu, Q., Mytkowicz, T., Kim, N.S.: Approximate computing: a survey. IEEE Des. Test 33(1), 8–22 (2016)

    Article  Google Scholar 

  47. Yazdanbakhsh, A., Mahajan, D., Esmaeilzadeh, H., Lotfi-Kamran, P.: AxBench: a multiplatform benchmark suite for approximate computing. IEEE Des. Test 34(2), 60–68 (2017)

    Article  Google Scholar 

  48. Yi, X., Chen, L., Mao, X., Ji, T.: Efficient automated repair of high floating-point errors in numerical libraries. In: POPL (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fakultät für Informatik, TU München, Münich, Germany

    Anastasiia Izycheva & Helmut Seidl

  2. MPI-SWS, Saarland Informatics Campus, Saarbrücken, Germany

    Eva Darulova

Authors
  1. Anastasiia Izycheva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eva Darulova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Helmut Seidl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anastasiia Izycheva .

Editor information

Editors and Affiliations

  1. Institute of Information Science, Academia Sinica, Taipei, Taiwan

    Yu-Fang Chen

  2. DENSO AUTOMOTIVE Deutschland GmbH, Eching, Germany

    Chih-Hong Cheng

  3. Institute of Computer Science, TU München, Munich, Germany

    Javier Esparza

A Benchmarks

A Benchmarks

All benchmarks are provided below. Error specification given in the ensuring clause corresponds to the small error, the larger errors are given in comments.

figure b
figure c
figure d

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Izycheva, A., Darulova, E., Seidl, H. (2019). Synthesizing Efficient Low-Precision Kernels. In: Chen, YF., Cheng, CH., Esparza, J. (eds) Automated Technology for Verification and Analysis. ATVA 2019. Lecture Notes in Computer Science(), vol 11781. Springer, Cham. https://doi.org/10.1007/978-3-030-31784-3_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31784-3_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31783-6

  • Online ISBN: 978-3-030-31784-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation