skip to main content
10.1145/3650200.3656624acmconferencesArticle/Chapter ViewAbstractPublication PagesicsConference Proceedingsconference-collections
research-article

Accurate Computation of the Logarithm of Modified Bessel Functions on GPUs

Published: 03 June 2024 Publication History

Abstract

Bessel functions are critical in scientific computing for applications such as machine learning, protein structure modeling, and robotics. However, currently, available routines lack precision or fail for certain input ranges, such as when the order v is large, and GPU-specific implementations are limited. We address the precision limitations of current numerical implementations while dramatically improving the runtime. We propose two novel algorithms for computing the logarithm of modified Bessel functions of the first and second kinds by computing intermediate values on a logarithmic scale. Our algorithms are robust and never have issues with underflows or overflows while having relative errors on the order of machine precision, even for inputs where existing libraries fail. In C++/CUDA, our algorithms have median and maximum speedups of 45x and 6150x for GPU and 17x and 3403x for CPU, respectively, over the ranges of inputs and third-party libraries tested. Compared to SciPy, the algorithms have median and maximum speedups of 77x and 300x for GPU and 35x and 98x for CPU, respectively, over the ranges of inputs tested.
The ability to robustly compute a solution and the low relative errors allow us to fit von Mises-Fisher, vMF, distributions to high-dimensional neural network features. This is, e.g., relevant for uncertainty quantification in metric learning. We obtain image feature data by processing CIFAR10 training images with the convolutional layers of a pre-trained ResNet50. We successfully fit vMF distributions to 2048-, 8192-, and 32768-dimensional image feature data using our algorithms. Our approach provides fast and accurate results while existing implementations in the Python libraries SciPy and mpmath fail to fit successfully.
Our approach is readily implementable on GPUs, and we provide a fast open-source implementation alongside this paper.

References

[1]
Wolfram Research 2003. MachinePrecision. Wolfram Research. https://reference.wolfram.com/language/ref/MachinePrecision.htmlAccessed: 2024-01-16.
[2]
Milton Abramowitz and Irene A. Stegun. 1972. Handbook of mathematical functions. US Dept.of Commerce 10 (1972), 1030 pages.
[3]
DEv Amos. 1986. Algorithm 644: A portable package for Bessel functions of a complex argument and nonnegative order. ACM Transactions on Mathematical Software (TOMS) 12, 3 (1986), 265–273.
[4]
Arindam Banerjee, Inderjit S Dhillon, Joydeep Ghosh, Suvrit Sra, and Greg Ridgeway. 2005. Clustering on the Unit Hypersphere using von Mises-Fisher Distributions.Journal of Machine Learning Research 6, 9 (2005).
[5]
Hadi Beik-Mohammadi, Søren Hauberg, Georgios Arvanitidis, Gerhard Neumann, and Leonel Rozo. 2021. Learning riemannian manifolds for geodesic motion skills. arXiv preprint arXiv:2106.04315 (2021).
[6]
Giorgio Biagetti, Paolo Crippa, Laura Falaschetti, and Claudio Turchetti. 2016. Discrete Bessel functions for representing the class of finite duration decaying sequences. In 2016 24th European Signal Processing Conference (EUSIPCO). IEEE, 2126–2130.
[7]
Wouter Boomsma, Kanti V Mardia, Charles C Taylor, Jesper Ferkinghoff-Borg, Anders Krogh, and Thomas Hamelryck. 2008. A generative, probabilistic model of local protein structure. Proceedings of the National Academy of Sciences 105, 26 (2008), 8932–8937.
[8]
Boost Community. 2024. Boost C++ Libraries. Boost. https://www.boost.org/Version 1.71.
[9]
Frank Bowman. 2012. Introduction to Bessel functions. Courier Corporation.
[10]
James Bremer. 2019. An algorithm for the rapid numerical evaluation of Bessel functions of real orders and arguments. Advances in Computational Mathematics 45 (2019), 173–211.
[11]
Remi Cuingnet. 2023. On the Computation of the Logarithm of the Modified Bessel Function of the Second Kind. arXiv preprint arXiv:2308.11964 (2023).
[12]
M. Galassi 2009. GNU Scientific Library: Reference Manual (3 ed.). http://www.gnu.org/software/gsl/
[13]
Riccardo Gatto. 2010. The Generalized von Mises-Fisher Distribution. In Advances in directional and linear statistics: A festschrift for Sreenivasa Rao Jammalamadaka. Springer, 51–68.
[14]
Chris Gregg and Kim Hazelwood. 2011. Where is the data? Why you cannot debate CPU vs. GPU performance without the answer. In (IEEE ISPASS) IEEE International Symposium on Performance Analysis of Systems and Software. IEEE, 134–144.
[15]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
[16]
International Organization for Standardization 2020. ISO/IEC 14882:2020 Programming languages – C++. International Organization for Standardization.
[17]
Boris Grigorevich Korenev. 2002. Bessel functions and their applications. CRC Press.
[18]
Samuel Kotz, Narayanaswamy Balakrishnan, and Norman L Johnson. 2004. Continuous multivariate distributions, Volume 1: Models and applications. John Wiley & Sons.
[19]
Alex Krizhevsky, Geoffrey Hinton, 2009. Learning multiple layers of features from tiny images. (2009).
[20]
Ming-Chieh Lin. 2016. An improved form of Bessel functions for efficiently and accurately simulating higher order modes in a cylindrical waveguide. In 2016 IEEE International Vacuum Electronics Conference (IVEC). IEEE, 1–2.
[21]
Kanti V Mardia, Peter E Jupp, and KV Mardia. 2000. Directional statistics. Vol. 2. Wiley Online Library.
[22]
Kevin Musgrave, Serge Belongie, and Ser-Nam Lim. 2020. A metric learning reality check. In Computer Vision–ECCV 2020: 16th European Conference, Proceedings. Springer, 681–699.
[23]
NVIDIA. 2023. CUDA, release: 12.3.
[24]
Changyong Oh, Kamil Adamczewski, and Mijung Park. 2020. Radial and directional posteriors for bayesian deep learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 5298–5305.
[25]
F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. R. Miller, B. V. Saunders, H. S. Cohl, M. A. McClain, and eds.2023. NIST Digital Library of Mathematical Functions. https://dlmf.nist.gov/, Release 1.1.11 of 2023-09-15.
[26]
EJ Rothwell. 2008. Computation of the logarithm of Bessel functions of complex argument and fractional order. Communications in numerical methods in engineering 24, 3 (2008), 237–249.
[27]
Justin Solomon. 2015. Numerical algorithms: methods for computer vision, machine learning, and graphics. CRC press.
[28]
Suvrit Sra. 2012. A short note on parameter approximation for von Mises-Fisher distributions: and a fast implementation of I s (x). Computational Statistics 27 (2012), 177–190.
[29]
DN Tumakov. 2019. The faster methods for computing Bessel functions of the first kind of an integer order with application to graphic processors. Lobachevskii Journal of Mathematics 40, 10 (2019), 1725–1738.
[30]
Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, 2020. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature methods 17, 3 (2020), 261–272.
[31]
Vasilis Vryniotis. 2021. How to train state-of-the-art models using torchvision’s latest primitives. https://pytorch.org/blog/how-to-train-state-of-the-art-models-using-torchvision-latest-primitives/. Accessed: 2024-01-15.
[32]
Frederik Warburg, Marco Miani, Silas Brack, and Søren Hauberg. 2024. Bayesian metric learning for uncertainty quantification in image retrieval. Advances in Neural Information Processing Systems 36 (2024).
[33]
George Neville Watson. 1922. A treatise on the theory of Bessel functions. Vol. 2. The University Press.
[34]
Geoffrey S Watson. 1983. Statistics on spheres. Wiley-Interscience.

Index Terms

  1. Accurate Computation of the Logarithm of Modified Bessel Functions on GPUs

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICS '24: Proceedings of the 38th ACM International Conference on Supercomputing
    May 2024
    582 pages
    ISBN:9798400706103
    DOI:10.1145/3650200
    This work is licensed under a Creative Commons Attribution International 4.0 License.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 June 2024

    Check for updates

    Author Tags

    1. Bessel Functions
    2. CUDA
    3. GPU
    4. Robust Computation
    5. von Mises-Fisher

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    Conference

    ICS '24
    Sponsor:

    Acceptance Rates

    ICS '24 Paper Acceptance Rate 45 of 125 submissions, 36%;
    Overall Acceptance Rate 629 of 2,180 submissions, 29%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 138
      Total Downloads
    • Downloads (Last 12 months)138
    • Downloads (Last 6 weeks)14
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media