skip to main content
10.1145/3528535.3531519acmconferencesArticle/Chapter ViewAbstractPublication PagesmiddlewareConference Proceedingsconference-collections
research-article

Bolt: Fast Inference for Random Forests

Published: 24 October 2022 Publication History

Abstract

Random forests use ensembles of decision trees to boost accuracy for machine learning tasks. However, large ensembles slow down inference on platforms that process each tree in an ensemble individually. We present Bolt, a platform that restructures whole random forests, not just individual trees, to speed up inference. Conceptually, Bolt maps every path in each tree to a lookup table which, if cache were large enough, would allow inference with just one memory access. When the size of the lookup table exceeds cache capacity, Bolt employs a novel combination of lossless compression, parameter selection, and bloom filters to shrink the table while preserving fast inference. We compared inference speed in Bolt to three state-of-the-art platforms: Python Scikit-Learn, Ranger, and Forest Packing. We evaluated these platforms using datasets with vision, natural language processing and categorical applications. We observed that on ensembles of shallow decision trees Bolt can run 2--14X faster than competing platforms and that Bolt's speedups persist as the number of decision trees in an ensemble increases.

References

[1]
Decision tree wikipedia. https://en.wikipedia.org/wiki/Decision_tree, 2021.
[2]
K. Angstadt, A. Subramaniyan, E. Sadredini, R. Rahimi, K. Skadron, W. Weimer, and R. Das. Aspen: A scalable in-sram architecture for pushdown automata. In 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 921–932. IEEE, 2018.
[3]
L. Arnould, C. Boyer, and E. Scornet. Analyzing the tree-layer structure of deep forests. arXiv e-prints, pages arXiv–2010, 2020.
[4]
Y. D. Barve, S. Shekhar, A. Chhokra, S. Khare, A. Bhattacharjee, Z. Kang, H. Sun, and A. Gokhale. Fecbench: A holistic interference-aware approach for application performance modeling. In 2019 IEEE International Conference on Cloud Engineering (IC2E), pages 211–221. IEEE, 2019.
[5]
U. Bhatt, A. Xiang, S. Sharma, A. Weller, A. Taly, Y. Jia, J. Ghosh, R. Puri, J. M. Moura, and P. Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 648–657, 2020.
[6]
B. H. Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13(7):422–426, 1970.
[7]
J. Boubin, N. Babu, C. Stewart, J. Chumley, and S. Zhang. Managing edge resources for fully autonomous aerial systems. In ACM Symposium on Edge Computing, 2019.
[8]
J. Browne, D. Mhembere, T. M. Tomita, J. T. Vogelstein, and R. Burns. Forest packing: Fast parallel, decision forests. In Proceedings of the 2019 SIAM International Conference on Data Mining, pages 46–54. SIAM, 2019.
[9]
T. Chen, T. He, M. Benesty, V. Khotilovich, and Y. Tang. Xgboost: extreme gradient boosting. R package version 0.4-2, pages 1–4, 2015.
[10]
E. Gansner, E. Koutsofios, and S. North. Drawing graphs with dot, 2006.
[11]
J. Grohmann, P. K. Nicholson, J. O. Iglesias, S. Kounev, and D. Lugones. Monitorless: Predicting performance degradation in cloud applications with machine learning. In Proceedings of the 20th international middleware conference, pages 149–162, 2019.
[12]
G. S. Halford, R. Baker, J. E. McCredden, and J. D. Bain. How many variables can humans process? Psychological science, 16(1):70–76, 2005.
[13]
S. Hong, T. You, S. Kwak, and B. Han. Online tracking by learning discriminative saliency map with convolutional neural network. In International conference on machine learning, pages 597–606, 2015.
[14]
P. Joshi, H. Afli, M. Hasanuzzaman, C. Thapa, and T. Scully. Enabling deep learning for all-in edge paradigm. arXiv preprint arXiv:2204.03326, 2022.
[15]
E. Kıcıman and M. Richardson. Towards decision support and goal achievement: Identifying action-outcome relationships from social media. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 547–556, 2015.
[16]
S. M. Lundberg, G. Erion, H. Chen, A. DeGrave, J. M. Prutkin, B. Nair, R. Katz, J. Himmelfarb, N. Bansal, and S.-I. Lee. From local explanations to global understanding with explainable ai for trees. Nature machine intelligence, 2(1):2522–5839, 2020.
[17]
A. K. Maji, S. Mitra, B. Zhou, S. Bagchi, and A. Verma. Mitigating interference in cloud services by middleware reconfiguration. In Proceedings of the 15th International Middleware Conference, pages 277–288, 2014.
[18]
I. Mollas, G. Tsoumakas, and N. Bassiliades. Lionforests: Local interpretation of random forests through path selection. arXiv preprint arXiv:1911.08780, 2019.
[19]
S. Moosavi, M. H. Samavatian, A. Nandi, S. Parthasarathy, and R. Ramnath. Short and long-term pattern discovery over large-scale geo-spatiotemporal data. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2905–2913, 2019.
[20]
N. Morris, C. Stewart, L. Chen, R. Birke, and J. Kelley:. Model-driven computational sprinting. In ACM European Conference on Computer Systems, 2018.
[21]
E. Romero. Bolt, 4 2022. Repo: https://github.com/EduardoRomero83/CADM.
[22]
E. Romero-Gainza, C. Stewart, A. Li, K. Hale, and N. Morris. Memory mapping and parallelizing random forests for speed and cache efficiency. In 50th International Conference on Parallel Processing Workshop, pages 1–5, 2021.
[23]
C. Stewart, D. Vasisht, and W. Shi. Toward fully autonomous and networked vehicles. IEEE Internet Computing, 25(3), 2021.
[24]
T. Tracy, Y. Fu, I. Roy, E. Jonas, and P. Glendenning. Towards machine learning on the automata processor. In International Conference on High Performance Computing, pages 200–218. Springer, 2016.
[25]
J. Wadden, T. Tracy, E. Sadredini, L. Wu, C. Bo, J. Du, Y. Wei, J. Udall, M. Wallace, M. Stan, et al. Automatazoo: A modern automata processing benchmark suite. In 2018 IEEE International Symposium on Workload Characterization (IISWC), pages 13–24. IEEE, 2018.
[26]
M. N. Wright and A. Ziegler. ranger: A fast implementation of random forests for high dimensional data in c++ and r. arXiv preprint arXiv:1508.04409, 2015.
[27]
Z. Xie, W. Dong, J. Liu, H. Liu, and D. Li. Tahoe: tree structure-aware high performance inference engine for decision tree ensemble on gpu. In Proceedings of the Sixteenth European Conference on Computer Systems, pages 426–440, 2021.
[28]
Yelp. Yelp dataset version 6, 2014. data retrieved from Kaggle, https://www.kaggle.com/yelp-dataset/yelp-dataset/version/6.
[29]
Z. Zhang, J. Boubin, C. Stewart, and S. Khanal. Whole-field reinforcement learning: A fully autonomous aerial scouting method for precision agriculture. Sensors, 20(22):6585, 2020.
[30]
Z.-H. Zhou and J. Feng. Deep forest. arXiv preprint arXiv:1702.08835, 2017.

Cited By

View all
  • (2024)Characterization of Parallelism Techniques for Decision Tree Ensembles2024 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)10.1109/ACSOS61780.2024.00038(169-174)Online publication date: 16-Sep-2024
  • (2023)AI-Driven Validation of Digital Agriculture ModelsSensors10.3390/s2303118723:3(1187)Online publication date: 20-Jan-2023
  • (2022)Performance Modeling for Short-Term Cache AllocationProceedings of the 51st International Conference on Parallel Processing10.1145/3545008.3545094(1-11)Online publication date: 29-Aug-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
Middleware '22: Proceedings of the 23rd ACM/IFIP International Middleware Conference
November 2022
110 pages
ISBN:9781450393409
DOI:10.1145/3528535
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2022

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Conference

Middleware '22
Sponsor:
Middleware '22: 23rd International Middleware Conference
November 7 - 11, 2022
QC, Quebec, Canada

Acceptance Rates

Middleware '22 Paper Acceptance Rate 8 of 21 submissions, 38%;
Overall Acceptance Rate 203 of 948 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)25
  • Downloads (Last 6 weeks)2
Reflects downloads up to 12 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Characterization of Parallelism Techniques for Decision Tree Ensembles2024 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)10.1109/ACSOS61780.2024.00038(169-174)Online publication date: 16-Sep-2024
  • (2023)AI-Driven Validation of Digital Agriculture ModelsSensors10.3390/s2303118723:3(1187)Online publication date: 20-Jan-2023
  • (2022)Performance Modeling for Short-Term Cache AllocationProceedings of the 51st International Conference on Parallel Processing10.1145/3545008.3545094(1-11)Online publication date: 29-Aug-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media