INDENT: Incremental Online Decision Tree Training for Domain-Specific Systems-on-Chip
Article No.: 164, Pages 1 - 9
Abstract
The performance and energy efficiency potential of heterogeneous architectures has fueled domain-specific systems-on-chip (DSSoCs) that integrate general-purpose and domain-specialized hardware accelerators. Decision trees (DTs) perform high-quality, low-latency task scheduling to utilize the massive parallelism and heterogeneity in DSSoCs effectively. However, offline trained DT scheduling policies can quickly become ineffective when applications or hardware configurations change. There is a critical need for runtime techniques to train DTs incrementally without sacrificing accuracy since current training approaches have large memory and computational power requirements. To address this need, we propose INDENT, an incremental online DT framework to update the scheduling policy and adapt it to unseen scenarios. INDENT updates DT schedulers at runtime using only 1--8% of the original training data embedded during training. Thorough evaluations with hardware platforms and DSSoC simulators demonstrate that INDENT performs within 5% of a DT trained from scratch using the entire dataset and outperforms current state-of-the-art approaches.
References
[1]
RF Convergence: From the Signals to the Computer by Dr. Tom Rondeau (Microsystems Technology Office, DARPA). https://futurenetworks.ieee.org/images/files/pdf/FirstResponder/Tom-Rondeau-DARPA.pdf. [Online; last accessed 15-May-2022.].
[2]
River Library for Hoeffding Tree Classifiers. https://riverml.xyz/dev/api/tree/HoeffdingTreeClassifier. [Online; last accessed 15-May-2022.].
[3]
A. Amarnath, S. Pal, H. T. Kassa, A. Vega, A. Buyuktosunoglu, H. Franke, J.-D. Wellman, R. Dreslinski, and P. Bose. Heterogeneity-Aware Scheduling on SoCs for Autonomous Vehicles. IEEE Computer Architecture Letters, 20(2):82--85, 2021.
[4]
S. E. Arda et al. DS3: A System-Level Domain-Specific System-on-Chip Simulation Framework. IEEE Trans. on Computers, 69(8):1248--1262, 2020.
[5]
A. Bifet, G. Holmes, B. Pfahringer, and E. Frank. Fast Perceptron Decision Tree Learning from Evolving Data Streams. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 299--310. Springer, 2010.
[6]
L. F. Bittencourt, R. Sakellariou, and E. R. Madeira. DAG Scheduling using a Lookahead Variant of the Heterogeneous Earliest Finish Time Algorithm. In IEEE Euromicro Conference on Parallel, Distributed and Network-based Processing, pages 27--34, 2010.
[7]
D. Che, Q. Liu, K. Rasheed, and X. Tao. Decision Tree and Ensemble Learning Algorithms with their Applications in Bioinformatics. Software Tools and Algorithms for Biological Systems, pages 191--199, 2011.
[8]
T. Chen, T. He, M. Benesty, V. Khotilovich, Y. Tang, H. Cho, K. Chen, et al. XGBoost: Extreme Gradient Boosting. R package version 0.4-2, 1(4):1--4, 2015.
[9]
X. Gao, C. Shan, C. Hu, Z. Niu, and Z. Liu. An Adaptive Ensemble Machine Learning Model for Intrusion Detection. IEEE Access, 7:82512--82521, 2019.
[10]
A. A. Goksoy et al. DAS: Dynamic Adaptive Scheduling for Energy-Efficient Heterogeneous SoCs. IEEE Embedded Systems Letters, 14(1):51--54, 2021.
[11]
D. Green et al. Heterogeneous Integration at DARPA: Pathfinding and Progress in Assembly Approaches. ECTC, May, 2018.
[12]
T. Hastie, R. Tibshirani, J. H. Friedman, and J. H. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, volume 2. Springer, 2009.
[13]
M. He, J. Zhang, and V. Vittal. Robust Online Dynamic Security Assessment using Adaptive Ensemble Decision-Tree Learning. IEEE Transactions on Power systems, 28(4):4089--4098, 2013.
[14]
J. L. Hennessy and D. A. Patterson. A New Golden Age for Computer Architecture. Communications of the ACM, 62(2):48--60, 2019.
[15]
G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Advances in Neural Information Processing Systems, 30, 2017.
[16]
R. Khasanov, J. Robledo, C. Menard, A. Goens, and J. Castrillon. Domain-Specific Hybrid Mapping for Energy-efficient Baseband Processing in Wireless Networks. ACM Transactions on Embedded Computing Systems (TECS), 20(5s):1--26, 2021.
[17]
P. Kontschieder, M. Fiterau, A. Criminisi, and S. R. Bulo. Deep Neural Decision Forests. In Proceedings of the IEEE International Conference on Computer Vision, pages 1467--1475, 2015.
[18]
A. Krishnakumar, S. E. Arda, A. A. Goksoy, S. K. Mandal, U. Y. Ogras, A. L. Sartor, and R. Marculescu. Runtime Task Scheduling using Imitation Learning for Heterogeneous Many-core Systems. IEEE Transactions on CAD of Integrated Circuits and Systems, 39(11):4064--4077, 2020.
[19]
A. Krishnakumar and U. Y. Ogras. Performance Analysis and Optimization of Decision Tree Classifiers on Embedded Devices: Work-in-Progress. In Proceedings of the 2021 International Conference on Embedded Software, pages 37--38, 2021.
[20]
P. Kumari, J. Choi, N. González-Prelcic, and R. W. Heath. IEEE 802.11 Ad-based Radar: An Approach to Joint Vehicular Communication-Radar System. IEEE Transactions on Vehicular Technology, 67(4):3012--3027, 2017.
[21]
Z. Lin, W. Zhang, and S. Sharad. Decision Tree based Hardware Power Monitoring for Run Time Dynamic Power Management in FPGA. In 27th International Conference on Field Programmable Logic and Applications, pages 1--8, 2017.
[22]
J. Mack, S. E. Arda, U. Y. Ogras, and A. Akoglu. Performant, Multi-Objective Scheduling of Highly Interleaved Task Graphs on Heterogeneous System on Chip Devices. IEEE Transactions on Parallel and Distributed Systems, 33(9):2148--2162, 2021.
[23]
J. Mack, S. Hassan, N. Kumbhare, M. C. Gonzalez, and A. Akoglu. CEDR-A Compiler-integrated, Extensible DSSoC Runtime. ACM Transactions on Embedded Computing Systems (TECS), 2022.
[24]
S. Malik, S. Ahmad, I. Ullah, D. H. Park, and D. Kim. An Adaptive Emergency First Intelligent Scheduling Algorithm for Efficient Task Management and Scheduling in Hybrid of Hard Real-time and Soft Real-time Embedded IoT Systems. Sustainability, 11(8):2192, 2019.
[25]
C. Manapragada, G. I. Webb, and M. Salehi. Extremely Fast Decision Tree. In Proc. of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1953--1962, 2018.
[26]
H. Mao, M. Alizadeh, I. Menache, and S. Kandula. Resource Management with Deep Reinforcement Learning. In ACM Workshop on Hot Topics in Networks, pages 50--56, 2016.
[27]
S. Parsa and R. Entezari-Maleki. RASA: A New Grid Task Scheduling Algorithm. International Journal of Digital Content Technology and its Applications, 3(4):91--99, 2009.
[28]
H. H. Patel and P. Prajapati. Study and Analysis of Decision Tree based Classification Algorithms. International Journal of Computer Sciences and Engineering, 6(10):74--78, 2018.
[29]
F. Pedregosa et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12:2825--2830, 2011.
[30]
L. E. Raileanu and K. Stoffel. Theoretical Comparison between the Gini Index and Information Gain Criteria. Annals of Mathematics and Artificial Intelligence, 41(1):77--93, 2004.
[31]
J. J. Rodriguez, L. I. Kuncheva, and C. J. Alonso. Rotation Forest: A New Classifier Ensemble Method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10):1619--1630, 2006.
[32]
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347, 2017.
[33]
A. Silva, M. Gombolay, T. Killian, I. Jimenez, and S.-H. Son. Optimization Methods for Interpretable Differentiable Decision Trees Applied to Reinforcement Learning. In International Conference on Artificial Intelligence and Statistics, pages 1855--1865. PMLR, 2020.
[34]
R. Struharik. Decision Tree Ensemble Hardware Accelerators for Embedded Applications. In IEEE 13th International Symposium on Intelligent Systems and Informatics (SISY), pages 101--106, 2015.
[35]
J. Su and H. Zhang. A Fast Decision Tree Learning Algorithm. In AAAI, volume 6, pages 500--505, 2006.
[36]
A. Suárez and J. F. Lutsko. Globally Optimal Fuzzy Decision Trees for Classification and Regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12):1297--1311, 1999.
[37]
V. Suhendra, C. Raghavan, and T. Mitra. Integrated Scratchpad Memory Optimization and Task Scheduling for MPSoC Architectures. In Proceedings of the International Conference on Compilers, Architecture and Synthesis for Embedded Systems, pages 401--410, 2006.
[38]
T. N. Theis and H.-S. P. Wong. The End of Moore's Law: A New Beginning for Information Technology. Computing in Science & Engineering, 19(2):41--50, 2017.
[39]
H. Topcuoglu, S. Hariri, and M.-Y. Wu. Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing. IEEE Transactions on Parallel and Distributed Systems, 13(3):260--274, 2002.
[40]
Y. Zhan, H. B. Ammar, and M. E. Taylor. Scalable Lifelong Reinforcement Learning. Pattern Recognition, 72:407--418, 2017.
Index Terms
- INDENT: Incremental Online Decision Tree Training for Domain-Specific Systems-on-Chip
Recommendations
DTRL: Decision Tree-based Multi-Objective Reinforcement Learning for Runtime Task Scheduling in Domain-Specific System-on-Chips
Special Issue ESWEEK 2023Domain-specific systems-on-chip (DSSoCs) combine general-purpose processors and specialized hardware accelerators to improve performance and energy efficiency for a specific domain. The optimal allocation of tasks to processing elements (PEs) with minimal ...
A Domain-Specific System-On-Chip Design for Energy Efficient Wearable Edge AI Applications
ISLPED '22: Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and DesignArtificial intelligence (AI) based wearable applications collect and process a significant amount of streaming sensor data. Transmitting the raw data to cloud processors wastes scarce energy and threatens user privacy. Wearable edge AI devices should ...
Incremental training of support vector machines
We propose a new algorithm for the incremental training of support vector machines (SVMs) that is suitable for problems of sequentially arriving data and fast constraint parameter variation. Our method involves using a "warm-start" algorithm for the ...
Comments
Information & Contributors
Information
Published In
October 2022
1467 pages
ISBN:9781450392174
DOI:10.1145/3508352
- Conference Chair:
- Tulika Mitra,
- Program Chairs:
- Evangeline Young,
- Jinjun Xiong
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
In-Cooperation
- IEEE-EDS: Electronic Devices Society
- IEEE CAS
- IEEE CEDA
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 December 2022
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Defense Advanced Research Projects Agency (DARPA)
Conference
ICCAD '22
Sponsor:
ICCAD '22: IEEE/ACM International Conference on Computer-Aided Design
October 30 - November 3, 2022
California, San Diego
Acceptance Rates
Overall Acceptance Rate 457 of 1,762 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 134Total Downloads
- Downloads (Last 12 months)34
- Downloads (Last 6 weeks)1
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in