Abstract
Due to the high energy consumption and scalability challenges of deep learning, there is a critical need to shift research focus towards dealing with energy consumption constraints. Tsetlin Machines (TMs) are a recent approach to machine learning that has demonstrated significantly reduced energy usage compared to neural networks alike, while performing competitively accuracy-wise on several benchmarks. However, TMs rely heavily on energy-costly random number generation to stochastically guide a team of Tsetlin Automata (TA) to a Nash Equilibrium of the TM game. In this paper, we propose a novel finite-state learning automaton that can replace the TA in TM learning, for increased determinism. The new automaton uses multi-step deterministic state jumps to reinforce sub-patterns. Simultaneously, flipping a coin to skip every d’th state update ensures diversification by randomization. The d-parameter thus allows the degree of randomization to be finely controlled. E.g., \(d=1\) makes every update random and \(d=\infty \) makes the automaton completely deterministic. Our empirical results show that, overall, only substantial degrees of determinism reduces accuracy. Energy-wise, random number generation constitutes switching energy consumption of the TM, saving up to 11 mW power for larger datasets with high d values. We can thus use the new d-parameter to trade off accuracy against energy consumption, to facilitate low-energy machine learning.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
An implementation of ADTM can be found at https://github.com/cair/Deterministic-Tsetlin-Machine.
- 2.
Available from https://archive.ics.uci.edu/ml/datasets/qualitative_bankruptcy.
- 3.
Available from http://archive.ics.uci.edu/ml/datasets/balance+scale.
- 4.
Available from https://archive.ics.uci.edu/ml/datasets/Breast+Cancer.
- 5.
Available from https://archive.ics.uci.edu/ml/datasets/Liver+Disorders.
- 6.
Available from https://archive.ics.uci.edu/ml/datasets/Statlog+%28Heart%29.
References
Abeyrathna, K.D., Granmo, O.C., Goodwin, M.: Extending the Tsetlin Machine with integer-weighted clauses for increased interpretability. arXiv preprint arXiv:2005.05131 (2020)
Abeyrathna, K.D., Granmo, O.-C., Zhang, X., Goodwin, M.: A scheme for continuous input to the Tsetlin Machine with applications to forecasting disease outbreaks. In: Wotawa, F., Friedrich, G., Pill, I., Koitz-Hristov, R., Ali, M. (eds.) IEA/AIE 2019. LNCS (LNAI), vol. 11606, pp. 564–578. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22999-3_49
Abeyrathna, K.D., Granmo, O.C., Zhang, X., Jiao, L., Goodwin, M.: The regression Tsetlin Machine - a novel approach to interpretable non-linear regression. Philos. Trans. R. Soc. A 378, 20190165 (2019)
Berge, G.T., Granmo, O.C., Tveit, T.O., Goodwin, M., Jiao, L., Matheussen, B.V.: Using the Tsetlin Machine to learn human-interpretable rules for high-accuracy text categorization with medical applications. IEEE Access 7, 115134–115146 (2019). https://doi.org/10.1109/ACCESS.2019.2935416
Chen, J., Ran, X.: Deep learning with edge computing: a review. Proc. IEEE 107(8), 1655–1674 (2019)
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)
García-Martín, E., Rodrigues, C.F., Riley, G., Grahn, H.: Estimation of energy consumption in machine learning. J. Parallel Distrib. Comput. 134, 75–88 (2019). https://doi.org/10.1016/j.jpdc.2019.07.007
Rahimi Gorji, S., Granmo, O.-C., Glimsdal, S., Edwards, J., Goodwin, M.: Increasing the inference and learning speed of Tsetlin Machines with clause indexing. In: Fujita, H., Fournier-Viger, P., Ali, M., Sasaki, J. (eds.) IEA/AIE 2020. LNCS (LNAI), vol. 12144, pp. 695–708. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-55789-8_60
Rahimi Gorji, S., Granmo, O.-C., Phoulady, A., Goodwin, M.: A Tsetlin Machine with multigranular clauses. In: Bramer, M., Petridis, M. (eds.) SGAI 2019. LNCS (LNAI), vol. 11927, pp. 146–151. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-34885-4_11
Granmo, O.C.: The Tsetlin Machine - a game theoretic bandit driven approach to optimal pattern recognition with propositional logic. arXiv:1804.01508
Granmo, O.C., Glimsdal, S., Jiao, L., Goodwin, M., Omlin, C.W., Berge, G.T.: The convolutional Tsetlin Machine. arXiv preprint arXiv:1905.09688 (2019)
McDermott, J., Forsyth, R.S.: Diagnosing a disorder in a classification benchmark. Pattern Recogn. Lett. 73, 41–43 (2016)
Narendra, K.S., Thathachar, M.A.: Learning Automata: An Introduction. Courier Corporation, North Chelmsford (2012)
Nori, H., Jenkins, S., Koch, P., Caruana, R.: InterpretML: a unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223 (2019)
Oommen, B.J.: Stochastic searching on the line and its applications to parameter learning in nonlinear optimization. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 27(4), 733–739 (1997)
Phoulady, A., Granmo, O.C., Gorji, S.R., Phoulady, H.A.: The weighted Tsetlin Machine: compressed representations with clause weighting. In: Ninth International Workshop on Statistical Relational AI (StarAI 2020) (2020)
Shafik, R., Wheeldon, A., Yakovlev, A.: Explainability and dependability analysis of learning automata based AI hardware. In: IEEE 26th International Symposium on On-Line Testing and Robust System Design (IOLTS). IEEE (2020)
Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in NLP. In: ACL (2019)
Thathachar, M.A.L., Sastry, P.S.: Networks of Learning Automata: Techniques for Online Stochastic Optimization. Kluwer Academic Publishers, Dordrecht (2004)
Tsetlin, M.L.: On behaviour of finite automata in random medium. Avtomat. i Telemekh 22(10), 1345–1354 (1961)
Wheeldon, A., Shafik, R., Rahman, T., Lei, J., Yakovlev, A., Granmo, O.C.: Learning automata based energy-efficient AI hardware design for IoT. Philos. Trans. R. Soc. A 378, 20190593 (2020)
Acknowledgement
The authors gratefully acknowledge the contributions from Jonathan Edwards at Temporal Computing on strategies for deterministic Tsetlin Machine learning.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Abeyrathna, K.D. et al. (2020). A Novel Multi-step Finite-State Automaton for Arbitrarily Deterministic Tsetlin Machine Learning. In: Bramer, M., Ellis, R. (eds) Artificial Intelligence XXXVII. SGAI 2020. Lecture Notes in Computer Science(), vol 12498. Springer, Cham. https://doi.org/10.1007/978-3-030-63799-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-63799-6_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63798-9
Online ISBN: 978-3-030-63799-6
eBook Packages: Computer ScienceComputer Science (R0)