Abstract
Probabilistic inference is a well suited approach to address the challenges of resource constrained embedded application scenarios. In particular, probabilistic models learned generatively are robust to missing data and are capable of encoding domain knowledge seamlessly. These traits have been leveraged to propose hardware-aware probabilistic learning and inference strategies that induce Pareto optimal accuracy versus resource consumption trade-offs. This paper proposes a model-complexity tuning strategy that relies on ensembles of probabilistic classifiers to identify the difficulty of the classification task on a given instance. It then dynamically switches to a higher or lower complexity setting accordingly. The strategy is evaluated on an embedded human activity recognition scenario and demonstrates a superior performance when compared to the Pareto-optimal trade-off obtained when the ensembles are deployed statically, especially in low cost regions of the trade-off space. This makes the strategy amenable to embedded computing scenarios, where one of the main constraints towards always-on functionality are the device’s strict resource constraints.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This is known as context specific independence [27].
- 2.
The models learned with the discriminative bias tend to have a higher classification accuracy than those learned generatively, so the ensembles are constructed on the basis of cost similarity.
- 3.
The overhead is in terms of the additional cost from evaluating classifier ensembles multiple times. It is assumed that there is no time overhead or latency increase.
References
Anguita, D., Ghio, A., Oneto, L., Parra, X., Reyes-Ortiz, J.L.: A public domain dataset for human activity recognition using smartphones. In: 21th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN) (2013)
Choi, Y., Van den Broeck, G.: On robust trimming of Bayesian network classifiers. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI) (2018)
Choi, Y., Vergari, A., Van den Broeck, G.: Lecture notes: Probabilistic circuits: Representation and inference (2020). http://starai.cs.ucla.edu/papers/LecNoAAAI20.pdf
Darwiche, A.: A differential approach to inference in Bayesian networks. J. ACM (JACM) 50(3), 280–305 (2003)
Fayyad, U., Irani, K.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI) (1993)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. J. Mach. Learn. 29(2), 131–163 (1997)
Galindez Olascoaga, L.I., Meert, W., Shah, N., Van den Broeck, G., Verhelst, M.: Discriminative bias for learning probabilistic sentential decision diagrams. In: Proceedings of the Symposium on Intelligent Data Analysis (IDA) (2020)
Galindez Olascoaga, L.I., Meert, W., Shah, N., Verhelst, M., Van den Broeck, G.: Towards hardware-aware tractable learning of probabilistic models. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 13726–13736 (2019)
Giraldo, J., O’Connor, C., Verhelst, M.: Efficient keyword spotting through hardware-aware conditional execution of deep neural networks. In: 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), pp. 1–8. IEEE (2019)
Hall, M.A.: Correlation-based Feature Subset Selection for Machine Learning. Ph.D. thesis, University of Waikato, Hamilton, New Zealand (1998)
Inoue, H.: Adaptive ensemble prediction for deep neural networks based on confidence level. In: The 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 1284–1293. PMLR (2019)
Karbachevsky, A., et al.: Hcm: hardware-aware complexity metric for neural network architectures. arXiv preprint arXiv:2004.08906 (2020)
Khosravi, P., Choi, Y., Liang, Y., Vergari, A., Van den Broeck, G.: On tractable computation of expected predictions. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 11169–11180 (2019)
Khosravi, P., Vergari, A., Choi, Y., Liang, Y., Van den Broeck, G.: Handling missing data in decision trees: a probabilistic approach. In: The Art of Learning with Missing Values Workshop at ICML (Artemiss) (2020)
Kisa, D., Van den Broeck, G., Choi, A., Darwiche, A.: Probabilistic sentential decision diagrams. In: International Conference on the Principles of Knowledge Representation and Reasoning (2014)
Liang, Y., Bekker, J., Van den Broeck, G.: Learning the structure of probabilistic sentential decision diagrams. In: Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI) (2017)
Liu, L., Deng, J.: Dynamic deep neural networks: optimizing accuracy-efficiency trade-offs by selective execution. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Perello-Nieto, M., Telmo De Menezes Filho, E.S., Kull, M., Flach, P.: Background check: a general technique to build more reliable and versatile classifiers. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 1143–1148. IEEE (2016)
Poon, H., Domingos, P.: Sum-product networks: a new deep architecture. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 689–690. IEEE (2011)
Shah, N., Galindez Olascoaga, L.I., Meert, W., Verhelst, M.: Problp: A framework for low-precision probabilistic inference. In: Proceedings of the 56th Annual Design Automation Conference (DAC) 2019, pp. 1–6 (2019)
Shah, N.A., Galindez Olascoaga, L.I., Meert, W., Verhelst, M.: Acceleration of probabilistic reasoning through custom processor architecture. In: 2020 Design, Automation and Test in Europe Conference and Exhibition (DATE), pp. 322–325 (2020)
Sze, V., Chen, Y.H., Yang, T.J., Emer, J.S.: Efficient processing of deep neural networks: a tutorial and survey. Proc. IEEE 105(12), 2295–2329 (2017)
Verachtert, A., Blockeel, H., Davis, J.: Dynamic early stopping for naive Bayes. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI), vol. 2016, pp. 2082–2088. AAAI Press (2016)
Verhelst, M., Murmann, B.: Machine learning at the edge. In: NANO-CHIPS 2030, pp. 293–322. Springer (2020)
Vlasselaer, J., Meert, W., Verhelst, M.: Towards resource-efficient classifiers for always-on monitoring. In: Brefeld, U., et al. (eds.) ECML PKDD 2018. LNCS (LNAI), vol. 11053, pp. 305–321. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-10997-4_19
Weber, L., Sommer, L., Oppermann, J., Molina, A., Kersting, K., Koch, A.: Resource-efficient logarithmic number scale arithmetic for SPN inference on FPGAs. In: 2019 International Conference on Field-Programmable Technology (ICFPT), pp. 251–254. IEEE (2019)
Zhang, N.L., Poole, D.: On the role of context-specific independence in probabilistic inference. In: 16th International Joint Conference on Artificial Intelligence, IJCAI 1999, Stockholm, Sweden, vol. 2, p. 1288 (1999)
Acknowledgements
This work was partially supported by the EU-ERC Project Re-SENSE grant ERC-2016-STG-71503, the “Onderzoeksprogramma Artificiële Intelligentie Vlaanderen” programme from the Flemish Government, and a gift from Intel.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Galindez Olascoaga, L.I., Meert, W., Shah, N., Verhelst, M. (2020). Dynamic Complexity Tuning for Hardware-Aware Probabilistic Circuits. In: Gama, J., et al. IoT Streams for Data-Driven Predictive Maintenance and IoT, Edge, and Mobile for Embedded Machine Learning. ITEM IoT Streams 2020 2020. Communications in Computer and Information Science, vol 1325. Springer, Cham. https://doi.org/10.1007/978-3-030-66770-2_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-66770-2_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66769-6
Online ISBN: 978-3-030-66770-2
eBook Packages: Computer ScienceComputer Science (R0)