research-article

TinyMLOps for real-time ultra-low power MCUs applied to frame-based event classification

Authors:

Julyan ArbelAuthors Info & Claims

EuroMLSys '23: Proceedings of the 3rd Workshop on Machine Learning and Systems

Pages 148 - 153

https://doi.org/10.1145/3578356.3592586

Published: 08 May 2023 Publication History

Abstract

TinyML applications such as speech recognition, motion detection, or anomaly detection are attracting many industries and researchers thanks to their innovative and cost-effective potential. Since tinyMLOps is at an even earlier stage than MLOps, the best practices and tools of tinyML are yet to be found to deliver seamless production-ready applications. TinyMLOps has common challenges with MLOps, but it differs from it because of its hard footprint constraints. In this work, we analyze the steps of successful tinyMLOps with a highlight on challenges and solutions in the case of real-time frame-based event classification on low-power devices. We also report a comparative result of our tinyMLOps solution against tf.lite and NNoM.

References

[1]

Jose M. Alvarez and Mathieu Salzmann. 2016. Learning the Number of Neurons in Deep Networks. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5--10, 2016, Barcelona, Spain, Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (Eds.). Curran Associates, Inc., 2262--2270. https://proceedings.neurips.cc/paper/2016/hash/6e7d2da6d3953058db75714ac400b584-Abstract.html

[2]

Mattia Antonini, Miguel Pincheira, Massimo Vecchio, and Fabio Antonelli. 2022. Tiny-MLOps: a framework for orchestrating ML applications at the far edge of IoT systems. In IEEE International Conference on Evolving and Adaptive Intelligent System, EAIS 2022, Larnaca, Cyprus, May 25--26, 2022, Plamen Angelov, George A. Papadopoulos, Giovanna Castellano, José A. Iglesias, Gabriella Casalino, Edwin Lughofer, and Daniel Leite (Eds.). IEEE, 1--8.

[3]

Colby R. Banbury, Vijay Janapa Reddi, Max Lam, William Fu, Amin Fazel, Jeremy Holleman, Xinyuan Huang, Robert Hurtado, David Kanter, Anton Lokhmotov, David A. Patterson, Danilo Pau, Jae-sun Seo, Jeff Sieracki, Urmish Thakker, Marian Verhelst, and Poonam Yadav. 2020. Benchmarking TinyML Systems: Challenges and Direction. CoRR abs/2003.04821 (2020). arXiv:2003.04821 https://arxiv.org/abs/2003.04821

[4]

Colby R. Banbury, Chuteng Zhou, Igor Fedorov, Ramon Matas Navarro, Urmish Thakker, Dibakar Gope, Vijay Janapa Reddi, Matthew Mattina, and Paul N. Whatmough. 2021. MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers. In Proceedings of Machine Learning and Systems 2021, MLSys 2021, virtual, April 5--9, 2021, Alex Smola, Alex Dimakis, and Ion Stoica (Eds.). mlsys.org. https://proceedings.mlsys.org/paper/2021/hash/a3c65c2974270fd093ee8a9bf8ae7d0b-Abstract.html

[5]

Roberto Cahuantzi, Xinye Chen, and Stefan Güttel. 2021. A Comparison of LSTM and GRU Networks for Learning Symbolic Sequences. arXiv:2107.02248 [cs] http://arxiv.org/abs/2107.02248

[6]

Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2020. Once-for-All: Train One Network and Specialize It for Efficient Deployment. (2020). arXiv:1908.09791 [cs, stat] http://arxiv.org/abs/1908.09791

[7]

Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang. 2020. A Survey of Model Compression and Acceleration for Deep Neural Networks. (2020). arXiv:1710.09282 [cs] http://arxiv.org/abs/1710.09282

[8]

Robert David, Jared Duke, Advait Jain, Vijay Janapa Reddi, Nat Jeffries, Jian Li, Nick Kreeger, Ian Nappier, Meghna Natraj, Shlomi Regev, Rocky Rhodes, Tiezhen Wang, and Pete Warden. 2021. TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems. (2021). arXiv:2010.08678 [cs] http://arxiv.org/abs/2010.08678

[9]

Hiroshi Doyu, Roberto Morabito, and Martina Brachmann. 2021. A TinyMLaaS Ecosystem for Machine Learning in IoT: Overview and Research Challenges. In International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2021, Hsinchu, Taiwan, April 19--22, 2021. IEEE, 1--5.

[10]

Jonathan Frankle and Michael Carbin. 2018. The Lottery Ticket Hypothesis: Training Pruned Neural Networks. CoRR abs/1803.03635 (2018). arXiv:1803.03635 http://arxiv.org/abs/1803.03635

[11]

Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. 2021. A Survey of Quantization Methods for Efficient Neural Network Inference. (2021). arXiv:2103.13630 [cs] http://arxiv.org/abs/2103.13630

[12]

Hui Han and Julien Siebert. 2022. TinyML: A Systematic Review and Synthesis of Existing Research. In 2022 International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2022, Jeju Island, Korea, Republic of, February 21--24, 2022. IEEE, 269--274.

[13]

Song Han, Huizi Mao, and William J. Dally. 2016. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. (2016). arXiv:1510.00149 [cs] http://arxiv.org/abs/1510.00149

Digital Library

[14]

Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning Both Weights and Connections for Efficient Neural Networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 (Cambridge, MA, USA, 2015). MIT Press, 1135--1143.

[15]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. arXiv:1512.03385 [cs] http://arxiv.org/abs/1512.03385

[16]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. (2015). arXiv:1503.02531 [cs, stat] http://arxiv.org/abs/1503.02531

[17]

Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. 2021. Sparsity in Deep Learning: Pruning and Growth for Efficient Inference and Training in Neural Networks. arXiv:2102.00554 [cs]

[18]

Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2017. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. (2017). arXiv:1712.05877 [cs, stat] http://arxiv.org/abs/1712.05877 Comment: 14 pages, 12 figures.

[19]

Dominik Kreuzberger, Niklas Kühl, and Sebastian Hirschl. 2022. Machine Learning Operations (MLOps): Overview, Definition, and Architecture. CoRR abs/2205.02302 (2022). arXiv:2205.02302

[20]

Liangzhen Lai, Naveen Suda, and Vikas Chandra. 2018. CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs. (2018). arXiv:1801.06601 [cs] http://arxiv.org/abs/1801.06601

[21]

Edgar Liberis, Łukasz Dudziak, and Nicholas D. Lane. 2021. μNAS: Constrained Neural Architecture Search for Microcontrollers. In Proceedings of the 1st Workshop on Machine Learning and Systems (New York, NY, USA, 2021-04-26) (EuroMLSys '21). Association for Computing Machinery, 70--79.

Digital Library

[22]

Jianjia Ma. 2020. A higher-level Neural Network library on Microcontrollers (NNoM).

[23]

Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Quoc V. Le. 2019. SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition. In Interspeech 2019, 20th Annual Conference of the International Speech Communication Association, Graz, Austria, 15--19 September 2019, Gernot Kubin and Zdravko Kacic (Eds.). ISCA, 2613--2617.

[24]

Muhammad Shafique, Theocharis Theocharides, Vijay Janapa Reddi, and Boris Murmann. 2021. TinyML: Current Progress, Research Challenges, and Future Roadmap. In 58th ACM/IEEE Design Automation Conference, DAC 2021, San Francisco, CA, USA, December 5--9, 2021. IEEE, 1303--1306.

Digital Library

[25]

Bharath Sudharsan, Simone Salerno, Duc-Duy Nguyen, Muhammad Yahya, Abdul Wahid, Piyush Yadav, John G. Breslin, and Muhammad Intizar Ali. 2021. TinyML Benchmark: Executing Fully Connected Neural Networks on Commodity Microcontrollers. In 7th IEEE World Forum on Internet of Things, WF-IoT 2021, New Orleans, LA, USA, June 14 - July 31, 2021. IEEE, 883--884.

[26]

Filip Svoboda, Javier Fernandez-Marques, Edgar Liberis, and Nicholas D. Lane. 2022. Deep Learning on Microcontrollers: A Study on Deployment Costs and Challenges. In Proceedings of the 2nd European Workshop on Machine Learning and Systems (Rennes France, 2022-04-05). ACM, 54--63.

Digital Library

[27]

Wei Tang, Gang Hua, and Liang Wang. 2017. How to Train a Compact Binary Neural Network with High Accuracy?. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (San Francisco, California, USA, 2017-02-04) (AAAI'17). AAAI Press, 2625--2631. https://www.ganghua.org/publication/AAAI17.pdf

Digital Library

[28]

Zhijun Tu, Xinghao Chen, Pengju Ren, and Yunhe Wang. 2022. AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XI (Lecture Notes in Computer Science, Vol. 13671), Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer, 379--395.

Digital Library

[29]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. (2017). arXiv:1706.03762 [cs] http://arxiv.org/abs/1706.03762

[30]

Junru Wu, Yue Wang, Zhenyu Wu, Zhangyang Wang, Ashok Veeraraghavan, and Yingyan Lin. 2018. Deep $k$-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions. (2018). arXiv:1806.09228 [cs, stat] http://arxiv.org/abs/1806.09228

[31]

Michael Zhu and Suyog Gupta. 2017. To Prune, or Not to Prune: Exploring the Efficacy of Pruning for Model Compression. (2017). arXiv:1710.01878 [cs, stat] http://arxiv.org/abs/1710.01878

Cited By

Karras AGiannaros AKarras CTheodorakopoulos LMammassis CKrimpas GSioutas S(2024)TinyML Algorithms for Big Data Management in Large-Scale IoT SystemsFuture Internet10.3390/fi1602004216:2(42)Online publication date: 25-Jan-2024
https://doi.org/10.3390/fi16020042
Huang ZZandberg KSchleiser KBaccelli E(2024)RIOT-ML: toolkit for over-the-air secure updates and performance evaluation of TinyML modelsAnnals of Telecommunications10.1007/s12243-024-01041-5Online publication date: 22-May-2024
https://doi.org/10.1007/s12243-024-01041-5
Lê Mde Foras EArbel J(2023)Regularization for Hybrid N-Bit Weight Quantization of Neural Networks on Ultra-Low Power MicrocontrollersArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44192-9_35(435-446)Online publication date: 22-Sep-2023
https://doi.org/10.1007/978-3-031-44192-9_35

Index Terms

TinyMLOps for real-time ultra-low power MCUs applied to frame-based event classification

Recommendations

Transductive Event Classification through Heterogeneous Networks
WebMedia '17: Proceedings of the 23rd Brazillian Symposium on Multimedia and the Web

Events can be defined as "something that occurs at specific place and time associated with some specific actions". In general, events extracted from news articles and social networks are used to map the information from web to the various phenomena that ...
A convolutional neural network based method for event classification in event-driven multi-sensor network

We investigate the similarities between an actual image and sensor data.We propose a CNN-based method to improve the event classification accuracy.An variant of AlexNet is designed and established for classifying.The results indicate this CNN-based ...
Ultra-Low Power Read-Decoupled SRAMs with Ultra-Low Write-Bitline Voltage Swing

We propose an ultra-low power memory design method based on the ultra-low ( $$\sim $$ ~ 0.2 V) write-bitline voltage swing to reduce the write power dissipation for read-decoupled SRAM (RD-SRAM) cells. By keeping the write bitlines at ground level (0 V) during ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EuroMLSys '23: Proceedings of the 3rd Workshop on Machine Learning and Systems

May 2023

176 pages

ISBN:9798400700842

DOI:10.1145/3578356

Workshop Co-chairs:
Eiko Yoneki
University of Cambridge
,
Luigi Nardi
Lund University
Stanford University

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

TDK InvenSense

Conference

EuroMLSys '23

Sponsor:

SIGOPS

EuroMLSys '23: 3rd Workshop on Machine Learning and Systems

May 8, 2023

Rome, Italy

Acceptance Rates

Overall Acceptance Rate 18 of 26 submissions, 69%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
139
Total Downloads

Downloads (Last 12 months)59
Downloads (Last 6 weeks)12

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Karras AGiannaros AKarras CTheodorakopoulos LMammassis CKrimpas GSioutas S(2024)TinyML Algorithms for Big Data Management in Large-Scale IoT SystemsFuture Internet10.3390/fi1602004216:2(42)Online publication date: 25-Jan-2024
https://doi.org/10.3390/fi16020042
Huang ZZandberg KSchleiser KBaccelli E(2024)RIOT-ML: toolkit for over-the-air secure updates and performance evaluation of TinyML modelsAnnals of Telecommunications10.1007/s12243-024-01041-5Online publication date: 22-May-2024
https://doi.org/10.1007/s12243-024-01041-5
Lê Mde Foras EArbel J(2023)Regularization for Hybrid N-Bit Weight Quantization of Neural Networks on Ultra-Low Power MicrocontrollersArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44192-9_35(435-446)Online publication date: 22-Sep-2023
https://doi.org/10.1007/978-3-031-44192-9_35

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten