Abstract
Traditional tiny machine learning systems are widely employed because of their limited energy consumption, fast execution, and easy deployment. However, such systems have limited access to labelled data and need periodic maintenance due to the evolution of data distribution (i.e., context drift). Continual machine learning algorithms can enable continuous learning on embedded systems by updating their parameters, addressing context drift, and allowing neural networks to learn new categories over time. However, the availability of labelled data is scarce, limiting such algorithms in supervised settings. This paper overcomes this limitation with an alternative approach which combines supervised deep learning with unsupervised clustering to enable unsupervised continual machine learning on the edge. Tiny Neural Deep Clustering (TinyNDC) is deployed in an OpenMV Cam H7 Plus and tested with the MNIST dataset reaching a classification accuracy of 92.3% and a frame rate of 44 FPS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cai H, Gan C, Wang T, Zhang Z, Han S (2020) Once for all: Train one network and specialize it for efficient deployment. In: International conference on learning representations
Pilati F, Sbaragli A (2023) Learning human-process interaction in manual manufacturing job shops through indoor positioning systems. Comput Ind 151:103984
Maltoni D, Lomonaco V (2019) Continuous learning in single-incremental-task scenarios. Neural Netw 116:56–73
Zenke F, Poole B, Ganguli S (2017) Continual learning through synaptic intelligence. In: International conference on machine learning. PMLR, pp 3987–3995
Ren H, Anicic D, Runkler TA (2021) Tinyol: Tinyml with online-learning on microcontrollers. In: 2021 international joint conference on neural networks (IJCNN). IEEE, pp 1–8
Zhan X, Xie J, Liu Z, Ong Y-S, Loy CC (2020)Online deep clustering for unsupervised representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. IEEE, pp 6688–6697
Taufique AMN, Jahan CS, Savakis A (2022) Unsupervised continual learning for gradually varying domains. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3740–3750
Lin J, Zhu L, Chen W-M, Wang W-C, Gan C, Han S (2022) On-device training under 256kb memory. Adv Neural Inf Process Syst 35:22 941–22 954
Albanese A, Nardello M, Fiacco G, Brunelli D (2023) Tiny machine learning for high accuracy product quality inspection. IEEE Sens J 23(2):1575–1583
Nardello M, Desai H, Brunelli D, Lucia B (2019) Camaroptera: a batteryless long-range remote visual sensing system. In: Proceedings of the 7th international workshop on energy harvesting and energy-neutral sensing systems, ser. ENSsys’19. Association for Computing Machinery, New York, NY, USA, pp 8–14
Desai H, Nardello M, Brunelli D, Lucia B (2022) Camaroptera: a long-range image sensor with local inference for remote sensing applications. ACM Trans Embed Comput Syst 21(3)
Albanese A, Nardello M, Brunelli D (2021) Automated pest detection with DNN on the edge for precision agriculture. IEEE J Emerg Sel Top Circuits Syst 11(3):458–467
Brunelli D, Albanese A, d’Acunto D, Nardello M (2019) Energy neutral machine learning based iot device for pest detection in precision agriculture. IEEE Internet Things Mag 2(4):10–13
Muhammad K, Hussain T, Del Ser J, Palade V, de Albuquerque VHC (2020) Deepres: A deep learning-based video summarization strategy for resource-constrained industrial surveillance scenarios. IEEE Trans Ind Inform 16(9):5938–5947
Thandiackal K, Piccinelli L, Pati P, Goksel O (2023) Multi-scale feature alignment for continual learning of unlabeled domains. arXiv:2302.01287
Acknowledgments
This work was partially supported by the GEMINI (”Green Machine Learning for the IoT”) national research project, funded by the Italian Ministry for University and Research (MUR) by the PRIN 2022 programme (Contract 20223M4HZ4). Moreover, this work was supported by the Italian Ministry for University and Research (MUR) under the program ”Dipartimenti di Eccellenza (2023-2027)”. Further, this work is based on and improves the achievements of iNEST (interconnected NordEst innovation Ecosystem, Project ID: ECS00000043) PNRR project (Mission 4.2, Investment 1.5) Spoke 3 ”Green and digital transition for advanced manufacturing technology”, funded by the European Commission under the NextGeneration EU programme.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Poletti, G., Albanese, A., Nardello, M., Brunelli, D. (2024). Tiny Neural Deep Clustering: An Unsupervised Approach for Continual Machine Learning on the Edge. In: Bellotti, F., et al. Applications in Electronics Pervading Industry, Environment and Society. ApplePies 2023. Lecture Notes in Electrical Engineering, vol 1110. Springer, Cham. https://doi.org/10.1007/978-3-031-48121-5_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-48121-5_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48120-8
Online ISBN: 978-3-031-48121-5
eBook Packages: EngineeringEngineering (R0)