Abstract
Purpose of the study is to improve efficiency of distributed transducer networks, which use models and methods of machine learning to analyze measures from sensors and perform reasoning on this measured data. The proposed model makes it possible to process data from different types of sensors to solve classification and regression problems. The model has hierarchical structure consisting of three-level calculators, which allows it to be used in the edge computing approach and to process data from different types of sensors, with varying measures dimensions and frequency of new data. To form the input tensor of a hierarchical neural network model, an algorithm for pre-processing and formatting input data from measures of different sensor types is proposed, considering the spatial and temporal characteristics of the obtained measurements. Also a method of neural networks compression based on hidden layers neurons pruning is proposed. Suggested method implements an unified approach to the compression of convolutional, recurrent and fully connected neural networks to solve classification and regression problems. The core mechanism of the proposed method is based on the dropout operation, which is used as a technique of neural network regularization. The method estimates rational probability of hidden layers neurons dropout on the basis of the neuron excessiveness parameter, which is estimated with the help of a special “trimmer” network. The proposed techniques could reduce time, memory and energy consumption for the neural network inference.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Assunção, M.D., Calheiros, R.N., Bianchi, S., Netto, M.A., Buyya, R.: Big data computing and clouds: trends and future directions. J. Parallel Distrib. Comput. 79–80, 3–15 (2015). https://doi.org/10.1016/j.jpdc.2014.08.003
Athmaja, S., Hanumanthappa, M., Kavitha, V.: A survey of machine learning algorithms for big data analytics. In: 2017 International Conference on Innovations in Information, Embedded and Communication Systems (ICIIECS), pp. 1–4 (2017). https://doi.org/10.1109/ICIIECS.2017.8276028
Aziz, F., Chalup, S.K., Juniper, J.: Big data in IoT systems. In: Khan, J.Y., Yuce, M.R. (eds.) Internet of Things (IoT): Systems and Applications, chap. 2. Pan Stanford Publishing Pte. Ltd., Singapore (2019)
Bhattacharya, S., Lane, N.D.: Sparsification and separation of deep learning layers for constrained resource inference on wearables. In: Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM, SenSys 2016, pp. 176–189. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2994551.2994564
Cheng, Y., Yu, F.X., Feris, R.S., Kumar, S., Choudhary, A., Chang, S.F.: An exploration of parameter redundancy in deep networks with circulant projections (2015). https://doi.org/10.48550/ARXIV.1502.03436. https://arxiv.org/abs/1502.03436
El-Sayed, H., et al.: Edge of things: the big picture on the integration of edge, IoT and the cloud in a distributed computing environment. IEEE Access 6, 1706–1717 (2018). https://doi.org/10.1109/ACCESS.2017.2780087
Gal, Y., Ghahramani, Z.: Bayesian convolutional neural networks with Bernoulli approximate variational inference (2015). https://doi.org/10.48550/ARXIV.1506.02158. https://arxiv.org/abs/1506.02158
Guo, Y., Yao, A., Chen, Y.: Dynamic network surgery for efficient DNNs (2016). https://doi.org/10.48550/ARXIV.1608.04493. https://arxiv.org/abs/1608.04493
Lin, T.: Deep learning for IoT (2021). https://doi.org/10.48550/ARXIV.2104.05569. https://arxiv.org/abs/2104.05569
Lobachev, I., Antoshcuk, S., Hodovychenko, M.: Distributed deep learning framework for smart building transducer network. Appl. Aspects Inf. Technol. 4(2), 127–139 (2021). https://doi.org/10.15276/aait.02.2021.1
Lobachev, I., Antoshcuk, S., Hodovychenko, M.: Methodology of neural network compression for multi-sensor transducer network models based on edge computing principles. Herald Adv. Inf. Technol. 4(3), 232–243 (2021). https://doi.org/10.15276/hait.03.2021.3
Mahdavinejad, M.S., Rezvan, M., Barekatain, M., Adibi, P., Barnaghi, P., Sheth, A.P.: Machine learning for internet of things data analysis: a survey. Digit. Commun. Netw. 4(3), 161–175 (2018). https://doi.org/10.1016/j.dcan.2017.10.002
Marco, V.S., Taylor, B., Wang, Z., Elkhatib, Y.: Optimizing deep learning inference on embedded systems through adaptive model selection (2019). https://doi.org/10.48550/ARXIV.1911.04946. https://arxiv.org/abs/1911.04946
Mishra, R., Gupta, H.P., Dutta, T.: A survey on deep neural network compression: challenges, overview, and solutions (2020). https://doi.org/10.48550/ARXIV.2010.03954. https://arxiv.org/abs/2010.03954
Ranjan, R., et al.: City data fusion: sensor data fusion in the internet of things. Int. J. Distrib. Syst. Technol. 7(1), 15–36 (2016). https://doi.org/10.4018/IJDST.2016010102
Samie, F., Bauer, L., Henkel, J.: IoT technologies for embedded computing: a survey. In: Proceedings of the Eleventh IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis, CODES 2016. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2968456.2974004
Stisen, A., et al.: Smart devices are different: assessing and mitigating mobile sensing heterogeneities for activity recognition. In: Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, SenSys 2015, pp. 127–140. Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2809695.2809718
Sulieman, N.A., Ricciardi Celsi, L., Li, W., Zomaya, A., Villari, M.: Edge-oriented computing: a survey on research and use cases. Energies 15(2) (2022). https://doi.org/10.3390/en15020452. https://www.mdpi.com/1996-1073/15/2/452
Xia, F., Tian, Y.C., Li, Y., Sun, Y.: Wireless sensor/actuator network design for mobile control applications (2008). https://doi.org/10.48550/ARXIV.0806.1569. https://arxiv.org/abs/0806.1569
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hodovychenko, M., Antoshchuk, S., Lobachev, I., Schöler, T., Lobachev, M. (2023). Approaches and Techniques to Improve Machine Learning Performance in Distributed Transducer Networks. In: Babichev, S., Lytvynenko, V. (eds) Lecture Notes in Data Engineering, Computational Intelligence, and Decision Making. ISDMCI 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 149. Springer, Cham. https://doi.org/10.1007/978-3-031-16203-9_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-16203-9_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16202-2
Online ISBN: 978-3-031-16203-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)