Abstract
The deployment of highly parameterized Neural Network (NN) models on resource-constrained hardware platforms such as IoT edge devices is a challenging task due to their large size, expensive computational costs, and high memory requirements. To address this, we propose a Simulated Annealing (SA) algorithm-based NN optimization approach to generate area-optimized hardware for multilayer perceptrons on IoT edge devices. Our SA loop aims to change hidden layer weights to integer values and uses a two-step process to round new weights that are proximate to integers to reduce the hardware due to operation strength reduction, making it a perfect solution for IoT devices. Throughout the optimization process, we prioritize SA moves that do not compromise the model’s efficiency, ensuring optimal performance in a resource-constrained environment. We validate our proposed methodology on five MLP benchmarks implemented on FPGA, and we observe that the best-case savings are obtained when the amount of perturbation (p) is 10% and the number of perturbations at each temperature (N) is 10,000, keeping constant temperature reduction function (\(\alpha \)) at 0.95. For the best-case solution, the average savings in Lookup Tables (LUTs) and filpflops (FFs) are 24.83% and 25.76%, respectively, with an average model accuracy degradation of 1.64%. Our proposed SA-based NN optimization method can significantly improve the deployment of area-efficient NN models on resource-constrained IoT edge devices without compromising model accuracy, making it a promising approach for various IoT applications.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ayres-de Campos, D., Bernardes, J., Garrido, A., Marques-de Sa, J., Pereira-Leite, L.: Sisporto 2.0: a program for automated analysis of cardiotocograms. J. Maternal-Fetal Med. 9(5), 311–318 (2000)
Dal Pozzolo, A.: Adaptive Machine Learning for Credit Card Fraud detection (2015)
Detrano, R.: UCI Machine Learning Repository: Heart Disease Data Set (2019)
Fausett, L.V.: Fundamentals of Neural Networks: Architectures, Algorithms and Applications. Pearson Education India, Noida (2006)
Fisher, R.A.: UCI Machine Learning Repository: Iris Data Set (2011)
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M.W., Keutzer, K.: A survey of quantization methods for efficient neural network inference. arXiv preprint arXiv:2103.13630 (2021)
Hu, P., Peng, X., Zhu, H., Aly, M.M.S., Lin, J.: OPQ: compressing deep neural networks with one-shot pruning-quantization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 7780–7788 (2021)
Joshi, R., Zaman, M.A., Katkoori, S.: Novel bit-sliced near-memory computing based VLSI architecture for fast sobel edge detection in IoT edge devices. In: 2020 IEEE International Symposium on Smart Electronic Systems (iSES)(Formerly iNiS), pp. 291–296. IEEE (2020)
Joshi, R., Kalyanam, L.K., Katkoori, S.: Simulated annealing based integerization of hidden weights for area-efficient IoT edge intelligence. In: 2022 IEEE International Symposium on Smart Electronic Systems (iSES), pp. 427–432 (2022). https://doi.org/10.1109/iSES54909.2022.00093
Joshi, R., Kalyanam, L.K., Katkoori, S.: Area efficient VLSI ASIC implementation of multilayer perceptrons. In: 2023 International VLSI Symposium on Technology, Systems and Applications (VLSI-TSA/VLSI-DAT), pp. 1–4. IEEE (2023)
Joshi, R., Zaman, M.A., Katkoori, S.: Fast Sobel edge detection for IoT edge devices. SN Comput. Sci. 3(4), 302 (2022)
Kaiming, H., Xiangyu, Z., Shaoqing, R., Jian, S.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Kalyanam, L.K., Joshi, R., Katkoori, S.: Range based hardware optimization of multilayer perceptrons with RELUs. In: 2022 IEEE International Symposium on Smart Electronic Systems (iSES), pp. 421–426 (2022). https://doi.org/10.1109/iSES54909.2022.00092
Kirkpatrick, S., Gelatt, C.D., Jr., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)
Lee, N., Ajanthan, T., Torr, P.H.: Snip: single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340 (2018)
Lin, S., Ji, R., Li, Y., Wu, Y., Huang, F., Zhang, B.: Accelerating convolutional networks via global & dynamic filter pruning. In: IJCAI, vol. 2, p. 8. Stockholm (2018)
Mamalet, F., Garcia, C.: Simplifying ConvNets for fast learning. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds.) ICANN 2012. LNCS, vol. 7553, pp. 58–65. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33266-1_8
Park, S., Lee, J., Mo, S., Shin, J.: Lookahead: a far-sighted alternative of magnitude-based pruning. arXiv preprint arXiv:2002.04809 (2020)
Wolberg, W., Street, W., Mangasarian, O.: Breast cancer wisconsin (diagnostic). UCI Machine Learning Repository (1995)
Wu, B., et al.: Shift: a zero flop, zero parameter alternative to spatial convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9127–9135 (2018)
Yu, R., et al.: NISP: pruning networks using neuron importance score propagation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9194–9203 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 IFIP International Federation for Information Processing
About this paper
Cite this paper
Joshi, R., Kalyanam, L.K., Katkoori, S. (2024). Simulated Annealing Based Area Optimization of Multilayer Perceptron Hardware for IoT Edge Devices. In: Puthal, D., Mohanty, S., Choi, BY. (eds) Internet of Things. Advances in Information and Communication Technology. IFIPIoT 2023. IFIP Advances in Information and Communication Technology, vol 683. Springer, Cham. https://doi.org/10.1007/978-3-031-45878-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-45878-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45877-4
Online ISBN: 978-3-031-45878-1
eBook Packages: Computer ScienceComputer Science (R0)