Abstract
Approximate Computing is a design principle which has been extensively used to trade in accuracy for energy efficiency in error-resilient applications such as signal processing, machine learning (ML), and embedded systems. To reap maximum energy benefits as well as ensure high quality of solution for applications, innovations are needed across the entire computing stack (from circuits and architectures all the way up to algorithms). This chapter discusses different approximate computing techniques in the context of machine learning applications by considering different approximate hardware primitives such as multipliers, adders, memories, and matrix vector multiplication units which are vital to build a machine learning accelerator. Several algorithm techniques that can utilize the aforementioned hardware level approximations to accelerate machine learning workloads are also discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu W, Lombardi F, Schulte M. Approximate computing: from circuits to applications. Proc IEEE. 2020;108(12):2103–7
Esmaeilzadeh H, Blem E, Amant RS, Sankaralingam K, Burger D. Dark silicon and the end of multicore scaling. In: 2011 38th annual international symposium on computer architecture (ISCA), 2011 Jun 4. New York: IEEE; 2011, pp 365–376
Venkataramani S, Chakradhar ST, Roy K, Raghunathan A. Approximate computing and the quest for computing efficiency. In: Proceedings of the 52nd Annu. Des. Automat. Conf., San Francisco, CA, Jun 2015; pp. 120-1–120-6
Chippa VK et al. Analysis and characterization of inherent application resilience for approximate computing. In Proceedings of the DAC; 2013
Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Adv Neur Inf Process Syst 2012;25:1097–105
Park J, Choo H, Muhammad K, Choi S, Im Y, Roy K. Non-adaptive and adaptive filter implementation based on sharing multiplication. In IEEE international conference on acoustics, speech, and signal processing (ICASSP); 2000, pp 460–463
Sarwar SS, Venkataramani S, Raghunathan A, Roy K. Multiplier-less artificial neurons exploiting error resiliency for energy-efficient neural computing. In 2016 design, automation & test in Europe conference & exhibition (DATE); 2016, pp 145–150
Gupta V, Mohapatra D, Park SP, Raghunathan A, Roy K. IMPACT: IMPrecise adders for low-power approximate computing. In IEEE/ACM international symposium on low power electronics and Design 2011 Aug 1. New York: IEEE; 2011, pp 409–414
Rabaey JM. Digital integrated circuits: a design perspective. Upper Saddle River, NJ: Prentice-Hall, Inc.; 1996
Nassif SR. Modeling and analysis of manufacturing variations. In Proc. IEEE conf. custom integr. circuits, May 2001, pp 223–228
Visweswariah C. Death, taxes and failing chips. In Proceedings of the 40th annual design automation conference (DAC); 2003, pp 343–347
Borkar S, Karnik T, Narendra S, Tschanz J, Keshavarzi A, De V. Parameter variations and impact on circuits and microarchitecture. In Proceedings of the 40th annu. des. automat. conf.; 2003, pp 338–342
Srinivasan G, Wijesinghe P, Sarwar SS, Jaiswal A, Roy K. Significance driven hybrid 8T-6T SRAM for energy-efficient synaptic storage in artificial neural networks. In Design, automation & test in Europe conference & exhibition (DATE), 2016 Mar 14. New York: IEEE; 2016, pp 151–156
Verma N, Jia H, Valavi H, Tang Y, Ozatay M, Chen LY, Zhang B, Deaville P. In-memory computing: advances and prospects. In IEEE solid-state circuits magazine, 2019 Aug 23; 2019, pp 11(3):43–55
Jouppi NP, Young C, Patil N, Patterson D, Agrawal G, Bajwa R, Bates S, Bhatia S, Boden N, Borchers A, Boyle R. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th annual international symposium on computer architecture (ISCA) 2017 Jun 24; 2017, pp 1–12
Sharma H, Park J, Suda N, Lai L, Chau B, Chandra V, Esmaeilzadeh H. Bit fusion: bit-level dynamically composable architecture for accelerating deep neural network. In ACM/IEEE 45th annual international symposium on computer architecture (ISCA), 2018 Jun 1; 2018, pp 764–775
Ryu S, Kim H, Yi W, Kim JJ. Bitblade: area and energy-efficient precision-scalable neural network accelerator with bitwise summation. In Proceedings of the 56th annual design automation conference (DAC), 2019 Jun 2; 2019, pp 1–6
Ankit A, Hajj IE, Chalamalasetti SR, Ndu G, Foltin M, Williams RS, Faraboschi P, Hwu WM, Strachan JP, Roy K, Milojicic DS. PUMA: a programmable ultra-efficient memristor-based accelerator for machine learning inference. In Proceedings of the twenty-fourth international conference on architectural support for programming languages and operating systems, 2019 Apr 4; 2019, pp. 715–731.
Hu M, Graves CE, Li C, Li Y, Ge N, Montgomery E, Davila N, Jiang H, Williams RS, Yang JJ, Xia Q. Memristor-based analog computation and neural network classification with a dot product engine. Adv Mater. 2018;30(9):1705914
Hu M, Strachan JP, Li Z, Grafals EM, Davila N, Graves C, Lam S, Ge N, Yang JJ, Williams RS. Dot-product engine for neuromorphic computing: programming 1T1M crossbar to accelerate matrix-vector multiplication. In 53nd ACM/EDAC/IEEE design automation conference (DAC), 2016 Jun 5. New York: IEEE; 2016, pp 1–6
Sanyal S, Ankit A, Vineyard CM, Roy K. Energy-efficient target recognition using ReRAM crossbars for enabling on-device intelligence. In IEEE workshop on signal processing systems (SiPS); 2020, pp 1–6
Chakraborty I, Ali MF, Kim DE, Ankit A, Roy K. Geniex: a generalized approach to emulating non-ideality in memristive xbars using neural networks. In 57th ACM/IEEE design automation conference (DAC), 2020 Jul 20; 2020, pp. 1–6
Reagen B, et al. Minerva: enabling low-power, highly-accurate deep neural network accelerators. In Proceedings of 43rd international symposium of computer architecture (ISCA); 2016, pp 267–278
Han S, Mao H, Dally WJ. Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. [Online]; 2015. Available: https://arxiv.org/abs/1510.00149
Sarwar SS, Srinivasan G, Han B, Wijesinghe P, Jaiswal A, Panda P, Raghunathan A, Roy K. Energy efficient neural computing: a study of cross-layer approximations. IEEE J Emerg Select Top Circ Syst. 2018;8(4):796–809
Muralimanohar N, Balasubramonian R, Jouppi NP. CACTI 6.0: a tool to model large caches. HP Lab., Palo Alto, CA; 2009, pp 22–31
Palm RB. Prediction as a candidate for learning deep hierarchical models of data. M.S. thesis, Tech. Univ. Denmark, Denmark, Lyngby; 2012, vol 5.
Vedaldi A, Lenc K, MatConvNet: convolutional neural networks for MATLAB. In Proc. 23rd ACM int. conf. multimedia; 2015, pp 689–692
Lin M, Chen Q, Yan S. Network in network; 2013 [Online]. Available: https://arxiv.org/abs/1312.4400
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Sanyal, S., Negi, S., Raghunathan, A., Roy, K. (2022). Approximate Computing for Machine Learning Workloads: A Circuits and Systems Perspective. In: Liu, W., Lombardi, F. (eds) Approximate Computing. Springer, Cham. https://doi.org/10.1007/978-3-030-98347-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-98347-5_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98346-8
Online ISBN: 978-3-030-98347-5
eBook Packages: Computer ScienceComputer Science (R0)