Shankar Ganesh Ramasubramanian
Anand Raghunathan
ABSTRACT
Deep Learning Networks (DLNs) are bio-inspired large-scale neural networks that are widely used in emerging vision, analytics, and search applications. The high computation and storage requirements of DLNs have led to the exploration of various avenues for their efficient realization. Concurrently, the ability of emerging post-CMOS devices to efficiently mimic neurons and synapses has led to great interest in their use for neuromorphic computing.
We describe SPINDLE, a programmable processor for deep learning based on spintronic devices. SPINDLE exploits the unique ability of spintronic devices to realize highly dense and energy-efficient neurons and memory, which form the fundamental building blocks of DLNs. SPINDLE consists of a three-tier hierarchy of processing elements to capture the nested parallelism present in DLNs, and a two-level memory hierarchy to facilitate data reuse. It can be programmed to execute DLNs with widely varying topologies for different applications. SPINDLE employs techniques to limit the overheads of spin-to-charge conversion, and utilizes output and weight quantization to enhance the efficiency of spin-neurons. We evaluate SPINDLE using a device-to-architecture modeling framework and a set of widely used DLN applications (handwriting recognition, face detection, and object recognition). Our results indicate that SPINDLE achieves 14.4X reduction in energy consumption and 20.4X reduction in EDP over the CMOS baseline under iso-area conditions.
- Y. LeCun et al. Gradient-based learning applied to document recognition. In Proc. IEEE, 1998.Google Scholar
- G. Hinton et al. A fast learning algorithm for deep belief nets. Trans. Neural Computation, 2006. Google ScholarDigital Library
- Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2009. Google ScholarDigital Library
- George Rosenberg. Improving photo search: A step across the semantic gap http://googleresearch.blogspot.com/2013/06/improving-photo-search-step-across.html. June 2009.Google Scholar
- Y. Netzer et al. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.Google Scholar
- J. Bergstra. Theano: Deep learning on GPUs with Python. In Big Learn workshop NIPS, 2011.Google Scholar
- M. Sankaradas et al. A massively parallel coprocessor for convolutional neural networks. In Proc. ASAP, 2009. Google ScholarDigital Library
- C. Farabet et al. Hardware accelerated convolutional neural networks for synthetic vision systems. In Proc. ISCAS, 2010.Google ScholarCross Ref
- E. Chen et al. Advances and future prospects of spin-transfer torque random access memory. IEEE Trans. Magnetics, 46(6):1873--1878, June 2010.Google ScholarCross Ref
- M. Sharad et al. Boolean and non-boolean computation with spin devices. In Proc. IEDM, 2012.Google ScholarCross Ref
- M. Sharad et al. Spin neuron for ultra low power computational hardware. In Proc. DRC, 2012.Google ScholarCross Ref
- T. Kimura et al. Switching magnetization of a nanoscale ferromagnetic particle using nonlocal spin injection. Phys. Rev. Lett., 2006.Google ScholarCross Ref
- R. Venkatesan et al. DWM-TAPESTRI - An energy efficient all-spin cache using domain wall shift based writes. In Proc. DATE, 2013. Google ScholarDigital Library
- G.D. Panagopoulos et al. Physics-Based SPICE-Compatible Compact Model for Simulating Hybrid MTJ/CMOS Circuits. Trans. TED, 2013.Google ScholarCross Ref
- L Gao et al. Analog-input analog-weight dot-product operation with Ag/a-Si/Pt memristive devices. In Proc. VLSI-SoC, 2012.Google Scholar
- CACTI. http://www.hpl.hp.com/research/cacti/.Google Scholar
- R. Venkatesan et al. TapeCache: A High Density, Energy Efficient Cache Based on Domain Wall Memory. In Proc. ISLPED, 2012. Google ScholarDigital Library
- R. Collobert et al. Torch7: A Matlab-like environment for machine learning. In BigLearn, NIPS Workshop, 2011.Google Scholar
- Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.Google Scholar
- E. Painkras et al. Spinnaker: A multi-core system-on-chip for massively-parallel neural net simulation. In Proc. CICC, 2012.Google ScholarCross Ref
- J. Seo et al. A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons. In Proc. CICC, 2011.Google ScholarCross Ref
- B. Rajendran et al. Specifications of nanoscale devices and circuits for neuromorphic computational systems. Trans. TED, 2013.Google ScholarCross Ref
- D. Kuzum et al. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Letters, 2012.Google ScholarCross Ref
- S. H. Jo et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Letters, 2010.Google ScholarCross Ref
Index Terms
- SPINDLE: SPINtronic deep learning engine for large-scale neuromorphic computing
Recommendations
Potential nanoelectronic integrated circuit technologies
Special issue on nanotechnologyWe discuss the opportunities for Nanoelectronic devices based on resonant tunneling to impact integrated circuit (IC) technology. We believe that resonant tunneling diodes (RTDs) integrated with conventional compound semiconductor transistors will be ...
A Pathway to Enable Exponential Scaling for the Beyond-CMOS Era: Invited
DAC '17: Proceedings of the 54th Annual Design Automation Conference 2017Many key technologies of our society, including so-called artificial intelligence (AI) and big data, have been enabled by the invention of transistor and its ever-decreasing size and ever-increasing integration at a large scale. However, conventional ...
Effects of Noise on Leaky Integrate-and-Fire Neuron Models for Neuromorphic Computing Applications
Computational Science and Its Applications – ICCSA 2022AbstractArtificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), ...
Comments