Dayane Reis
Xiaobo Sharon Hu
ABSTRACT
Over the past decades, emerging, data-driven machine learning (ML) paradigms have increased in popularity, and revolutionized many application domains. To date, a substantial effort has been devoted to devising mechanisms for facilitating the deployment and near ubiquitous use of these memory intensive ML models. This review paper presents the use of in-memory computing (IMC) accelerators for emerging ML paradigms from a bottom-up perspective through the choice of devices, the design of circuits/architectures, to the application-level results.
- T. Hospedales, et al. Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):5149--5169, 2022.Google Scholar
- S. Bhatia et al. Transformer networks of human conceptual knowledge. Psychological Review, 2021.Google Scholar
- A. Ranjan, et al. X-MANN: A Crossbar Based Architecture for Memory Augmented Neural Networks. In DAC, New York, NY, USA, 2019. ACM.Google Scholar
- W. Kang, et al. In-Memory Processing Paradigm for Bitwise Logic Operations in STT-MRAM. TMAG, 53(11):1--4, 2017.Google Scholar
- D. Reis, et al. Computing in memory with FeFETs. In ISLPED, pages 1--6, 2018.Google ScholarDigital Library
- X. Yin, et al. An Ultra-dense 2FeFET TCAM Design based on a Multi-Domain FeFET Model. IEEE TCAS II: Express Briefs, pages 1--1, 2018.Google Scholar
- S. Jeloka, et al. A 28 nm Configurable Memory (TCAM/BCAM/SRAM) Using Push-Rule 6T Bit Cell Enabling Logic-in-Memory. JSSC, 51(4):1009--1021, 2016.Google Scholar
- J. Zhang, et al. In-Memory Computation of a Machine Learning Classifier in a Standard 6T SRAM Array. JSSC, 52(4):915--924, 2017.Google Scholar
- S. Dunkel, et al. A FeFET based super-low-power ultra-fast embedded NVM technology for 22nm FDSOI and beyond. In IEDM, 2017.Google ScholarCross Ref
- Q. Xia et al. Memristive crossbar arrays for brain-inspired computing. Nature materials, 18(4):309--323, 2019.Google ScholarCross Ref
- Y. Pan, et al. A rram-based associative memory cell. In 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1--5, 2021.Google ScholarCross Ref
- A. Kazemi, et al. Fefet multi-bit content-addressable memories for in-memory nearest neighbor search. IEEE Transactions on Computers, 71(10):2565--2576, 2022.Google ScholarDigital Library
- S. Jain, et al. Computing in Memory With Spin-Transfer Torque Magnetic RAM. TVLSI, PP(99):1--14, 2017.Google Scholar
- S. Aga, et al. Compute caches. In 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA), pages 481--492, 2017.Google ScholarCross Ref
- X. Zhang, et al. FeMAT: Exploring In-Memory Processing in Multifunctional FeFET-Based Memory Array. In ICCD, pages 541--549, 2019.Google ScholarCross Ref
- D. Reis, et al. Attention-in-Memory for Few-Shot Learning with Configurable Ferroelectric FET Arrays. In ASP-DAC, 2021. (In press).Google ScholarDigital Library
- N. H. Weste et al. CMOS VLSI design: a circuits and systems perspective. Pearson Education India, 2015.Google Scholar
- G. E. Moore et al. Cramming more components onto integrated circuits, 1965.Google Scholar
- R. D. Isaac. The future of cmos technology. IBM J. Res. Dev., 44(3):369--378, may 2000.Google ScholarDigital Library
- Z. Shen, et al. Advances of rram devices: Resistive switching mechanisms, materials and bionic synaptic application. Nanomaterials, 10(8), 2020.Google Scholar
- C. H. Cheng et al. Low-leakage-current dram-like memory using a one-transistor ferroelectric mosfet with a hf-based gate dielectric. IEEE Electron Device Lett., 35(1):138--140, Jan 2014.Google ScholarCross Ref
- K. Ni, et al. Ferroelectric ternary content-addressable memory for one-shot learning. Nature Electronics, 2(11):521--529, 2019.Google ScholarCross Ref
- D. Reis, et al. Exploiting fefets via cross-layer design from in-memory computing circuits to meta-learning applications. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 306--311. IEEE, 2021.Google Scholar
- A. Shafiee, et al. Isaac: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In ACM/IEEE 43rd ISCA, pages 14--26, 2016.Google ScholarDigital Library
- X. Yin, et al. Ferroelectric fet based tcam designs for energy efficient computing. In 2019 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), pages 437--442. IEEE, 2019.Google ScholarCross Ref
- S. Li, et al. Pinatubo: A processing-in-memory architecture for bulk bitwise operations in emerging non-volatile memories. In Proceedings of the 53rd Annual Design Automation Conference, DAC '16, New York, NY, USA, 2016. Association for Computing Machinery.Google ScholarDigital Library
- Q. Huang, et al. Computing-in-memory using ferroelectrics: From single-to multi-input logic. IEEE Design & Test, 39(2):56--64, 2021.Google ScholarCross Ref
- D. Reis, et al. A fast and energy efficient computing-in-memory architecture for few-shot learning applications. In 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 127--132. IEEE, 2020.Google Scholar
- M. Li, et al. Imars: An in-memory-computing architecture for recommendation systems. In Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC '22, page 463--468, New York, NY, USA, 2022. Association for Computing Machinery.Google ScholarDigital Library
- A. Vaswani, et al. Attention is all you need. In I. Guyon, et al., editors, Advances in Neural Information Processing Systems, volume 30, pages 5998--6008. Curran Associates, Inc., 2017.Google Scholar
- M. Marcus, et al. Building a large annotated corpus of english: The penn treebank. 1993.Google ScholarCross Ref
- J. W. Rae, et al. Compressive transformers for Long-Range sequence modelling. Nov. 2019.Google Scholar
- W. Fedus, et al. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Jan. 2021.Google Scholar
- A. F. Laguna, et al. Hardware-Software Co-Design of an In-Memory transformer network accelerator. Frontiers in Electronics, 3, 2022.Google Scholar
- A. F. Laguna, et al. In-Memory computing based accelerator for transformer networks for long sequences. In 2021 Design, Automation Test in Europe Conference Exhibition (DATE), pages 1839--1844, Feb. 2021.Google Scholar
- M. Naumov, et al. Deep learning recommendation model for personalization and recommendation systems. CoRR, abs/1906.00091, 2019.Google Scholar
- P. Covington, et al. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems, 2016.Google ScholarDigital Library
Index Terms
- In-Memory Computing Accelerators for Emerging Learning Paradigms
Recommendations
UbiPhone: Human-Centered Ubiquitous Phone System
Emerging rich wireless networking modalities facilitate the development of new intelligent, innovative services on smart phones. The authors propose a ubiquitous phone (UbiPhone) system that demonstrates innovative context-aware human-centric phone ...
Learning in Ubiquitous Computing Environments
The application of ubiquitous technologies in the improvement of education strategies is called Ubiquitous Learning. GlobalEdu is a model created to support ubiquitous learning. The model has the necessary support to implement learning-related ...
Anticipating ethical issues in emerging IT
In this essay, a new approach to the ethics of emerging information technology will be presented, called anticipatory technology ethics (ATE). The ethics of emerging technology is the study of ethical issues at the R&D and introduction stage of ...
Comments