research-article

COSIME: FeFET Based Associative Memory for In-Memory Cosine Similarity Search

Authors:

Ann Franchesca Laguna,

Michael Niemier,

Xiaobo Sharon Hu,

Xunzhao YinAuthors Info & Claims

ICCAD '22: Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design

Article No.: 137, Pages 1 - 9

https://doi.org/10.1145/3508352.3549412

Published: 22 December 2022 Publication History

Abstract

In a number of machine learning models, an input query is searched across the trained class vectors to find the closest feature class vector in cosine similarity metric. However, performing the cosine similarities between the vectors in Von-Neumann machines involves a large number of multiplications, Euclidean normalizations and division operations, thus incurring heavy hardware energy and latency overheads. Moreover, due to the memory wall problem that presents in the conventional architecture, frequent cosine similarity-based searches (CSSs) over the class vectors requires a lot of data movements, limiting the throughput and efficiency of the system. To overcome the aforementioned challenges, this paper introduces COSIME, a general in-memory associative memory (AM) engine based on the ferroelectric FET (FeFET) device for efficient CSS. By leveraging the one-transistor AND gate function of FeFET devices, current-based translinear analog circuit and winner-take-all (WTA) circuitry, COSIME can realize parallel in-memory CSS across all the entries in a memory block, and output the closest word to the input query in cosine similarity metric. Evaluation results at the array level suggest that the proposed COSIME design achieves 333× and 90.5× latency and energy improvements, respectively, and realizes better classification accuracy when compared with an AM design implementing approximated CSS. The proposed in-memory computing fabric is evaluated for an HDC problem, showcasing that COSIME can achieve on average 47.1× and 98.5× speedup and energy efficiency improvements compared with an GPU implementation.

References

[1]

Z. Gong, P. Zhong, Y. Yu, and W. Hu, "Diversity-promoting deep structural metric learning for remote sensing scene classification," IEEE Transactions on Geoscience and Remote Sensing (TGRS), vol. 56, no. 1, pp. 371--390, 2017.

[2]

S. Salahuddin, K. Ni, and S. Datta, "The era of hyper-scaling in electronics," Nature Electronics, vol. 1, no. 8, pp. 442--450, 2018.

[3]

X. S. Hu, M. Niemier, A. Kazemi, A. F. Laguna, K. Ni, R. Rajaei, M. M. Sharifi, and X. Yin, "In-memory computing with associative memories: A cross-layer perspective," in 2021 IEEE International Electron Devices Meeting (IEDM), pp. 25--2, IEEE, 2021.

[4]

H. Amrouch, D. Gao, X. S. Hu, A. Kazemi, A. F. Laguna, K. Ni, M. Niemier, M. M. Sharifi, S. Thomann, X. Yin, et al., "Iccad tutorial session paper ferroelectric fet technology and applications: From devices to systems," in 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1--8, IEEE, 2021.

[5]

R. Karam, R. Puri, S. Ghosh, and S. Bhunia, "Emerging trends in design and applications of memory-based computing and content-addressable memories," Proceedings of the IEEE, vol. 103, no. 8, pp. 1311--1330, 2015.

[6]

K. Ni, X. Yin, A. F. Laguna, S. Joshi, S. Dünkel, M. Trentzsch, J. Müller, S. Beyer, M. Niemier, X. S. Hu, et al., "Ferroelectric ternary content-addressable memory for one-shot learning," Nature Electronics, vol. 2, no. 11, pp. 521--529, 2019.

[7]

A. Kazemi, M. M. Sharifi, A. F. B. Laguna, F. Muller, X. Yin, T. Kampfe, M. Niemier, and X. S. Hu, "Fefet multi-bit content-addressable memories for in-memory nearest neighbor search," IEEE Transactions on Computers (TC), 2021.

[8]

A. Kazemi, M. M. Sharifi, A. F. Laguna, F. Müller, R. Rajaei, R. Olivo, T. Kämpfe, M. Niemier, and X. S. Hu, "In-memory nearest neighbor search with fefet multi-bit content-addressable memories," in Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1084--1089, IEEE, 2021.

[9]

M. Imani, A. Rahimi, D. Kong, T. Rosing, and J. M. Rabaey, "Exploring hyperdimensional associative memory," in IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 445--456, IEEE, 2017.

[10]

G. Karunaratne, M. Schmuck, M. Le Gallo, G. Cherubini, L. Benini, A. Sebastian, and A. Rahimi, "Robust high-dimensional memory-augmented neural networks," Nature communications, vol. 12, no. 1, pp. 1--12, 2021.

[11]

T. Böscke, J. Müller, D. Bräuhaus, U. Schröder, and U. Böttger, "Ferroelectricity in hafnium oxide: Cmos compatible ferroelectric field effect transistors," in IEEE International Electron Devices Meeting (IEDM), 2011.

[12]

T. Soliman, F. Müller, T. Kirchner, T. Hoffmann, H. Ganem, E. Karimov, T. Ali, M. Lederer, C. Sudarshan, T. Kämpfe, et al., "Ultra-low power flexible precision fefet based analog in-memory computing," in IEEE International Electron Devices Meeting (IEDM), 2020.

[13]

X. Yin, F. Müller, Q. Huang, C. Li, M. Imani, Z. Yang, J. Cai, M. Lederer, R. Olivo, N. Laleni, et al., "An ultra-compact single fefet binary and multi-bit associative search engine," arXiv preprint arXiv:2203.07948, 2022.

[14]

D. Saito, T. Kobayashi, H. Koga, N. Ronchi, K. Banerjee, Y. Shuto, J. Okuno, K. Konishi, L. D. Piazza, A. Mallik, J. V. Houdt, M. Tsukamoto, K. Ohkuri, T. Umebayashi, and T. Ezaki, "Analog in-memory computing in fefet-based 1t1r array for edge ai applications," in Symposium on VLSI Technology, 2021.

[15]

X. Yin, K. Ni, D. Reis, S. Datta, M. Niemier, and X. S. Hu, "An ultra-dense 2fefet tcam design based on a multi-domain fefet model," IEEE Transactions on Circuits and Systems II: Express Briefs (TCAS-II), vol. 66, no. 9, pp. 1577--1581, 2018.

[16]

T. Kohonen, Associative memory: A system-theoretical approach, vol. 17. Springer Science & Business Media, 2012.

[17]

X. Yin, C. Li, Q. Huang, L. Zhang, M. Niemier, X. S. Hu, C. Zhuo, and K. Ni, "Fecam: A universal compact digital and analog content addressable memory using ferroelectric," IEEE Transactions on Electron Devices (TED), vol. 67, no. 7, pp. 2785--2792, 2020.

[18]

C. Li, F. Müller, T. Ali, R. Olivo, M. Imani, S. Deng, C. Zhuo, T. Kämpfe, X. Yin, and K. Ni, "A scalable design of multi-bit ferroelectric content addressable memory for data-centric computing," in 2020 IEEE International Electron Devices Meeting (IEDM), pp. 29--3, IEEE, 2020.

[19]

R. Rajaei, M. M. Sharifi, A. Kazemi, M. Niemier, and X. S. Hu, "Compact single-phase-search multistate content-addressable memory design using one fefet/cell," IEEE Transactions on Electron Devices (TED), vol. 68, no. 1, pp. 109--117, 2020.

[20]

A. F. Laguna, H. Gamaarachchi, X. Yin, M. Niemier, S. Parameswaran, and X. S. Hu, "Seed-and-vote based in-memory accelerator for dna read mapping," in 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1--9, IEEE, 2020.

[21]

B. A. Minch, "Translinear circuits," tech. rep., 2009.

[22]

A. G. Andreou and K. A. Boahen, "Translinear circuits in subthreshold mos," Analog Integrated Circuits and Signal Processing, vol. 9, no. 2, pp. 141--166, 1996.

Digital Library

[23]

J. A. Starzyk and X. Fang, "Cmos current mode winner-take-all circuit with both excitatory and inhibitory feedback," ELECTRONICS LETTERS, 1993.

[24]

J. Lazzaro, S. Ryckebusch, M. A. Mahowald, and C. A. Mead, "Winner-take-all networks of o (n) complexity," Advances in neural information processing systems (NIPS), vol. 1, 1988.

[25]

J. Lazzaro, S. Ryckebush, M. Mahowald, and C. A. Mead, "Winner-take-all networks of o(n) complexity," tech. rep., Cal-tech, 1988.

[26]

Nanoscale Integration and Modeling (NIMO) Group, "Predictive technology model." http://ptm.asu.edu/, 2007.

[27]

K. Ni, M. Jerry, J. A. Smith, and S. Datta, "A circuit compatible accurate compact model for ferroelectric-fets," in Symposium on VLSI Technology, 2018.

[28]

Y. Ni, Y. Kim, T. Rosing, and M. Imani, "Algorithm-hardware co-design for efficient brain-inspired hyperdimensional learning on edge," in Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 294--299, IEEE, 2022.

[29]

G. Karunaratne, M. Le Gallo, G. Cherubini, L. Benini, A. Rahimi, and A. Sebastian, "In-memory hyperdimensional computing," Nature Electronics, vol. 3, no. 6, pp. 327--337, 2020.

[30]

A. Kazemi, S. Sahay, A. Saxena, M. M. Sharifi, M. Niemier, and X. S. Hu, "A flash-based multi-bit content-addressable memory with euclidean squared distance," in IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), pp. 1--6, IEEE, 2021.

[31]

P.-Y. Chen, X. Peng, and S. Yu, "Neurosim+: An integrated device-to-algorithm framework for benchmarking synaptic devices and array architectures," in 2017 IEEE International Electron Devices Meeting (IEDM), pp. 6--1, IEEE, 2017.

[32]

Y. Xiang, A. Alahi, and S. Savarese, "Learning to track: Online multi-object tracking by decision making," in Proceedings of the IEEE international conference on computer vision (ICCV), pp. 4705--4713, 2015.

Digital Library

[33]

D. Amodei, S. Ananthanarayanan, R. Anubhai, J. Bai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, Q. Cheng, G. Chen, et al., "Deep speech 2: End-to-end speech recognition in english and mandarin," in International conference on machine learning (ICML), pp. 173--182, PMLR, 2016.

[34]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems (NIPS), vol. 25, 2012.

[35]

K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 770--778, 2016.

[36]

P. Kanerva, "Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors," Cognitive computation, 2009.

[37]

A. Hernandez-Cane, N. Matsumoto, E. Ping, and M. Imani, "Onlinehd: Robust, efficient, and single-pass online learning using hyperdimensional system," in Design, Automation & Test in Europe Conference & Exhibition (DATE), 2021.

[38]

M. Imani, J. Morris, J. Messerly, H. Shu, Y. Deng, and T. Rosing, "Bric: Locality-based encoding for energy-efficient brain-inspired hyperdimensional computing," in Proceedings of the 56th Annual Design Automation Conference (DAC), pp. 1--6, 2019.

Digital Library

[39]

M. Imani, Z. Zou, S. Bosch, S. A. Rao, S. Salamat, V. Kumar, Y. Kim, and T. Rosing, "Revisiting hyperdimensional learning for fpga and low-power architectures," in 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pp. 221--234, IEEE, 2021.

[40]

D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, "Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine," in International workshop on ambient assisted living, Springer, 2012.

[41]

A. Angelova, Y. Abu-Mostafam, and P. Perona, "Pruning training sets for learning of object categories," in Conference on Computer Vision and Pattern Recognition (CVPR), 2005.

[42]

"Uci machine learning repository." http://archive.ics.uci.edu/ml/datasets/ISOLET.

Cited By

Zhou YHuang XYang JNi KAmrouch HZhuo CYin X(2024)Low Power and Temperature- Resilient Compute-In-Memory Based on Subthreshold-FeFET2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546647(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546647
Xu ZLiu CLi CMao RYang JKämpfe TImani MLi CZhuo CYin X(2024)FeReX: A Reconfigurable Design of Multi-Bit Ferroelectric Compute-in-Memory for Nearest Neighbor Search2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546615(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546615
Huang QBarkam HYang ZYang JKämpfe TNi KZhang GLi BSchlichtmann UImani MZhuo CYin X(2024)A FeFET-based Time-Domain Associative Memory for Multi-bit Similarity Computation2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546522(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546522
Show More Cited By

Index Terms

COSIME: FeFET Based Associative Memory for In-Memory Cosine Similarity Search
1. Computer systems organization
  1. Architectures
    1. Other architectures
  2. Dependable and fault-tolerant systems and networks
    1. Processors and memory architectures
2. Hardware
  1. Emerging technologies
    1. Analysis and design of emerging devices and systems
      1. Emerging architectures
    2. Spintronics and magnetic technologies
  2. Integrated circuits
    1. Semiconductor memory
      1. Dynamic memory

Index terms have been assigned to the content through auto-classification.

Recommendations

WOM-Code Solutions for Low Latency and High Endurance in Phase Change Memory
This paper describes a write-once-memory-code phase change memory (WOM-code PCM) architecture for next-generation non-volatile memory applications. Specifically, we address the long latency of the write operation in PCM—attributed to PCM SET—...
A Novel Memory Block Management Scheme for PCM Using WOM-Code
HPCC-CSS-ICESS '15: Proceedings of the 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conf on Embedded Software and Systems

Phase Change Memory (PCM) is a promising DRAM replacement in embedded systems due to its attractive characteristics including low static power consumption and high density. However, long write latency is one of the major drawbacks in current PCM ...
Mellow writes: extending lifetime in resistive memories through selective slow write backs
ISCA'16

Emerging resistive memory technologies, such as PCRAM and ReRAM, have been proposed as promising replacements for DRAM-based main memory, due to their better scalability, low standby power, and non-volatility. However, limited write endurance is a major ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICCAD '22: Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design

October 2022

1467 pages

ISBN:9781450392174

DOI:10.1145/3508352

Conference Chair:
Tulika Mitra
National University of Singapore
,
Program Chairs:
Evangeline Young
The Chinese University of Hong Kong
,
Jinjun Xiong
University at Buffalo (UB)

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDA: ACM Special Interest Group on Design Automation

In-Cooperation

IEEE-EDS: Electronic Devices Society
IEEE CAS
IEEE CEDA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 December 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

ICCAD '22

Sponsor:

SIGDA

ICCAD '22: IEEE/ACM International Conference on Computer-Aided Design

October 30 - November 3, 2022

California, San Diego

Acceptance Rates

Overall Acceptance Rate 457 of 1,762 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
364
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)13

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhou YHuang XYang JNi KAmrouch HZhuo CYin X(2024)Low Power and Temperature- Resilient Compute-In-Memory Based on Subthreshold-FeFET2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546647(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546647
Xu ZLiu CLi CMao RYang JKämpfe TImani MLi CZhuo CYin X(2024)FeReX: A Reconfigurable Design of Multi-Bit Ferroelectric Compute-in-Memory for Nearest Neighbor Search2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546615(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546615
Huang QBarkam HYang ZYang JKämpfe TNi KZhang GLi BSchlichtmann UImani MZhuo CYin X(2024)A FeFET-based Time-Domain Associative Memory for Multi-bit Similarity Computation2024 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE58400.2024.10546522(1-6)Online publication date: 25-Mar-2024
https://doi.org/10.23919/DATE58400.2024.10546522
Rakka MAmer WChen HImani MKurdahi F(2024)HDRLPIM: A Simulator for Hyper-Dimensional Reinforcement Learning Based on Processing In-MemoryACM Journal on Emerging Technologies in Computing Systems10.1145/369587520:4(1-17)Online publication date: 28-Nov-2024
https://dl.acm.org/doi/10.1145/3695875
Fan ZWan ZLiu CLu ABhardwaj KRaychowdhury A(2024)Benchmarking Test-Time DNN Adaptation at Edge with Compute-In-MemoryACM Journal on Autonomous Transportation Systems10.1145/36658981:3(1-26)Online publication date: 23-Jul-2024
https://dl.acm.org/doi/10.1145/3665898
Li CXu ZWen BMao RLi CKampfe TNi KYin XDe V(2024)FeBiM: Efficient and Compact Bayesian Inference Engine Empowered with Ferroelectric In-Memory ComputingProceedings of the 61st ACM/IEEE Design Automation Conference10.1145/3649329.3656538(1-6)Online publication date: 23-Jun-2024
https://dl.acm.org/doi/10.1145/3649329.3656538
Chen XChen HYang C(2024)PointCIM: A Computing-in-Memory Architecture for Accelerating Deep Point Cloud Analytics2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO61859.2024.00097(1309-1322)Online publication date: 2-Nov-2024
https://doi.org/10.1109/MICRO61859.2024.00097
Wang JSun SYin X(2024)Single-Ferroelectric FET based Associative Memory for Data-Intensive Pattern Matching2024 25th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED60706.2024.10528715(1-7)Online publication date: 3-Apr-2024
https://doi.org/10.1109/ISQED60706.2024.10528715
Chen HNi YHuang WImani M(2024)Scalable and Interpretable Brain-Inspired Hyper-Dimensional Computing Intelligence with Hardware-Software Co-Design2024 IEEE Custom Integrated Circuits Conference (CICC)10.1109/CICC60959.2024.10529049(1-8)Online publication date: 21-Apr-2024
https://doi.org/10.1109/CICC60959.2024.10529049
Wang FZhang WLi ZLin NBao RXu XDou CWang ZShang D(2024)In-memory search with learning to hash based on resistive memory for recommendation accelerationnpj Unconventional Computing10.1038/s44335-024-00009-x1:1Online publication date: 1-Oct-2024
https://doi.org/10.1038/s44335-024-00009-x
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten