Elsevier

Neurocomputing

Volume 465, 20 November 2021, Pages 157-166
Neurocomputing

Using volatile/non-volatile memristor for emulating the short-and long-term adaptation behavior of the biological neurons

https://doi.org/10.1016/j.neucom.2021.08.132Get rights and content

Abstract

Adaptive response to the timely constant stimulus is the common feature of real neurons. The circuit of the adaptive neuron model consumes less power and requires less data transmission bandwidth compared to the circuit of the non-adaptive neuron model, especially for encoding time-varying signals. Memristor is a good candidate for mimicking the behavior of neurons so that the simple memristor-based circuit can directly emulate many specific behaviors of the neurons with low power and low area consumption. In this work, for the first time, we show that as the nonvolatile switching property of the memristor can be useful for representing long-term adaptation behavior in the memristor-based neuron, the short-term adaptation behavior can also be emulated directly using the same memristor-based circuit due to the volatile switching property of the memristor. Here, short term adaptation is realized using the volatile property of memristor, unlike neuron circuits where adaptation is realized using leakage modulation. As a result, in the memristor-based neuron extra power dissipation can be reduced. Two different types of memristors are used for implementing the proposed circuit of adaptive leaky integrate-and-fire neuron, the volatile/non-volatile memristor and threshold switching memristor are in the charge and discharge path of the capacitor, respectively. Results show that the volatile or non-volatile resistance change of charging memristor upon different input patterns to the neuron circuit determines the type of adaptive behavior of the neuron response, i.e. the neuron may show short-term adaptation or long-term adaptation or does not show an adaptation behavior at all. Comparison with similar works shows that the energy consumption per spiking of the proposed neuron is relatively low, while the circuit is very area-efficient.

Introduction

Emulating some of the features in the living organism by electronic circuits is very beneficial and can help us solve complicated problems. The key concept is to follow the methods employed by biological systems interacting with their environment in various technological areas. Reproducing the nonlinear behavior of the living organisms needs some circuit elements with specific dynamical behaviors and data storage property.

Neuromorphic computing mimics the neuro-biological behaviors present in the human nervous system. Neuromorphic computing based on the spiking neural network has been widely considered due to its low power consumption, high execution speed, and high similarity to its biological counterpart. Artificial synapses and neurons are two basic elements in the implementation of artificial neural networks.

Information in neural networks is represented by the temporal and spatial pattern of the spikes [1], [2]. The strength of the inputs stimuli determines the firing rate of neurons, in which the firing rate of the neuron is increased in response to the stronger stimulation. The generated spike train is affected by the recent history of the electrical activity of neurons, known as spike-frequency adaptation [3]. According to this mechanism, during a sustained stimulus the firing rate of the neuron is reduced. It is believed that the different time constants and thresholds of different ion channels embedded in the neuron membrane are the main reasons for spike-frequency adaptation. In the adaptive type of the spike-based neuron model, the information is encoded in the relative timing of the output spikes rather than the rate of the spikes. If adaptive behavior implemented in electronics, it may find useful applications where adaptation to the incoming signal is of interest, for example, pattern recognition [4], [5], [6]. Also, these biologically resemble electronic tools can be used to predict biological response before it is tested on the actual organism in experiments.

First, the existence of memristor is postulated by professor Chua [7]. Memristor functionally acts like a synapse, interesting features of this device such as, flexibility, extreme scalability, and ability to remember the last resistance state make it a good candidate for the implementation of biological synapses. As a result, many papers on the implementation of synapses using memristors have been published in recent years [8], [9], [10], [11], [12]. Memristors can also be used to implement neurons. If both of the neurons and synapses are made of memristors, neuromorphic architecture with high density and high connectivity like the human brain can be provided. Most of the neurons in the literature are implemented using CMOS. Chua has shown that sodium and potassium ion-channel memristors in the Hodgkin–Huxley equations are the key to generating the action potential, and also the key to solving several unresolved anomalies associated with these equations [13]. The implementation of neurons using memristors is reported in several studies [14], [15], [16], [17], [18], [19], [20], [21], [22], [23].

A neuristor built with two nanoscale Mott memristors with extremely high-density integration capability is presented by Pickett et al. [14]. The ability to provide completely transistor-free neuromorphic architecture with high integration density and the ability to exhibit the important neural functions of all-or-nothing spiking with signal gain and refractory period are the main features of this neuristor. Stoliar et al. introduced a single-component device based on a Mott memristor, which can implement the basic functions of LIF spiking neurons; Leaky, Integrate, and Fire [15]. This device implements a firing spike by delivering an outgoing current pulse. Al-Shedivat et al. proposed a stochastically spiking neuron based on memristor, which spikes stochastically similar to real biological neurons [16]. By considering the probabilistic nature of switching of metal–oxide and amorphous silicon memristors in the sub-threshold regime, the stochastic behavior in neuron response is observed. Feali et al. showed that in terms of threshold switching effect and also stochastic behavior, memristors can be considered as an electronic analogous of the Hodgkin–Huxley ion channels [17]. Stochastic switching of memristor caused memristor-based neuristor to be more reliable in response to the fluctuating stimulus, similar to biological neurons. Ignatov et al. introduced a spiking neuron model in a circuit comprising memcapacitive and memristive devices based on vanadium dioxide material. The circuit shows a refractory period and emulates basic neuronal functionalities, including spike coding in real time and firing frequency adaptation [18]. They show that the use of a memcapacitor in the neuron circuit allows mimicking the firing frequency adaptation in real neurons, in which the oscillation frequency is changed depending on the number of spikes generated before. Integrate-and-fire artificial neuron based on a Ag/SiO2/Au threshold switching memristor is demonstrated by Zhang et al. [19]. This neuron could exhibit four critical features of biological neurons: threshold-driven spiking, all-or-nothing spiking, refractory period, and firing-rate coding. Also, the digit recognition system composed of these neurons was successfully simulated. Kalita et al. used the vertical-MoS2/graphene based threshold switching memristor to produce the integrate-and-fire response of a neuron [20]. Most essential properties of a neuron, such as the threshold driven spiking, all or nothing spiking, refractory period, and strength modulated frequency can be emulated using this device. Lu et al. demonstrated experimentally the functions of LIF neurons with low power feature based on a RC circuit and a single Pt/Ag/TiN/HfAlOx/Pt memristive device [21]. They show that the power consumption of the proposed neurons is low due to the small Vth and ultrahigh ROFF characteristics of the volatile memristor.

A novel spiking neuron circuit based on memristors is designed by Zhao et al. [22]. This neuron is capable of providing various release thresholds and amplitudes of the spikes by adjusting its circuit parameters. Using this memristor-based neuron, an STDP learning circuit is set up to implement the adaptive-learning of synaptic weights. The neuristor with adaptive behavior is implemented in SPICE environment by Feali et al. [23]. Using two different methods (memcapacitor instead of the capacitor or memristor instead of coupling resistor in the primary neuristor circuit), an adaptive response is induced to the neuristor response. The neuristor, according to the results, shows the spike frequency adaptation behavior in response to the continuous external stimulus.

Although non-volatile switching property is a common feature of the memristive devices, experimental studies show that practical memristors may acquire a temporary resistance state under weak stimulus [24], [25].

The majority of researches on the implementation of synapse and neuron with memristor are based on the non-volatile type of memristor. Forgetting in the brain is an important key for adaptive computing so that without forgetting, the details of every piece of information ever experienced soon overwhelms the biological memory. Therefore, volatile memory has been used as a synapse in some studies in brain-like computing [10], [26], [27], [28]. Chang et al. experimentally demonstrated the striking resemblance between retention loss in a nanoscale memristor device and memory loss in biological systems [25]. Based on the results of this study, in addition to the total number of stimuli and their shape, the time interval between stimulus pulses is effective in the memory transition.

Long-term adaptation and short-term adaptation in real neurons play a major role in encoding time-varying signals and forgetting process. It is believed that adaptation is fundamental in the synchronization of neural assembles, selective attention, and forward masking [29], [30], [31]. Also, adaptation decreases the sensitivity of the neuron to the noise and mismatch. Functionally, for encoding sensory signals, adaptation may enhance the limited response range of neurons to much larger dynamic ranges by changing the range of stimulus amplitudes [32].

There has been some work on the circuit implementation of the neuron models with adaptive behavior [33], [34], [35], [36]. In these researches, the spike frequency adaptation is implemented in the neuron circuit using the module consisting of CMOS transistors. However, these CMOS-based designs consume large power and high circuit area.

All of the studies about the implementation of the memristor-based adaptive neuron have used the non-volatile memristor to induce adaptive behavior to the response of the implemented neuron. By removing the voltage or current stimulation, the resistance state of the non-volatile memristor does not change over time. So, the volatile properties of the real neurons such as short-term adaptation could not be implemented using this type of memristor.

In this work, two different types of memristors are used for implementing the proposed circuit of adaptive leaky integrate-and-fire neuron. Volatile/non-volatile memristor and the threshold switching memristor. Given the characteristics of the input stimulus, volatile/non-volatile memristor can show both types of resistive switching, volatile switching, and non-volatile switching. Here, we show that using memristors in the neuron circuit makes it possible to achieve a simple low power and low area memristor-based neuron circuit so that this circuit can easily represent the short and long-term adaptation behaviors of the real neurons. While, for example, mimicking such behaviors of the neurons using CMOS-based circuits is relatively costly.

In this paper, for the first time, we show that as the nonvolatile switching property of the memristor can be useful for representing long-term adaptation behavior in the memristor-based neuron, the short-term adaptation behavior can also be emulated directly using the same memristor-based circuit due to the volatile switching property of the memristor. So, both short-term adaptation and long-term adaptation behaviors can be easily represented using a simple memristor-based circuit. Here, short term adaptation is realized using the volatile property of memristor, unlike neuron circuits where adaptation is realized using leakage modulation. Leakage modulation increases power consumption. As a result, in the memristor-based neuron circuit power dissipation can be reduced.

Section snippets

Proposed memristor-based adaptive neuron

In the brain, the membrane potential of the target neuron increases when it receives the spikes from other neurons through connected synapses. Consequently, when the threshold value is reached, the target neuron will spike. Among the various models of spiking neurons, the computationally efficient integrate-and-fire model describes the neuron functions with a simple structure [37]. In this way, two basic functions of neurons are implemented, integration of inputs and threshold response. The

Results and discussion

To evaluate the response of the proposed neuron, at first, we should choose an appropriate SPICE model of memristors used in the circuit of the neuron. For threshold switching memristor (M2), the SPICE model presented in [38] is used, a dynamical model of Mott memristor based on the Joule heating rate of a thermally driven insulator-to-metal phase transition. The current–voltage relationship and state evolution function of this memristor model are described as follow [38]:v=f(u,i)=Rch(u)idudt=g(u,

Conclusion

We have proposed the neuron circuit with adaptive behavior based on two types of memristors, volatile/non-volatile memristor and threshold switching memristor. For the first time, we used a volatile/non-volatile memristor in the charging path of the capacitor in the neuron circuit. The resistance of this memristor represents the sensitivity of the membrane in the real neurons so that its value affects the time constant of charging the capacitor. Results show that as the nonvolatile switching

CRediT authorship contribution statement

Mohammad Saeed Feali: Conceptualization, Methodology, Software, Data curation, Writing - original draft, Visualization, Investigation, Supervision, Software, Validation, Writing - review & editing.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Kermanshah branch, Islamic Azad University, Kermanshah, IRAN.

Acknowledgement

The author is grateful to the Islamic Azad University, Kermanshah Branch for the support of this research.

Mohammad Saeed Feali received the B.Sc. degree in electrical engineering from Azad University, Kermanshah, Iran, in 2010, and the M.Sc. degree in electronics engineering from Azad University, Tehran South, Tehran, Iran, in 2012 (both with Honors) and the Ph.D. degree in electronics from the University of Razi, IRAN, in 2018. He was with Islamic Azad University, Kermanshah branch, Kermanshah, Iran, as a Faculty Member. He is currently an Assistant Professor in the Electrical Engineering

References (46)

  • E.D. Adrian

    The impulses produced by sensory nerve endings

    J. Physiol.

    (1926)
  • B. Ahmed et al.

    Estimates of the net excitatory currents evoked by visual stimulation of identified neurons in cat visual cortex

    Cereb. Cortex

    (1998)
  • M. Di Ventra et al.
  • C. Yan et al.

    3D room layout estimation from a single RGB image

    IEEE Trans. Multimedia

    (2020)
  • C. Yan et al.

    Deep multi-view enhancement hashing for image retrieval

    IEEE Trans. Pattern Anal. Machine

    (2021)
  • L. Chua

    Memristor-the missing circuit element

    IEEE Trans. Circuit Theory

    (1971)
  • S.H. Jo et al.

    Nanoscale memristor device as synapse in neuromorphic systems

    Nano Lett.

    (2010)
  • J. Shamsi et al.

    Modular neuron comprises of memristor-based synapse

    Neural Comput. Appl.

    (2017)
  • Z. Wang et al.

    Memristors with diffusive dynamics as synaptic emulators for neuromorphic computing

    Nat. Mater.

    (2017)
  • S. Kim et al.

    Analog synaptic behavior of a silicon nitride memristor

    ACS Applied Mater Interfaces

    (2017)
  • L. Chua

    Memristor, Hodgkin-Huxley, and edge of chaos

    Nanotechnology

    (2013)
  • M.D. Pickett et al.

    A scalable neuristor built with Mott memristors

    Nat. Mater.

    (2013)
  • P. Stoliar et al.

    A leaky-integrate-and-fire neuron analog realized with a mott insulator

    Adv. Funct. Mater.

    (2017)
  • Cited by (15)

    • A 300 MHz MOS-only memristor emulator

      2023, AEU - International Journal of Electronics and Communications
    • A GHz-level memristor emulator with only MOSFET

      2024, International Journal of Circuit Theory and Applications
    View all citing articles on Scopus

    Mohammad Saeed Feali received the B.Sc. degree in electrical engineering from Azad University, Kermanshah, Iran, in 2010, and the M.Sc. degree in electronics engineering from Azad University, Tehran South, Tehran, Iran, in 2012 (both with Honors) and the Ph.D. degree in electronics from the University of Razi, IRAN, in 2018. He was with Islamic Azad University, Kermanshah branch, Kermanshah, Iran, as a Faculty Member. He is currently an Assistant Professor in the Electrical Engineering Department, Islamic Azad University, Kermanshah branch. His research interests include Memristors, Neuromorphic, Microfluidics and Digital system design.

    View full text