Abstract

Neuron circuits are the fundamental building blocks in the modern neuromorphic system. Designing compact and low-power neuron circuits can significantly improve the overall area and energy efficiencies of a neuromorphic chip architecture. Here, practical neuron circuits must overcome the variations arising from nonideal behaviors of synaptic devices, such as stuck-at-fault and conductance deviation. In this study, a compact leaky integrate-and-fire neuron circuit has been designed, with resilience to synaptic device state variations, for hardware implementation of spiking neural networks (SNNs). The proposed neuron circuit is simulated on the 0.35-μm Si complementary metal-oxide-semiconductor technology node by a series of circuit simulations based on HSPICE. The proposed circuit occupies a reduced area and exhibits low power consumption (14.7 µW per spike). Furthermore, the optimized circuit design results in a high degree of tolerance toward input-current variations arising from conductance-state variations in the synapse array. Hence, the proposed neuron circuit would be capable of substantially improving the area efficiency and reliability in the realization of the hardware-oriented SNN architectures.

1. Introduction

In recent years, neuromorphic computing has gained popularity and is used in a wide range of brain-inspired models, devices, and computers that mitigate the memory bottleneck of Von Neumann computer architecture. Serial computation processes of the Von Neumann architectures require high-energy usage to accomplish massive operations [1]. However, neuromorphic architectures are remarkable for their high connectivity and parallelism, low power consumption, and memory and processing collocation, which have increasingly captured the attention of the computing world as the end of Moore’s law approaches [2]. These qualities provide strong motivation for building electronic devices in neuromorphic architecture [3]. Neuromorphic computing can be based on spiking neural networks (SNNs), which imitate biological systems in which neurons transmit information in the form of spikes through electrical synapses. The development of hardware-oriented neuromorphic systems encompassing from synthetic synapses and neurons to spike-based computing platforms is under active progress [4]. Taking a further step, SNN is the third-generation ANN that offers area- and energy-efficient system architecture [57]. Neurons are integral components of such an SNN architecture and require highly optimized area and energy considerations for their hardware design. SNN can be adopted to any form of a neural network as long as the system requires high-power efficiency [8, 9]. Among the various applications, low-power accelerated multiplicate-and-accumulate (MAC) computing, low-power image, and speech recognition, and low-power processing units in autonomous vehicles can be sought at the first hand.

System-level implementation of such neuron circuits frequently necessitates the simplification of complicated biological neuron functions, as can be found in the examples of the Hodgkin–Huxley (HH) model, Izhikevich neuron model, and other threshold-based neuron models [1011]. Despite its simplicity, the leaky integrate-and-fire (LIF) neuron has become one of the most widely utilized models, because it emphasizes two fundamental neuronal dynamics: the leaky accumulation of postsynaptic currents and consequent spiking. Therefore, hardware-based neurons are implemented as electrical circuits that mimic the behaviors of LIF neuron models [12, 13]. The LIF neuron circuit is designed to accumulate the currents from the weighted synaptic connections and instantly generate an action potential when the membrane potential surpasses the threshold voltage of the neuron circuit. The input current to the neuron circuit emerges from a synaptic array composed of memory cells, with data nonvolatility in many cases, as can be found in resistive-switching random-access memory (RRAM), flash memory, phase-change memory (PCM), and magnetic random-access memory (MRAM) [14, 15]. The merging of nonvolatile memories (NVMs) is particularly susceptible to reliability issues at the cell level, such as stuck-at-state faults, conductance drift, and cycle-to-cycle variations [16]. RRAM-based neuromorphic chip design has received marked research interest, mainly because of its low write voltage, area efficiency, and complementary metal-oxide-semiconductor (CMOS) viability [1719]. However, mitigating the nonidealistic features can be a more critical issue with RRAM-based synaptic arrays, since the set and reset currents of RRAM have relatively low concentration in distribution, which might lead to variations in the bitline currents by which the accelerated MAC operations are performed. This poses a serious challenge to neuron circuits that receive these bitline currents as their input signal. It is critical for the neuron circuit to withstand such synapse array-induced variations in their input signals. To the best of our knowledge, the resilience of a neuron circuit against the fluctuation in input current has seldom been investigated in previous studies.

Hence, in this work, we propose a novel LIF neuron circuit that can tolerate the input current fluctuation originating from either nonidealities or dynamic behaviors of the synapse devices [20]. The present neuron circuit was motivated by an explicit design principle for implementing area-efficiency and low-power circuits [21]. Furthermore, the resilience of the LIF neuron circuit against the input current fluctuations was quantified for various conductance-state-based faults of the synaptic device. All the circuit simulations were performed using the HSPICE circuit simulator (run on a Linux environment; ver. released in 2019. 12) employing the 0.35-μm Si-CMOS technology node library.

2. Analogy between Biological Neuron and Electronic Neuron Circuit

2.1. Neurons and Synapses

The human brain is the most complex organ in the body, composed of approximately 86 billion neurons. Neural networks are associated with highly organized connections between neurons and their interactions, resulting in effective information transfer. A conventional neural structure consists of three parts: dendrites, somas, and axons, as shown in Figure 1. Most neurons are polarizing cells, and polarization belongs to the spatial diversity in form, structure, and function inside a cell. All cell types have a degree of polarity that allows them to execute certain activities. Dendrites in nerve cells are organized in a dendrite pattern and transfer the input signals to the soma. The soma modifies its membrane potential in response to information received from dendrites, and an action potential is triggered when the membrane potential crosses a specific threshold. The spike signal is transmitted to the axon terminal, whereas the action potential is output via the axon. Information is exchanged through the synapses that connect neurons. Presynaptic and postsynaptic neurons are located before and after the synapse, respectively. Presynaptic neurons are linked to postsynaptic neurons via the space between them, which is known as the synaptic gap [22, 23]. The action potential travels through the axon to the presynaptic terminal, where it triggers the production of vesicular-packaged chemicals known as neurotransmitters. These vesicles are fuzed to the membrane by an action potential and are subsequently released back into the synaptic gap, where they attach to receptors on the cell surface of the postsynaptic terminal, which in turn affects the synaptic transmission of numerous neurotransmitters [24].

2.2. Neuron Circuit Analogy

In the early 1950s, Hodgkin and Huxley designed the first neuron model (H–H model), which showed the relation between the ionic currents through the neuron cell membrane and the membrane voltage. In the 1990s, a CMOS neuron model, popularly known as the LIF neuron circuit, was demonstrated. LIF is modeled on the H–H neuron and adapts biological neuron functions into the equivalent electrical circuitry [2530].

LIF is a simplistic form of the neuron model, which is universally treated as the fundamental part of SNNs due to its comparatively low computational difficulty. As shown in Figure 1, postsynaptic input currents (Is) acquired via dendrites are summed (Im = ) to charge the membrane capacitor, as the current onto the membrane (Im), resulting in a membrane potential (Vmem). When a soma crosses a certain threshold, the neuron generates spikes and completes its LIF function. The generated spike passes through the axon, and the event is transmitted to other associated neurons. The weighted connection between the LIF neuron circuits forms a synapse array, as schematically shown in Figure 1. In the electrical sense, these weights in connections can be described as conductance values that can be modulated based on the occurrence of pre- and postsynaptic spikes.

Operations of neurons and synapses require power consumption in the SNN system. Biological brain-inspired network features have an enormous number of presynapses (1,000–10,000) per neuron, indicating that synaptic power dominates the total power consumption in the neural network. Previous researches have shown that, even in the worst-case scenarios, i.e., the neuron consumes 15%–50% of the total power consumption in various digital and analog SNNs specifically adapted for inference tasks [3136]. If the ratio in number (10,000 : 1) is considered, it can be judged that a neuron circuit consumes a huge power per unit, which calls for compact and highly power-efficient neuron circuits.

3. Leaky Integrate-and-Fire Neuron Circuit

3.1. Circuit Description

The LIF neuron circuit presented in this work was designed and simulated with 0.35-μm Si-CMOS process technology through a series of HSPICE simulations, as shown in Figure 2. The neuron circuit is composed of two parts: the integration-and-reset part and the trigger-and-fire part. The former part consists of a membrane capacitor of 0.01 pF and an n-type MOSFET (M6) with W/L = 0.70/0.35 μm taking charge of the reset function. The latter part includes three n-type MOSFETs, two p-type MOSFETs, and two inverters in which all the MOSFETs have a dimension of W/L = 0.70/0.35 μm and an operation voltage VDD = 1.0 V. Over the series of simulations, the input current was assumed to be 10 pA. In particular, Cmem was responsible for integrating the current signal from the synaptic array. The value of Cmem can be varied, depending on the quantity of current delivered from the synapse array depending on the targeted firing frequency. Also, Cmem determined the W and L of the p-type MOSFET (M4) to retune the neuron. The integration function was conducted by Cmem because current signals from the synapse array were frequently delivered to the neuron. The membrane potential (Vmem) increased to the threshold voltage (Vth) of the n-type MOSFET (M1) as charges accumulated in the Cmem. Whenthe Vmem exceeded the Vth of the M1, M1 was switched on, causing node 1 voltage to decrease. The output node of inverter 1 (INV 1) was then switched from low to high voltage, allowing the n-type MOSFET (M2) to initialize Vmem and the high voltage at node 1 was dragged down to a low voltage. Subsequently, to enable M2 to initialize Vmem, INV1 is changed from a low state to a high state. After the n-type MOSFET (M3) is turned on, the output node voltage of inverter 2 (INV2) returns to its initial value. MOSFETs M2, M3, and M4 exist to stabilize the corresponding node states of the neuron circuit. For the simulation tasks, the Berkeley short-channel insulated-gate field-effect transistor model (BSIM) version 3.3 (level 49) designed for 0.35-μm Si CMOS technology was used [37].

3.2. Circuit Operation

The most essential function of a neuron circuit is to accumulate the input current from the synapse array until it reaches the threshold voltage of the circuit. Once the neuron circuit exceeds the threshold voltage, a digital spike takes place, and the circuit returns to its initial (fully discharged) state. The neuron circuit designed in this work demonstrates the fundamental behavior of a neuron circuit, as can be confirmed in Figure 3(a). The figure shows a transient analysis of the designed LIF neuron circuit validating the integration function realized by the membrane capacitor, Cmem, concerning the input current pulse. The input signal utilized in the simulations was in the form of a square pulse. The choice of this input signal shape makes the transient analysis smooth, which is indispensable in validating the functionality and the performance of the neuron circuit, in consideration of the expandability of neuron circuits cooperating with the digital integrated circuit that feeds the input to the neuron usually in the form of a square pulse. When Vmem exceeds the threshold voltage of M1, a spike is generated at the output node (Vspk), and then it turns on M6, which is responsible for the complete discharging from Cmem. Finally, the circuit is reset to the initial state. Cyclic operations are repeated as the input current signals are continuously brought into. From Figure 3(b), the firing rate, i.e., the number of spikes for a given time versus the input current, shows that the firing rate of the neuron circuit proportionally increases with the input current. Also, the firing rate is inversely proportional to the Cmem, as shown in Figure 3(c). Thus, the circuit can control the firing rate by modulating the input current and Cmem, as per the designed synaptic device.

4. Simulation Results and Discussion

As discussed earlier, the input currents to the neuron circuit are fed from the synaptic array, which is mainly composed of emerging NVMs such as RRAM, PCM, and MRAM. These devices undergo state changes representing the individual conductive states. In general, they either remain in a low-resistance state (LRS) or a high-resistance state (HRS), depending on the conduction mechanism. However, these devices tend to deviate from their intended state in a synapse array, leading to a variety of nonideal behaviors, such as failures in set and reset operations and incomplete state switching. Hence, we can consider some representative specific fault cases that may occur in the synapse array for designing a neuron circuit.

First, we consider a scenario in which a fraction of the synapse devices are stuck in the LRS. Therefore, the current from the synapse array fluctuates within in value (∼10 pA) with a lower bound, which is determined as a fraction (between 0% and 10%) of the original current. For the random fluctuation in input signals, the input data were generated using Python, by which random fluctuations from 0% to 10% of the amplitude of the reference input pulse were introduced. This variability was conceived to bring more realistic scenarios where the input signals might exhibit inherent noises or process-induced variations. In applying these random fluctuations, a piecewise linear signal type was employed. This type of signal representation allows to model the time-varying characteristics of the input signal in a more time-efficient way. This approach not only ensures transparency in simulation but also reflects the realistic variability frequently encountered in integrated circuits. Figure 4(a) shows the input current, membrane voltage, and spiking behavior of the LIF neuron for the stuck-at-LRS synapse array condition. The spike position and amplitude did not change considerably as the stuck-at-LRS fraction increased to 10%. Similarly, for the HRS (Figure 4(b)), the resistance increased because the fluctuation of the input current pulse amplitude decreased. Finally, we consider a scenario in which there is a fraction of both stuck-at-LRS and stuck-at-HRS devices in the synapse array, as shown in Figure 4(c). Spiking behavior is clearly not affected by such a combination of fault states. The above results are confirmed by the fluctuation analysis results in Figure 4(d)4(f), which correspond to Figure 4(a)4(c) in sequence, respectively. In Figure 4(f), although the spike shifts do not occur in any specific direction, alternations through left and right shifts are found, depending on whether the input current variation is based on LRS or HRS fault at a specific time. Figure 4(d) reveals that there are only nominal leftward shifts in the spikes, as indicated by the marked arrows. This takes place since the required threshold voltage is exceeded in a short time due to the additional amount of current provided by the stuck-at-LRS synaptic devices. The output spikes are shifted in the rightward direction as the fluctuation percentage increases, as shown in Figure 4(e) in conjunction with Figure 4(c). This result is brought since a longer time is required for the membrane potential to exceed the threshold voltage of the neuron circuit owing to a drop in the input current. The circuit can be optimally designed with the capability of stronger tolerance against the fluctuation in input current with minimizing the loss in functionalities of the neuron circuit. Equation (1) defines the index for a shift in spike time: the amount of shift taking place at the output node with respect to fluctuation in the input signal in the unit of percentage.

where is defined as the spike time shift index that represents the time difference between spike times at which fluctuations of 0% (reference point) and x% take place while the input signals are fed into the neuron circuit. Here, and indicate the spike times. Figure 5(a)5(c) shows the spike time-shift index of the fluctuation for different states of the synaptic device. The overall shift index level for LRS was 14% (Figure 5(a)); for HRS, it was 10% (Figure 5(b)), and for the combined state, the maximum shift index was 2% (Figure 5(c)), respectively. In consequence, it can be addressed that the designed LIF neuron circuit can tolerate a 10% fluctuation with a minor spike time shift index, considering different stuck-at-fault states. Moreover, conventional neuron circuits usually contain a couple of capacitors. In the study of Kang et al. [21], a typical neuron circuit uses two capacitors: one for accumulating the input current and the other for controlling the width of the output spike. In an integrated circuit, a capacitor occupies a large area even if the capacitance is only a few picofarads. If the gate oxide thickness of 9 nm for a 0.35-μm technology node [37] and 1 pF of capacitance are assumed, the area can be simply predicted as 261 μm2, which corresponds to an area of a square with a 16.2-μm edge. Thus, it can be figured out that the capacitor seriously threatens the area efficiency of a neuron circuit with the tremendous imbalance with the MOSFET scalability. In this work, we have designed a neuron circuit that has only one capacitor, the membrane capacitor, truncating the secondary capacitor, confirming that the neuron circuit functionality is not lost at all. Reduction of one capacitor significantly reduces the area of a neuron circuit, and thus, higher compactness of the neuron circuit in this work is substantially expected. Compared with the conventional neuron circuit having two capacitors and a CMOS circuit part, at least 30% of area reduction is practically realized. In addition, the performances of the neuron circuit designed in this work are compared with the circuits in the previous reports, as shown in Table 1. For the designed neuron circuits, the power and energy consumption can be calculated based on the following equations, which are numerically calculated by the circuit simulation package.

where T is the length of a single period for an integrate-and-fire event. For a unit integrate-and-fire operation, the designed LIF neuron circuit consumes an energy of 2.2 nJ per event, which corresponds to a power consumption of 14.7 μW. Also, the unique feature of the designed neuron circuit is its resilience against the fluctuations in the synaptic devices, and the power/energy efficiency can be plausibly accompanied for the hardware SNNs realization.

5. Conclusion

The LIF neuron circuit presented in this paper was developed and simulated with 0.35-μm Si-CMOS process technology using the HSPICE simulator. The designed circuit fulfills the fundamental characteristics of the LIF neuron circuit. The focus of this study was to design a simple and compact neuron circuit that can deal with the fluctuation of a synapse array. We showed that the LIF circuit can tolerate up to 10% fluctuation, with minor output spike time shifts for different states of the synaptic device. Hence, the results of this study describe how the LIF neuron circuit might behave properly, depending on the synaptic device states in the SNN hardware, and how it can be specifically designed to maximize parallelism in data-intensive decision-making. Furthermore, featuring the low-power operation capability, the application of SNNs can be expanded into other areas, such as autonomous vehicles, drug discovery, and brain–computer interfaces.

Data Availability

Data for this research article are available upon request.

Disclosure

All the simulation results and figures were produced by the authors without necessitating approvals or permissions for the dataset.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean Ministry of Science and ICT (MSIT) (2021M3F3A2A01037927) and was also funded by the Gachon University research fund of 2021 (GCU-202106410001). The EDA tool for circuit simulation was supported by the IC Design Education Centre (IDEC).