DenRAM: Revolutionary Synaptic Architecture for Temporal Signal Processing

By Dr Silpaja Chandrasekar, PhDMay 3 2024Reviewed by Susha Cheriyedath, M.Sc.

In a paper published in the journal Nature Communications, researchers highlighted the importance of spatial dendritic branching in pyramidal neurons for non-linear computation and temporal signal processing. They introduced "DenRAM," a feed-forward spiking neural network with dendritic compartments implemented using analog electronic circuits and resistive random access memory (RRAM) technology.

DenRAM effectively replicated synaptic delay profiles and performed spatiotemporal pattern recognition. System-level simulations demonstrated DenRAM's resilience to analog hardware noise and superior accuracy compared to recurrent architectures. This advancement promises enhanced temporal processing capabilities, reduced memory footprint, and high accuracy in low-power signal processing technologies.

Related Work

Previous research underscores the importance of dendritic structures in neural computation for spatio-temporal feature detection. While most artificial neural networks (ANNs) overlook dendritic complexities, recent studies highlight their significance for accurate temporal pattern processing. Coincidence detection (CD) within dendrites enables efficient multi-time scale processing, which is crucial for tasks like sensory processing.

Delays have emerged as key temporal variables enhancing the efficiency of spiking neural networks (SNNs). Implementing delays in neuromorphic hardware remains challenging, but emerging memory technologies like Resistive RRAM offer promising solutions. Past work has explored leveraging RRAM devices for efficient delay implementation, paving the way for real-time sensory processing on neuromorphic hardware.

DenRAM Circuit Development 

Designing and fabricating DenRAM circuits involved creating dendritic circuits with two main sections: generating delay and weighting output currents. Input spikes, represented as voltage pulses, triggered the opening of an n-channel metal-oxide-semiconductor (nMOS) transistor, causing a voltage drop across a capacitor connected to a 1T1R Delay device. This delay mechanism relied on the time constant τ=RC for recharging the capacitor.

A thresholding unit is detected when the voltage across the capacitor crosses a set threshold, generating an output spike with a delay compared to the input spike. The delayed output spike was then applied to the word line of the weight 1T1R, generating an output current proportional to the conductance of the weight RRAM.

Fabrication and integration of DenRAM circuits were carried out using a 130 nm technology, with RRAM devices fabricated using a titanium nitride/silicon doped hafnium oxide/titanium/titanium (TiN/Si: HfO/Ti/TiN) stack. Each RRAM device was coupled with an access transistor to form 1T1R structures, allowing individual selection during programming. Peripheral circuits were designed to interface with each array row and column, allowing the programming and reading of RRAM devices.

Characteristics of RRAM devices rely on resistive switching mechanisms to create and dissolve conductive pathways, resulting in distinct resistive states within the device. SET and RESET operations were executed by applying positive or negative pulses across the device. The team performed programming and reading of RRAM devices using source measure units and a waveform generator, ensuring stability and precision in the operation.

The dendritic circuit's input and output signals were analyzed using pulse generators and oscilloscopes, with statistics recorded on read-to-read delay variability. The analysts employed RRAM-aware noise-resilient training to address the stochasticity of RRAM devices during inference, ensuring robustness to variations in device behavior. The DenRAM architecture was evaluated on heartbeat anomaly detection and keyword spotting using datasets like the Massachusetts Institute of Technology - Beth Israel Hospital (MIT)-BIH and Spiking Heidelberg Digit (SHD), demonstrating its effectiveness in real-world applications.

DenRAM: Efficient Synaptic Architecture

The DenRAM architecture forms the backbone of the system. Comprising delaying and weighting units facilitated by RRAM devices, this circuit integrates delay and weight functionalities through meticulous control of two RRAM devices per synapse. Each RRAM device is intricately connected to bit line (BL), word line (WL), and source line (SL), with their modes toggled between reading and programming via multiplexer (MUX) circuits. These multiplexers regulate the connections between various paths during programming or reading operations, ensuring precise control over the delay and weight components of the synaptic circuit.

Fabricated in a 130 nm complementary metal-oxide-semiconductor (CMOS) process and integrated with RRAM devices, the dendritic circuit undergoes extensive testing to validate its functionality across different states. After starting in a pristine state without conductive filaments, RRAM devices undergo a forming operation to transition to a low-resistive state (LRS), facilitating programmability between LRS and a high-resistive state (HRS).

Additionally, conventional transistor gate oxide capacitance governs the capacitor's behavior within the circuit, optimizing its performance. RRAM devices are meticulously controlled during the read mode, ensuring precise voltage regulation and signaling propagation, which is vital for accurate temporal processing.

By harnessing the biologically inspired temporal delay mechanism, DenRAM demonstrates remarkable efficiency in various tasks, including heartbeat anomaly detection and keyword spotting. Leveraging hardware-aware training procedures and noise-resilient techniques, DenRAM surpasses conventional Spiking Recurrent Neural Networks (SRNN) in accuracy, memory footprint, and power consumption. Through meticulous calibration of RRAM weights and controlled delay distributions, DenRAM emerges as a pioneering architecture, offering unparalleled efficiency in temporal feature detection and processing, critical for real-time applications in diverse domains.

Revolutionary Integration

In conclusion, DenRAM emerged as a groundbreaking synaptic architecture that seamlessly integrated RRAM devices to enable precise delay and weight functionalities. Its meticulous control over RRAM devices ensured optimal performance across different states, from pristine to low-resistive and high-resistive states.

DenRAM showcased remarkable efficiency in tasks like heartbeat anomaly detection and keyword spotting, outperforming conventional SRNNs in accuracy, memory footprint, and power consumption. DenRAM set a new temporal feature detection and processing standard, revolutionizing real-time applications across diverse domains with its hardware-aware training procedures and noise-resilient techniques,