In his latest post on The Memory Guy, contributor Ron Neale reviews a novel use for ReRAM cells in which a neural processing system mimics the direction-finding mechanism of a barn owl’s ears. This is based on research performed by CEA-Leti in France, which was recently published in the journal Nature.
The potential for the use of the unique characteristics of ReRAMs, PCM and CeRAMs as brain-gates, neuromorphic devices, and in-memory computation has long been recognised.
In a paper recently published in Nature , inspired by the auditory system of the barn owl, a team from: CEA-Leti, Grenoble, and Universities in France, Italy, and Switzerland have used HfO-based ReRAMs to build a position location system.
The barn owl is a passive listener, using the noise from its prey to locate its position. This latest work uses micro-machined piezoelectric transducers (pMUTs) as the two ears to detect the signal reflected from a target transmitted by a third similar piezo transducer to create an ultrasound radar-like system. The essential features of the system are illustrated in Figure 1, with the outgoing pulse in yellow and the return pulses for each ear in red and blue.
The ReRAMs serve two roles in the position locating system: the variable conductance is used to determine the weight of signals applied to a synapse-neuron combination, and as feedback interconnections.
The received wisdom is that, in the neuroanatomy of the owl, axons and neuron nerve fibres and positioning are used to delay the signal received by one ear to obtain coincidence with the signal received by the second ear.
So the first task for those wishing to emulate the owl is to find a low cost means of generating the required delay. Conceptually it’s easy: apply the first received time-of-flight pulse to a simple RC circuit. Then add the second received time-of-flight pulse to the decaying voltage of that RC circuit, with the delay time marker firing whenever the sum of the two exceeds some threshold value.
While such a circuit will provide a delay, it is unable to give any information with respect to precise position. What is needed is a means of precisely and uniquely distinguishing pairs of signals coming from the left and right side of the ears, described as the interaural time difference (ITD).
The first building block for such a system is illustrated in the figure below. It is the weight generating circuit which can provide an output pulse delayed in time with respect to the arrival of an input pulse. The conductance of the ReRAM is used to control the delay time. If the conductance of the ReRAMs is set to a low enough value the circuit can be used to effectively block incoming signals.
Note In this post the units of conductance are Siemens, which some readers may be more familiar with as a Mhos, the inverse of Ohms. More specifically, measurements stated below are expressed in micro-Siemens (µS).
The circuit in this experiment monolithically integrates a thin film TiN-HfO-Ti-TiN ReRAM memory structure with a 130nm CMOS process. I asked team member Elisa Vianello of CEA why the team chose HfO-based ReRAMs, when other emerging ReRAMs are claimed to be able to offer similar performance? Her observation:
“HfO based ReRAM is cost-effective and easy to integrate into any CMOS fab.”
To the right in the figure, are two components implemented in conventional CMOS electronics. They are the synapse, implemented as a differential pair integrator (DPI), and the leaky integrate and fire (LIF) neuron. In this work these circuits, common in neuromorphic work and to the neuro community, were formed from conventional CMOS, with which the ReRAMs were monolithically integrated to form a modified front end.
In simple terms the differential pair integrator (DPI), is in effect a suitably transistor-buffered capacitor, which creates a CR differentiating circuit. It acts to differentiate any input pulse to produce a decaying output pulse, the peak value of which is proportional the value (weight) of the input pulse. Two examples of the way in which a time delay can be generated are shown in the figure here below.
The leaky integrate and fire (LIF) neuron is a circuit which integrates the input current pulse as a voltage until, at some threshold value (shown as the “Fire Threshold” in this diagram, and Ft in a later diagram), the circuit fires and provides an output pulse. In effect the LIF neuron is like a simple RC integrating circuit followed by a threshold sensor.
In simple terms, like its biological equivalent, the device is “Leaky” which means it is able to respond to both the amplitude and the rate at which pulses are received. Imagine if you will, something like an RC integrating circuit with a leaky capacitor, trying to charge to some threshold value.
As shown in the first circuit diagram, the front end of the circuit receives the processed reflected signals as pulses Vin0 and Vin1. The conductance of the ReRAMs determine the weight (current amplitude) of the signal delivered by the DPI.
In operation the negative feedback from the amplifier, via a current mirror, maintains the positive side input of the differential amp at whatever voltage (Vtop) is applied to its negative input. This results in a current through the ReRAM of Iweight = G0 * (Vtop -Vbot) and the same current to the DPI from the other side of the current mirror.
In typical operation one of the ReRAMs is programmed to a higher-conductance value, while the other is programmed to a very low conductance state. In this case, the conductance of G0 is programmed to produce a current of the required weight and G1 is basically turned off, or in a very low conductance state.
The feature of this circuit is that any input to Vin1 is effectively blocked because the low conductance state of the ReRAM G1 results in very little current output. This means that only the pulse Vin0 produces a useful output current for Vin0 and the circuit effectively ignores input Vin1, with the conductance of the ReRAM G0 determining the value or weight of the current delivered to the DPI and neuron.
As the DPI/LIF waveform characteristics diagram illustrates, the time delay is the result of integrating in the LIF, as a voltage, the decay of the current pulse delivered by the DPI synapse. A larger amplitude pulse will result in the integral reaching the same neuron firing level in a shorter time.
The Neuromorphic Platform
With the two essential building blocks, delay and current weight control circuit demonstrated, the next step for the team was the construction of the complete position location system.
At its heart this requires what is described as the neuromorphic platform (NP) requiring the use of eight ReRAMs, as illustrated below on the left. It is formed in part from two of the circuit element building blocks shown earlier in the second figure. This allows it to be programmed either to detect signals in the sequence Vin0 followed by Vin1 or the reverse, an essential requirement in order to determine which side of the centre line the object is located.
The ReRAMs are laid out in two columns. Each column is roughly equivalent to the circuit in the earlier diagram and likewise drives a synapse and neuron combination.
When the position of the object is on one side of the system centre line, the left-hand column of ReRAMs will deal with the first-arriving pulse, and the ReRAMs in the other column for the sequence and timing of the two pulses together. When the position of the object is on the other side of the system centre line the role of the columns will be reversed.
In this scheme one of the two ReRAMs from each horizontal pair, odd or even numbered, will be in a high-conductance state and the other in a low-conductance state, that is: If G0 is in a high conductance state then G1 will be set to a low conductance. If G2 is set to a low-conductance state then G3 is set to a high-conductance state. The low-conductance states prevent the input signals from having an effect upon the left or right side of the circuit. The high-conductance states are set so as to result in the desired current weight.
When an input arrives, Vin0 or Vin1, the conductance settings of the ReRAMs determine whether the left or right neuron, N0 or N1, is presented with a pulse. That pulse starts the neuron’s integration, when the neuron fires an output as a voltage, V0 or V1, is fed back into the appropriate side by ReRAMs G4-G7. If the even-numbered ReRAM is in a high-conductance state the neuron’s output is added to the left side, and if the odd-numbered ReRAM is in a high-conductance state then the neuron’s output is fed into the right side. As with G0-G3, one of the two ReRAMs from each pair, odd or even, will be in a high-conductance state and the other is in a low-conductance state, and the low-conductance state blocks the input signal, while the high-conductance state is set to achieve the desired current weight.
Key to the operation of this particular NP are the ReRAMs marked G0 (blue), G3 (red) and G7 (yellow), programmed respectively to conductance values of 73.5µS, 67.3µS and 40.2µS. For this NP those values determine the specific angular position and the location of the object with respect to the system centre line it will detect.
The ReRAMs coloured black are programmed to the low-conductance state (LCS) and will not be involved in detecting the position for this particular NP.
In operation the first reflected signal Vin0 arrives and generates a current weight determined by the conductance of the blue coloured ReRAM and follows the path I have highlighted in blue. After a delay time determined by the current weight and the actions of the DPI and LIF (N0) circuits, neuron N0 fires.
The output of N0 follows the feedback loop (blue) to the transistor of G7 (yellow) which produces a current of a weight determined by G7’s conductance value and starts to charge N1. However the weight of that current alone is insufficient to charge N1 to its firing level before the end of the sequence.
N1 can only fire if a signal Vin1 arrives and provides, via the path I have highlighted in red, an additional current pulse with a weight determined by the ReRAM G3 (red). The two currents from G7 and G3 when added together at the right time will charge the LIF capacitor to a voltage that will fire N1 for the angle ascribed to this NP.
The figure alongside the NP circuit uses the same colour scheme to illustrate the timing for the sequence of events from the arrival of Vin0 to the desired output pulse from N1. The unique time interval between the two input pulses for this NP is the sum of the two charging times determined by the conductance values of G0 and G7, i.e. (tb + tg).
If for this NP Vin1 arrives first it will produce a pulse from ReRAM G3 which will only be able to fire neuron N1 if G7 has been activated. In the same way if Vin1 arrives too late, with respect to Vin0, there will be an insufficient contribution from G3 to provide a sum of the inputs sufficient to fire N1 and no output pulse will be produced by N1.
The ability to program the ReRAMs to any conductance, from a continuum of values, means that effects of device-to-device variability can be avoided. As well as providing timing precision, iterative programming of the ReRAM also allows any possible problems associated with the variability of CMOS dynamics to be dealt with.
The block schematic to the left illustrates all the possible interconnect options for the NP which are determined by the different programmed states of the ReRAMs.
The complete position detecting system is illustrated below. Forty NPs, each identical to the one just described, are individually programmed to detect a different target angle. Only the NP that matches the time interval of the received pulses (and thus the angular position of the target with respect to the system centre-line) will fire.
The ReRAM neuromorphic computational map carries out the in-memory processing to provide an instantaneous output for the unique angular position of the object, and its location, left or right, with respect to the system centre line.
Position Resolution
It was reported the individual NP detector modules, each consisting of 8 ReRAMs, are able to provide an angular resolution of about 4 degrees. The time difference between pulses arriving from an object at an angle of 30 degrees and 34 degrees, would be of the order of 17µs.
At the system level, when all other system variables are taken into consideration, this translates to a 10-degree precision for an object located in front of the sensory system at a distance of 50 cm.
It is claimed those results compare favourably with other examples of sound localization neuromorphic systems reported by other authors.
The authors suggest that further improvements in localization precision should be able to be achieved by adding extra pMUTs (transducers), boosting the acoustic signal level, and bringing down the electronic noise.
Comparative Performance
With 40 NP modules in the computational map, the energy per operation (i.e. energy to localize an object) estimated by SPICE simulations is 21.6 nJ. The neuromorphic system is activated only at the arrival of an input event so standby power is not a consideration.
The in-memory computing from the stack of neuromorphic platform elements, forming the computational map, provides instant results. The alternative would be to capture the time-of-flight data in memory and at some time later number crunch it to extract the position information.
How do the two methods compare? The ReRAM-based system provides an improvement of two orders of magnitude when compared with a microcontroller-based system. There is some trade-off between the number of NP elements, the angular resolution, and power, which could be used to improve a computational map-based system.
The Next Steps
What the authors have demonstrated is an object location system that illustrates how the unique features of some of the emerging memory technologies, ReRAM in their example, can provide brain-like functions and efficient in-memory computation.
They have, by combining ReRAMs with micro-machined piezoelectric ultrasound transducers, provided some new ReRAM-based building blocks for those who wish to follow.
I have selected and concentrated on the ReRAM related aspects of the team’s work, from the more extensive detail in presented [1]. I would like to thank Elisa Vianello of CEA-Leti for assistance.
References
[1] Neuromorphic object localization using resistive memories and ultrasonic transducers, by Filippo Moro, Emmanuel Hardy, Bruno Fain et al, Nature Communications, June 2022. (https://doi.org/10.1038/s41467-022-31157-y)
One thought on “ReRAMs find a Neuromorphic Role in Owl-Inspired Object Location”
Comments are closed.