AI systems can spend substantial effort moving information between memory and processors. IBM’s phase-change chip takes a different approach: it stores neural network connection strengths in memory and performs operations there. In demonstrations involving speech recognition, the chip matched the performance of an equivalent system on traditional processors while using less energy.
Memory that can do computation
Phase-change memory stores information by changing the state of a small patch of material. A slow cooling process creates an orderly crystal that conducts electricity more easily, while rapid cooling produces a disordered state with higher resistance. The contrast between those states can store a bit until enough voltage melts the material again.
For neural networks, the material has another useful property: its resistance can be set to values between the two extremes. Those intermediate values can represent the strength of a connection between nodes. Passing current through the memory bit then performs part of the network’s computation where the connection strength is stored.
This approach is called in-memory processing. It can reduce the need to shuttle data between separate memory and computing units, a source of energy use in large AI systems. IBM had demonstrated the underlying idea before; this chip brings together a larger array and the hardware needed to connect its components into a more functional processor.
A chip built from large arrays
The chip’s basic building block is a tile: a crossbar array of phase-change bits arranged 512 units wide by 2,048 units deep. There are 34 tiles on each chip, for about 35 million phase-change bits. Hardware on the chip supports communication within and across tiles without requiring analog-to-digital conversion for that communication.
Traditional processing units and static RAM also play a role. They help manage communication and translate between the chip’s analog and digital parts. Connection strengths can use a variable number of bits, and multiple chips can work together on larger problems. In the largest demonstration described, 140 million phase-change bits were spread across five chips.
To configure the system, researchers first used an existing AI system and set the phase-change bits to match its connection strengths. Once configured, the analysis could run repeatedly without requiring additional energy from the phase-change portion of the chip.
Speech recognition with lower energy use
The researchers tested two speech recognition tasks. One involved recognizing a limited set of keywords, like a system might use to handle spoken responses on an automated call. The other performed general speech recognition with a condensed vocabulary.
On both tasks, the chip matched the performance of an equivalent AI system running on traditional processors. At peak performance, it reached 12.4 trillion operations for each watt of power used. The source reports that this is many times less power than a traditional processor uses for equivalent operations.
That result points to a potential benefit for AI workloads that fit the chip’s design: a configured network can repeat its analysis without repeatedly drawing energy to reset the phase-change memory. The demonstration shows a way to bring memory and computation together at a scale larger than a basic proof of concept.
The limits of a fixed network
The chip is not a general-purpose AI processor. It works with a specific type of neural network, and some problems are not a good fit for that network structure. Its energy savings also depend on keeping the network’s connections static. Changing them requires resetting the phase-change bits, which takes significantly more energy.
That limitation makes the chip a poor fit for AI training. The neural network’s training process had to be tailored so its results could be translated to the phase-change hardware. The chip is better suited to running a compatible, already configured network than to repeatedly changing that network.
There may be room to improve its energy use. The chip was made using a 14-nanometer process, and the researchers said they had not optimized energy consumption in the communication and digital-to-analog conversion portions. The speech recognition results establish a promising use case, while the fixed-network requirement defines where the approach currently applies.