Called Illusion, it uses processors built with resistive RAM memory in a 3D stack built above the silicon logic, so it costs little energy or time to fetch data. By itself, even this isn’t enough, because neural networks are increasingly too large to fit in one chip. So the scheme also requires multiple such hybrid processors and an algorithm that both intelligently slices up the network among the processors and also knows when to rapidly turn processors off when they’re idle.
WHY IT MATTERS: not sure how practical this system is but sure seems easier to manufacture than Cerebras wafer-scale solution. No matter what architecture wins, we can see that there will be computer designs that will maximize neural net with billions of nodes (GPT3 has 175B) training.
See also cerebras link: http://blog.fmcs.digital/?q=cerebras