Skip to main content

Wait, if this scales up would this let us run these things on, say, our phones? If it replaces the transformer, could it replace other uses of the transformer, like Whisper? If it can do that and deliver equivalent or better results for those kinds of speed-ups, ML applications on edge computing will explode, again. And given the accessibility benefits of some existing tech, imagine being able to run that on something other than the cloud!
This new technology could blow away GPT-4 and everything like it zdnet.com/article/this-new-tec… #AI #MachineLearning

This new technology could blow away GPT-4 and everything like it

The Hyena code is able to handle amounts of data that make GPT-style technology run out of memory and fail.
Tiernan Ray (ZDNET)
#AI #machinelearning
in reply to x0

Mikołaj Hołysz
Mikołaj Hołysz
• •
It very much seems like it. If this actually works, it’s going to be a very, very big thing.
in reply to x0

x0
x0
• •
Or speech synthesis. Can you use the transformer architecture for speech synthesis? Is that what WaveNet did?
in reply to x0

Mikołaj Hołysz
Mikołaj Hołysz
• •
You can. Not sure what WaveNet does, but Tacotron still mostly uses LSTMs (the architecture that was used before transformers were a thing), with a bit of attention sprinkled on top. Tortoise, the slow but much higher quality engine uses transformers, though.
⇧