The new Hyena Technology- edited by Aishee Biswas

The Hyena technology has demonstrated comparable accuracy in benchmark tests, such as question answering, with significantly less computing power. It can even handle larger amounts of text than GPT-style technology, which can run out of memory and fail. This success at a smaller scale suggests that attention, which was the focus of the 2017 report "Attention is all you need" introducing  Google's Transformer AI program , may not be the only key component.
 

However, the Transformer has a significant drawback. It relies on "attention" to transfer information from one set of symbols, such as words, to another set of symbols, such as the response generated by ChatGPT. The attention mechanism is the fundamental feature of all major language models, including GPT-4 and ChatGPT. However, it has a "quadratic" computational complexity, which means that the time required for ChatGPT to generate an answer grows proportionally to the square of the volume of input data.

 

The excessive amount of data in  prompts or conversations with language models like ChatGPT can cause computational issues , slowing down the program or requiring more GPU chips to run faster. In their paper 'Hyena Hierarchy: Towards Larger Convolutional Language Models', published on arXiv preprint server, lead author Michael Poli and his team suggest using Hyena, a sub-quadratic alternative to the Transformer's attention function.

 

The name "Hyena" given to the new language model proposed by the authors is not explained, but it can be speculated that the program was named after the African animal that hunts for miles and miles. The authors, however, are more interested in the concept of "hierarchy", which is reflected in the title of their paper. Hyenas in the animal kingdom also follow a strict hierarchy within their families, with varying levels of rank establishing dominance. In a similar fashion, the Hyena program utilizes simple operations repeatedly to build a hierarchy of data processing. It is this combination of operations that inspired the name of the program.