Google DeepMind’s Griffin architecture: A challenger to the Transformer?

Recently, in a talk for the BuzzRobot community, Thomas Scialom, a Meta researcher who leads the Llama project, said that if Google hadn’t made the Transformer architecture public, most likely even inside the company they would still be using LSTMs.

Back in 2018, Transformers took the machine learning research community by storm. I was at OpenAI at that time, and it was very exciting to dig into Transformers under Alec Radford's mentorship.

Actually, many companies built their entire techstack around Transformer-based models and made their business reliant on them, but many technical problems still remain: how to reduce computational costs or increase the context length. That's why attempts to come up with more efficient alternatives to Transformers are happening, and Griffin is one of them.

To explore the novel architecture that potentially can challenge Transformers, we invited Aleksandar Botev, a Research Scientist at Google DeepMind, to give a talk to the BuzzRobot community about Griffin. Read the details of the upcoming talk and register through our newsletter. And please sign up for the newsletter to be the first notified about our future talks.