Sign up for our newsletter and get the latest big data news and analysis.

Research Highlights: Transformer Feed-Forward Layers Are Key-Value Memories

In this regular column, we take a look at highlights for important research topics of the day for big data, data science, machine learning, AI and deep learning. It’s important to keep connected with the research arm of the field in order to see where we’re headed. In this edition, if you (like me) have wondered what the feed-forward layers in transformer models are actually doing, this is a pretty interesting paper on that topic. Enjoy!

Habana Labs and Hugging Face Partner to Accelerate Transformer Model Training

Habana® Labs, a pioneer in high-efficiency, purpose-built deep learning processors, and Hugging Face, the home of Transformer models, announced that they’re joining forces to make it easier and quicker to train high-quality transformer models. Thanks to the integration of Habana’s SynapseAI software suite with the Hugging Face Optimum open-source library, data scientists and machine learning engineers can now accelerate their Transformer training jobs on Habana processors with just a few lines of code and enjoy greater productivity as well as lower training cost.