Over the last 12 months, we’ve seen significant breakthroughs on natural language processing tasks such as machine translation, document generation and syntactic parsing. A few of these breakthroughs were produced by models such as Google AI’s BERT, OpenAI’s GPT, and Microsoft’s MT-DNN. A key enabler of these results is in fact the Transformer model architecture that was co-authored by Ashish Vaswani and colleagues in 2017 at NIPS and has now been cited >1,500 times.

At this year’s RAAIS, we’re excited to be hosting Ashish, who is a Senior Research Scientist in the Brain team within GoogleAI in San Francisco. His research has focussed on developing pure attention based models, such as the Transformer for generation and classification. Before joining Google, he was a PhD student, and later Research Scientist, in natural language processing at the University of Southern California Information Sciences Institute.