Hello! I’m a Staff Research Engineer at Google Research NY where I develop methods for sequence modeling / NLP, machine learning, and AI. My research interests are in all things related to improving, expanding, or rethinking the functionality of Transformers and other state-of-the-art sequence models. At Google, I have made significant contributions to efficient token-free models, generative retrieval, safety, and Bard creativity.
Prior to research, I worked on a wide variety of projects across Google including natural language generation for news, fact checking, database visualization, and distributed systems. In a previous life I was a full-stack software developer, and prior to that I attended Brown University as an undergrad (‘16).
Transcending scaling laws with 0.1% extra compute (2022)
Yi Tay, Jason Wei, Hyung Won Chung, Vinh Q. Tran, David R. So, Siamak Shakeri, Xavier Garcia, Huaixiu Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc V. Le, Mostafa Dehghani
UL2: Unifying Language Learning Paradigms (2022)
Yi Tay*, Mostafa Dehghani*, Vinh Q. Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, Donald Metzler
Transformer Memory as a Differentiable Search Index (2022)
Yi Tay*, Vinh Q. Tran*, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, Donald Metzler
A New Generation of Perspective API: Efficient Multilingual Character-level Transformers (2022)
Alyssa Lees*, Vinh Q. Tran*, Yi Tay*, Jeffrey Sorensen, Jai Gupta, Donald Metzler, Lucy Vasserman
KDD 2022 ADS
ExT5: Towards extreme multi-task scaling for transfer learning (2021)
Vamsi Aribandi*, Yi Tay*, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, and Donald Metzler.
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization (2021)
Yi Tay*, Vinh Q. Tran*, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, and Donald Metzler.