Vinh Q. Tran

Staff Research Scientist @ Google DeepMind

image

Hello! I’m a Staff Research Scientist at Google DeepMind in NY, working on core Gemini modeling.

These days my primary focus is improving Gemini training at the frontier, mostly on the pretraining side. I lead a small research group working on Gemini at the intersection of pretraining data, objectives, methodology, and architectures.

My general research interests revolve around anything that fundamentally changes the way we understand, train, or use language models at every stage of the modeling lifecycle.

Prior to research, I worked on a wide variety of projects across Google and Google Research including planet-scale distributed systems, database visualization, misinformation, fact checking, and news. In a previous life I was a full-stack web developer, and prior to that I attended Brown University as an undergrad (‘16).

Selected Publications

Transcending scaling laws with 0.1% extra compute (2022)
Yi Tay, Jason Wei, Hyung Won Chung, Vinh Q. Tran, David R. So, Siamak Shakeri, Xavier Garcia, Huaixiu Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc V. Le, Mostafa Dehghani
EMNLP 2023

UL2: Unifying Language Learning Paradigms (2022)
Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, Donald Metzler
ICLR 2023

Transformer Memory as a Differentiable Search Index (2022)
Yi Tay*, Vinh Q. Tran*, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, Donald Metzler
NeurIPS 2022

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization (2021)
Yi Tay*, Vinh Q. Tran*, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, and Donald Metzler.
ICLR 2022

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers (2022)
Alyssa Lees*, Vinh Q. Tran*, Yi Tay*, Jeffrey Sorensen, Jai Gupta, Donald Metzler, Lucy Vasserman
KDD 2022 ADS

Please see my full list and recent publications on my Scholar.

Mentoring & Service

Mentored Student Researchers & Interns @ Google

  1. Hritik Bansal (2024 Student Researcher, PhD @ UCLA)
  2. Ronak Pradeep (2023 Student Researcher, PhD @ Waterloo)
  3. Sanket Vaibhav Mehta (2022 Research Intern, now RS on my team!)
  4. Yuanzhe (Richard) Pang (2020 Research Intern, now RS @ Meta)
  5. Kate Lin (2019 STEP Intern, now SWE @ Google Research)
  6. Amy Pu (2019 STEP Intern, now SWE @ YouTube Music Recommendations)
  7. Daniil Dmitriev (2017 SWE Intern, now PhD @ ETH Zurich)

Reviewer for NeurIPS, ICLR, SIGIR, etc.