Video From Applied Physics/Physics Colloquium: Ethan Dyer - Lessons from scale for large language models and quantitative reasoning

APPLIED PHYSICS/PHYSICS COLLOQUIUM

Tuesday, November 15, 2022

3:30 p.m. on campus in Hewlett Teaching Center, Rm. 200

Ethan Dyer

“Lessons from scale for large language models and quantitative reasoning”

Large language models trained on diverse training data have shown impressive results on many tasks involving natural language -- in many cases matching or exceeding human performance. Some measures of progress exhibit remarkably robust power-law improvement over many orders of magnitude in dataset, model and compute scale, while other capabilities remain difficult to extrapolate. One domain which has traditionally been challenging for such models is multi-step quantitative reasoning for mathematics and science. I will discuss recent progress attempting to understand and extrapolate model capabilities with scale and Minerva, a large language model designed to perform multi-step STEM problem solving.