Applied Physics/Physics Colloquium: Surya Ganguli- "Theories of Neural Computation Underlying Learning, Imagination, Reasoning and Scaling: of Mice and Machines"

Abstract: Three remarkable abilities of brains and machines are to: (1) learn new behaviors from a single example, (2) creatively imagine new possibilities, (3) learn language, and (4) perform mathematical reasoning.  I will discuss simple analytic yet quantitatively predictive theories of how (1) mice learn to accurately navigate on the first encounter in a new environment; (2) how diffusion models creatively imagine exponentially many new images; (3) how the structure of natural language governs how much data is required to learn it, and (4) how language models can better do mathematically reasoning.  Theoretical physics approaches are essential in deriving all of these theories, spanning techniques like statistical mechanics, pattern formation, nonlinear dynamics, high dimensional geometry, scaling analysis, and control of entropy.  More generally, just as biology once provided a new frontier of complexity for physics to study, I suggest that AI now provides a new frontier in which physics can expand to yield a new, fundamental scientific understanding of intelligence.

Surya Ganguli is a professor of Applied Physics at Stanford, an Associate Director of Stanford’s Human Centered AI Institute, and a Venture Partner at General Catalyst. Dr. Ganguli triple majored in physics, mathematics, and EECS at MIT, completed a PhD in string theory at Berkeley, and a postdoc in theoretical neuroscience at UCSF. He has also been a visiting researcher at both Google and Meta AI, and a venture partner at a16z. His research spans the fields of AI, physics, and neuroscience, focusing on understanding and improving how both biological and artificial neural networks learn striking emergent computations.