Models of Learning

How does a brain decide what is worth learning — and how fast? For decades, dopamine was cast as a teaching signal that reports errors in reward prediction. We found the story is richer, and more powerful: as animals learn directly from their own actions, mesolimbic dopamine sets the rate of that learning — an adaptive learning rate rather than a signed error signal.

By pairing quantitative behavior with computational models and causal, cell-type-specific manipulations, we aim to explain not just what animals learn, but how the brain's own algorithms make learning so remarkably fast. Three threads run through this work.

An adaptive rate, not an error signal

Reinforcement learning distinguishes value learning — refining predictions of reward — from policy learning, which directly improves the actions an animal performs. Classic work mapped midbrain dopamine onto the reward-prediction error of value learning. But dopamine is also tightly intertwined with the production and monitoring of action.

Following naive, head-fixed mice as they first learned, we found that individual differences in early dopamine responses predicted the emergence of a learned behavioral policy — not the value of a predictive cue. A neural-network model (ACTR) in which dopamine sets an adaptive learning rate, rather than a signed error, reproduced the behavior, and calibrated optogenetic manipulations matched its predictions. Dopamine, in short, tunes how fast an animal learns from its own actions.

A global learning rate for foraging

Natural foraging means choosing among many resources at once. We let naive, thirsty mice freely sample six options of differing quality spread across a large (~2 m) arena — and within minutes they settled into a near-optimal “matching” strategy, after only about a hundred rewarded choices.

A reinforcement-learning model with separate “stay” and “leave” states and a single, dynamic global learning rate captured this behavior. Dopamine in the nucleus accumbens core — but not the dorsomedial striatum — tracked that learning-rate term, and optogenetic manipulation of accumbens dopamine bidirectionally altered learning exactly as the model predicted.

Reward magnitude is a lever on cognition

The field has long minimized reward size to maximize the number of trials, assuming learning depends mostly on repetition. We asked what happens with much larger rewards — and found that outsized rewards can improve learning efficiency by one to two orders of magnitude across navigation, motor-skill, and decision-making tasks.

Larger rewards drive longer, more sustained dopamine release, and prolonged optogenetic dopamine reproduced much of the benefit — boosting both the learning rate and sustained task engagement. Because engagement is the largest source of variability between individuals, bigger rewards markedly narrow the gap between fast and slow learners.