Flexible inference of learning rules from de novo learning data using neural networks
Yuhan Helena Liu, Victor Geadah, Jonathan Pillow
Understanding how animals learn is a central challenge in neuroscience, with growing relevance to the development of animal- or human-aligned artificial intelligence. However, most existing approaches to this problem assume specific parametric forms for the learning rule (e.g., Q-learning, policy gradient) or are limited to simplified settings like bandit tasks, which do not involve learning a new input–output mapping from scratch. In contrast, animals must often learn to perform new behaviors de novo, which presents a rich challenge for learning rule inference. Here, we propose a flexible approach for inferring learning rules directly from animal decision-making data during de novo task learning. Specifically, we use a deep neural network (DNN) to parameterize a learning rule that governs the update of an animal's policy weights after each trial. We validate our approach by showing that this model can recover specific ground-truth learning rules in simulated datasets. We then consider more complex learning rules by incorporating a recurrent neural network (RNN), which allows per-trial weight updates to depend on the history of stimuli, decisions, and rewards over multiple trials. We applied our methods to a large behavioral dataset of mice learning a sensory decision-making task over multiple weeks and observed improved predictions on held-out data. The inferred learning rules revealed that mice make substantially different weight updates after correct and incorrect trials, and that learning is non-Markovian, with strong dependencies on history over multiple trials. Overall, these results provide a flexible new framework for inferring biological learning rules from behavioral data and yield powerful insights into the features of de novo task learning.