Causal Discovery and Inference through Next-Token Prediction
Eivinas Butkus, Nikolaus Kriegeskorte
Some argue that deep neural networks are fundamentally _statistical_ systems that fail to capture the causal generative processes behind their training data. Here we demonstrate that a GPT-style transformer trained for next-token prediction can simultaneously discover instances of linear Gaussian structural causal models (SCMs) and learn to answer counterfactual queries about them. First, we show that the network generalizes to counterfactual queries about SCMs for which it saw _only_ strings describing noisy interventional data. Second, we decode the implicit SCM from the network's residual stream activations and use gradient descent to intervene on that "mental" SCM with predictable effects on the model's output. Our results suggest that neural networks trained using statistical prediction objectives on passively observed data may nevertheless discover and learn to use causal models of the world.