Quantifying Elicitation of Latent Capabilities in Language Models
Elizabeth Donoway, Hailey Joren, Arushi Somani, Henry Sleight, Julian Michael, Michael Deweese, John Schulman, Ethan Perez, Fabien Roger, Jan Leike
Large language models often possess latent capabilities that lie dormant unless explicitly elicited, or surfaced, through fine-tuning or prompt engineering. Predicting, assessing, and understanding these latent capabilities pose significant challenges in the development of effective, safe AI systems. In this work, we recast elicitation as an information-constrained fine-tuning problem and empirically characterize an upper bound on the minimal number of parameters needed to reach a certain task accuracy. We find that training as few as 10–100 randomly chosen parameters---several orders of magnitude fewer than state-of-the-art parameter-efficient methods---can recover up to 50\% of the performance gap between pretrained-only and fully fine-tuned models, and 1,000s to 10,000s of parameters can recover 95\% of this performance gap. We show that a logistic curve fits the relationship between the number of trained parameters and the fraction of the performance gap recovered. To help explain this behavior, we consider a simplified picture of elicitation via fine-tuning where each trainable parameter serves as an encoding mechanism for accessing task-specific knowledge. We observe a relationship between the number of trained parameters and the effectiveness in eliciting latent capabilities, offering a potential route to distinguish elicitation from teaching.