Are Large Language Models Sensitive to the Motives Behind Communication?
Addison J. Wu, Ryan Liu, Kerem Oktar, Ted Sumers, Tom Griffiths
Human communication is $\textit{motivated}$: people speak, write, and create content with a particular communicative intent in mind. As a result, the information large language models (LLMs) and associated AI agents receive is inherently biased by humans' intentions and incentives. People are remarkably attuned to navigating such biased information---we easily identify benevolent or self-serving motives in order to know what information to trust. For LLMs to be effective in the real world, they too must critically evaluate content by accounting for the motivations of the source: for example, discounting the claims made in a sales pitch. In this paper we undertake a comprehensive study of whether LLMs have this capacity of $\textit{motivational vigilance}$. We first use controlled experiments from cognitive science to identify that LLMs follow rational models of learning from motivated testimony, successfully discounting information from biased sources in a human-like manner. We then extend our evaluation to online recommendations, a more naturalistic reflection of LLM agents' information ecosystems. In these settings, we find that LLMs' inferences do not track the rational models' predictions nearly as closely---in part due to the presence of additional information that distract LLMs from vigilance-relevant considerations. Accordingly, a simple steering intervention that boosts the salience of intentions and incentives substantially increases the correspondence between LLMs and the rational model. These results suggest that LLMs possess a basic sensitivity to the motivations of others, but generalizing to novel real-world settings will require further improvements to these models.