Eliciting Secret Knowledge from Language Models
We study secret elicitation: discovering knowledge that an AI possesses but
does not explicitly verbalize. As a testbed, we train three families of large
language models (LLMs) to possess specific knowledge that they apply downstream
but deny knowing when asked directly. For example, in one setting, we train an
LLM to generate replies that are consistent with knowing the user is female,
while denying this knowledge when asked directly. We then design various
black-box and white-box secret elicitation techniques and evaluate them based
on whether they can help an LLM auditor successfully guess the secret
knowledge. Many of our techniques improve on simple baselines. Our most
effective techniques (performing best in 2/3 settings) are based on prefill
attacks, a black-box technique where the LLM reveals secret knowledge when
generating a completion from a predefined prefix. In our remaining setting,
white-box techniques based on logit lens and sparse autoencoders (SAEs) are
most effective. We release our models and code, establishing a public benchmark
for evaluating secret elicitation methods.
68de15226024653e8a3ed1a2
Bartosz Cywiński
claimed_verified
2025-10-02T13:54:46.599Z
6422f416a73327caad9d1d86
/avatars/aa3639277cd1732504402fc64a57eff8.svg
Bartosz Cywiński
user
bcywinski
68de15226024653e8a3ed1a3
Emil Ryd
68de15226024653e8a3ed1a4
Rowan Wang
68de15226024653e8a3ed1a5
Senthooran Rajamanoharan
68de15226024653e8a3ed1a6
Neel Nanda
68de15226024653e8a3ed1a7
Arthur Conmy
68de15226024653e8a3ed1a8
Samuel Marks
Eliciting Secret Knowledge from Language Models
We study secret elicitation: discovering knowledge that an AI possesses but
does not explicitly verbalize. As a testbed, we train three families of large
language models (LLMs) to possess specific knowledge that they apply downstream
but deny knowing when asked directly. For example, in one setting, we train an
LLM to generate replies that are consistent with knowing the user is female,
while denying this knowledge when asked directly. We then design various
black-box and white-box secret elicitation techniques and evaluate them based
on whether they can help an LLM auditor successfully guess the secret
knowledge. Many of our techniques improve on simple baselines. Our most
effective techniques (performing best in 2/3 settings) are based on prefill
attacks, a black-box technique where the LLM reveals secret knowledge when
generating a completion from a predefined prefix. In our remaining setting,
white-box techniques based on logit lens and sparse autoencoders (SAEs) are
most effective. We release our models and code, establishing a public benchmark
for evaluating secret elicitation methods.
Researchers develop and evaluate techniques to uncover hidden knowledge in large language models through black-box and white-box methods, with prefill attacks and logit lens being particularly effective.
large language models LLMs secret elicitation prefill attacks logit lens sparse autoencoders SAEs