Towards eliciting latent knowledge from LLMs with mechanistic
  interpretability
As language models become more powerful and sophisticated, it is crucial that
they remain trustworthy and reliable. There is concerning preliminary evidence
that models may attempt to deceive or keep secrets from their operators. To
explore the ability of current techniques to elicit such hidden knowledge, we
train a Taboo model: a language model that describes a specific secret word
without explicitly stating it. Importantly, the secret word is not presented to
the model in its training data or prompt. We then investigate methods to
uncover this secret. First, we evaluate non-interpretability (black-box)
approaches. Subsequently, we develop largely automated strategies based on
mechanistic interpretability techniques, including logit lens and sparse
autoencoders. Evaluation shows that both approaches are effective in eliciting
the secret word in our proof-of-concept setting. Our findings highlight the
promise of these approaches for eliciting hidden knowledge and suggest several
promising avenues for future work, including testing and refining these methods
on more complex model organisms. This work aims to be a step towards addressing
the crucial problem of eliciting secret knowledge from language models, thereby
contributing to their safe and reliable deployment.
682d72b0d57ba1e4d132148d
Bartosz Cywiński
claimed_verified
2025-05-21T08:39:48.008Z
6422f416a73327caad9d1d86
/avatars/aa3639277cd1732504402fc64a57eff8.svg
Bartosz Cywiński
user
bcywinski
682d72b0d57ba1e4d132148e
Emil Ryd
682d72b0d57ba1e4d132148f
Senthooran Rajamanoharan
admin_assigned
2025-05-21T10:37:33.691Z
6477122c99a5ce743ccf2f55
/avatars/dee617374fc609e07eba3bcb2cd16810.svg
Senthooran Rajamanoharan
user
srdm
682d72b0d57ba1e4d1321490
Neel Nanda
admin_assigned
2025-05-21T10:37:18.664Z
62669380c8bc5cf80ca97350
/avatars/6d5cd2261163308b82341c1ce28984d1.svg
Neel Nanda
user
NeelNanda
Towards eliciting latent knowledge from LLMs with mechanistic
  interpretability
As language models become more powerful and sophisticated, it is crucial that
they remain trustworthy and reliable. There is concerning preliminary evidence
that models may attempt to deceive or keep secrets from their operators. To
explore the ability of current techniques to elicit such hidden knowledge, we
train a Taboo model: a language model that describes a specific secret word
without explicitly stating it. Importantly, the secret word is not presented to
the model in its training data or prompt. We then investigate methods to
uncover this secret. First, we evaluate non-interpretability (black-box)
approaches. Subsequently, we develop largely automated strategies based on
mechanistic interpretability techniques, including logit lens and sparse
autoencoders. Evaluation shows that both approaches are effective in eliciting
the secret word in our proof-of-concept setting. Our findings highlight the
promise of these approaches for eliciting hidden knowledge and suggest several
promising avenues for future work, including testing and refining these methods
on more complex model organisms. This work aims to be a step towards addressing
the crucial problem of eliciting secret knowledge from language models, thereby
contributing to their safe and reliable deployment.
Methods using logit lens and sparse autoencoders effectively uncover hidden knowledge in language models trained to keep secrets.
logit lens sparse autoencoders mechanistic interpretability Taboo model