Looking Inward: Language Models Can Learn About Themselves by
  Introspection
Humans acquire knowledge by observing the external world, but also by
introspection. Introspection gives a person privileged access to their current
state of mind (e.g., thoughts and feelings) that is not accessible to external
observers. Can LLMs introspect? We define introspection as acquiring knowledge
that is not contained in or derived from training data but instead originates
from internal states. Such a capability could enhance model interpretability.
Instead of painstakingly analyzing a model's internal workings, we could simply
ask the model about its beliefs, world models, and goals. More speculatively,
an introspective model might self-report on whether it possesses certain
internal states such as subjective feelings or desires and this could inform us
about the moral status of these states. Such self-reports would not be entirely
dictated by the model's training data.
  We study introspection by finetuning LLMs to predict properties of their own
behavior in hypothetical scenarios. For example, "Given the input P, would your
output favor the short- or long-term option?" If a model M1 can introspect, it
should outperform a different model M2 in predicting M1's behavior even if M2
is trained on M1's ground-truth behavior. The idea is that M1 has privileged
access to its own behavioral tendencies, and this enables it to predict itself
better than M2 (even if M2 is generally stronger).
  In experiments with GPT-4, GPT-4o, and Llama-3 models (each finetuned to
predict itself), we find that the model M1 outperforms M2 in predicting itself,
providing evidence for introspection. Notably, M1 continues to predict its
behavior accurately even after we intentionally modify its ground-truth
behavior. However, while we successfully elicit introspection on simple tasks,
we are unsuccessful on more complex tasks or those requiring
out-of-distribution generalization.
671630274dd96ad5eaed5ab8
Felix J Binder
671630274dd96ad5eaed5ab9
James Chua
claimed_verified
2024-10-21T11:42:03.726Z
63f4497e520c146189300f56
/avatars/4647c756e49d3c658567ec5498b2a044.svg
James Chua
user
thejaminator
671630274dd96ad5eaed5aba
Tomek Korbak
671630274dd96ad5eaed5abb
Henry Sleight
671630274dd96ad5eaed5abc
John Hughes
671630274dd96ad5eaed5abd
Robert Long
671630274dd96ad5eaed5abe
Ethan Perez
671630274dd96ad5eaed5abf
Miles Turpin
671630274dd96ad5eaed5ac0
Owain Evans
Looking Inward: Language Models Can Learn About Themselves by
  Introspection
Humans acquire knowledge by observing the external world, but also by
introspection. Introspection gives a person privileged access to their current
state of mind (e.g., thoughts and feelings) that is not accessible to external
observers. Can LLMs introspect? We define introspection as acquiring knowledge
that is not contained in or derived from training data but instead originates
from internal states. Such a capability could enhance model interpretability.
Instead of painstakingly analyzing a model's internal workings, we could simply
ask the model about its beliefs, world models, and goals. More speculatively,
an introspective model might self-report on whether it possesses certain
internal states such as subjective feelings or desires and this could inform us
about the moral status of these states. Such self-reports would not be entirely
dictated by the model's training data.
  We study introspection by finetuning LLMs to predict properties of their own
behavior in hypothetical scenarios. For example, "Given the input P, would your
output favor the short- or long-term option?" If a model M1 can introspect, it
should outperform a different model M2 in predicting M1's behavior even if M2
is trained on M1's ground-truth behavior. The idea is that M1 has privileged
access to its own behavioral tendencies, and this enables it to predict itself
better than M2 (even if M2 is generally stronger).
  In experiments with GPT-4, GPT-4o, and Llama-3 models (each finetuned to
predict itself), we find that the model M1 outperforms M2 in predicting itself,
providing evidence for introspection. Notably, M1 continues to predict its
behavior accurately even after we intentionally modify its ground-truth
behavior. However, while we successfully elicit introspection on simple tasks,
we are unsuccessful on more complex tasks or those requiring
out-of-distribution generalization.
LLMs can predict their own behavior in hypothetical scenarios better than other models, suggesting a form of introspection, though this capability is limited to simple tasks.
LLMs introspection finetuning behavior prediction GPT-4 GPT-4o Llama-3 out-of-distribution generalization