Self-Recognition in Language Models
A rapidly growing number of applications rely on a small set of closed-source
language models (LMs). This dependency might introduce novel security risks if
LMs develop self-recognition capabilities. Inspired by human identity
verification methods, we propose a novel approach for assessing
self-recognition in LMs using model-generated "security questions". Our test
can be externally administered to keep track of frontier models as it does not
require access to internal model parameters or output probabilities. We use our
test to examine self-recognition in ten of the most capable open- and
closed-source LMs currently publicly available. Our extensive experiments found
no empirical evidence of general or consistent self-recognition in any examined
LM. Instead, our results suggest that given a set of alternatives, LMs seek to
pick the "best" answer, regardless of its origin. Moreover, we find indications
that preferences about which models produce the best answers are consistent
across LMs. We additionally uncover novel insights on position bias
considerations for LMs in multiple-choice settings.
66910b82f84c9502c4422cb0
Tim R. Davidson
extracted_confirmed
2024-07-12T11:40:19.097Z
65c9e65bd6bed59d801a5eb3
/avatars/431bc46350cbbe996bb058e03ccc2fd0.svg
Tim Davidson
user
trdavidson
66910b82f84c9502c4422cb1
Viacheslav Surkov
admin_assigned
2024-07-12T12:02:17.091Z
656f19b7d6f611cd4d61ec8c
/avatars/489fc5dcca17bf18ece4091783c14fa9.svg
Viacheslav Surkov
user
surokpro
66910b82f84c9502c4422cb2
Veniamin Veselovsky
admin_assigned
2024-07-12T12:02:23.224Z
64f456db8db247808c1fc0cc
/avatars/2032dccca99ac0dac6f69eeecfc90087.svg
Veniamin Veselovsky
user
vminvsky
66910b82f84c9502c4422cb3
Giuseppe Russo
66910b82f84c9502c4422cb4
Robert West
66910b82f84c9502c4422cb5
Caglar Gulcehre
admin_assigned
2024-07-12T12:02:49.619Z
64d28085db0b63b977a6c3f8
/avatars/994cc3ff7c18650fe2535e473d56b240.svg
Caglar Gulcehre
user
caglarg
Self-Recognition in Language Models
A rapidly growing number of applications rely on a small set of closed-source
language models (LMs). This dependency might introduce novel security risks if
LMs develop self-recognition capabilities. Inspired by human identity
verification methods, we propose a novel approach for assessing
self-recognition in LMs using model-generated "security questions". Our test
can be externally administered to keep track of frontier models as it does not
require access to internal model parameters or output probabilities. We use our
test to examine self-recognition in ten of the most capable open- and
closed-source LMs currently publicly available. Our extensive experiments found
no empirical evidence of general or consistent self-recognition in any examined
LM. Instead, our results suggest that given a set of alternatives, LMs seek to
pick the "best" answer, regardless of its origin. Moreover, we find indications
that preferences about which models produce the best answers are consistent
across LMs. We additionally uncover novel insights on position bias
considerations for LMs in multiple-choice settings.
A study assesses self-recognition in language models using model-generated security questions, revealing no evidence of self-recognition and emphasizing models' preference for the "best" answer.
language models self-recognition security questions multiple-choice settings position bias