On the Reliability of Watermarks for Large Language Models
Large language models (LLMs) are now deployed to everyday use and positioned
to produce large quantities of text in the coming decade. Machine-generated
text may displace human-written text on the internet and has the potential to
be used for malicious purposes, such as spearphishing attacks and social media
bots. Watermarking is a simple and effective strategy for mitigating such harms
by enabling the detection and documentation of LLM-generated text. Yet, a
crucial question remains: How reliable is watermarking in realistic settings in
the wild? There, watermarked text might be mixed with other text sources,
paraphrased by human writers or other language models, and used for
applications in a broad number of domains, both social and technical. In this
paper, we explore different detection schemes, quantify their power at
detecting watermarks, and determine how much machine-generated text needs to be
observed in each scenario to reliably detect the watermark. We especially
highlight our human study, where we investigate the reliability of watermarking
when faced with human paraphrasing. We compare watermark-based detection to
other detection strategies, finding overall that watermarking is a reliable
solution, especially because of its sample complexity - for all attacks we
consider, the watermark evidence compounds the more examples are given, and the
watermark is eventually detected.
64813306de559d48dbb4de79
John Kirchenbauer
claimed_verified
2023-06-09T15:32:30.484Z
63d98af1897746d6496177df
/avatars/c5d0031c796a3c11bcb0d01b959168dc.svg
John Kirchenbauer
user
jwkirchenbauer
64813306de559d48dbb4de7a
Jonas Geiping
claimed_verified
2023-06-08T16:07:32.819Z
63d86dbf3130cadcaf8bdd11
/avatars/29d79a0c6dcec01111ef192fecd0fa7a.svg
Jonas Geiping
user
JonasGeiping
64813306de559d48dbb4de7b
Yuxin Wen
claimed_verified
2023-06-08T16:07:27.789Z
62cda27d248f9e6bc20c4261
/avatars/907b9464071ff7ed1ff064331595f12f.svg
Yuxin Wen
user
ywen
64813306de559d48dbb4de7c
Manli Shu
claimed_verified
2023-06-27T07:28:06.203Z
63bdcb129a15a3e94198e7d3
https://cdn-avatars.huggingface.co/v1/production/uploads/63bdcb129a15a3e94198e7d3/wFVtkekqyQLbS0-78fqcs.jpeg
Manli Shu
user
Manli
64813306de559d48dbb4de7d
Khalid Saifullah
claimed_verified
2023-06-12T09:12:42.578Z
603406de3d343696d96ef2a6
https://cdn-avatars.huggingface.co/v1/production/uploads/603406de3d343696d96ef2a6/FS3xGIC9G_JpyQ806VAyc.jpeg
Khalid Saifullah
user
khalidsaifullaah
64813306de559d48dbb4de7e
Kezhi Kong
64813306de559d48dbb4de7f
Kasun Fernando
64813306de559d48dbb4de80
Aniruddha Saha
64813306de559d48dbb4de81
Micah Goldblum
64813306de559d48dbb4de82
Tom Goldstein
On the Reliability of Watermarks for Large Language Models
Large language models (LLMs) are now deployed to everyday use and positioned
to produce large quantities of text in the coming decade. Machine-generated
text may displace human-written text on the internet and has the potential to
be used for malicious purposes, such as spearphishing attacks and social media
bots. Watermarking is a simple and effective strategy for mitigating such harms
by enabling the detection and documentation of LLM-generated text. Yet, a
crucial question remains: How reliable is watermarking in realistic settings in
the wild? There, watermarked text might be mixed with other text sources,
paraphrased by human writers or other language models, and used for
applications in a broad number of domains, both social and technical. In this
paper, we explore different detection schemes, quantify their power at
detecting watermarks, and determine how much machine-generated text needs to be
observed in each scenario to reliably detect the watermark. We especially
highlight our human study, where we investigate the reliability of watermarking
when faced with human paraphrasing. We compare watermark-based detection to
other detection strategies, finding overall that watermarking is a reliable
solution, especially because of its sample complexity - for all attacks we
consider, the watermark evidence compounds the more examples are given, and the
watermark is eventually detected.
Watermarking LLM-generated text is reliable and effective, especially in scenarios involving human paraphrasing and multiple text sources, with evidence compounding as more samples are observed.
large language models LLMs machine-generated text watermarking spearphishing attacks social media bots detection schemes human paraphrasing sample complexity