Robust Multi-bit Text Watermark with LLM-based Paraphrasers
We propose an imperceptible multi-bit text watermark embedded by paraphrasing
with LLMs. We fine-tune a pair of LLM paraphrasers that are designed to behave
differently so that their paraphrasing difference reflected in the text
semantics can be identified by a trained decoder. To embed our multi-bit
watermark, we use two paraphrasers alternatively to encode the pre-defined
binary code at the sentence level. Then we use a text classifier as the decoder
to decode each bit of the watermark. Through extensive experiments, we show
that our watermarks can achieve over 99.99\% detection AUC with small (1.1B)
text paraphrasers while keeping the semantic information of the original
sentence. More importantly, our pipeline is robust under word substitution and
sentence paraphrasing perturbations and generalizes well to
out-of-distributional data. We also show the stealthiness of our watermark with
LLM-based evaluation. We open-source the code:
https://github.com/xiaojunxu/multi-bit-text-watermark.
6756459a8e874bcf29364ca1
Xiaojun Xu
claimed_verified
2024-12-09T11:06:12.238Z
675644c7d83c390221b45a61
/avatars/ee2d659662e39a9adf3ea859fbb8d051.svg
Xiaojun Xu
user
xiaojunxu
6756459a8e874bcf29364ca2
Jinghan Jia
admin_assigned
2024-12-10T13:10:20.552Z
62b35eb9003cd12329d748a4
/avatars/871cedeb9aa62c5ac57d54287b12afa7.svg
Jinghan Jia
user
flyingbugs
6756459a8e874bcf29364ca3
Yuanshun Yao
6756459a8e874bcf29364ca4
Yang Liu
6756459a8e874bcf29364ca5
Hang Li
admin_assigned
2024-12-10T13:10:59.902Z
61342a4b488458a484dee6c4
https://cdn-avatars.huggingface.co/v1/production/uploads/1630808595161-noauth.png
Hang Li
user
hanglics
Robust Multi-bit Text Watermark with LLM-based Paraphrasers
We propose an imperceptible multi-bit text watermark embedded by paraphrasing
with LLMs. We fine-tune a pair of LLM paraphrasers that are designed to behave
differently so that their paraphrasing difference reflected in the text
semantics can be identified by a trained decoder. To embed our multi-bit
watermark, we use two paraphrasers alternatively to encode the pre-defined
binary code at the sentence level. Then we use a text classifier as the decoder
to decode each bit of the watermark. Through extensive experiments, we show
that our watermarks can achieve over 99.99\% detection AUC with small (1.1B)
text paraphrasers while keeping the semantic information of the original
sentence. More importantly, our pipeline is robust under word substitution and
sentence paraphrasing perturbations and generalizes well to
out-of-distributional data. We also show the stealthiness of our watermark with
LLM-based evaluation. We open-source the code:
https://github.com/xiaojunxu/multi-bit-text-watermark.
Proposed method uses fine-tuned LLMs for sentence-level text watermarking, ensuring imperceptibility and robust detection.
LLMs paraphrasers text watermark binary code text classifier detection AUC word substitution sentence paraphrasing out-of-distributional data LLM-based evaluation