ExaGPT: Example-Based Machine-Generated Text Detection for Human
  Interpretability
Detecting texts generated by Large Language Models (LLMs) could cause grave
mistakes due to incorrect decisions, such as undermining student's academic
dignity. LLM text detection thus needs to ensure the interpretability of the
decision, which can help users judge how reliably correct its prediction is.
When humans verify whether a text is human-written or LLM-generated, they
intuitively investigate with which of them it shares more similar spans.
However, existing interpretable detectors are not aligned with the human
decision-making process and fail to offer evidence that users easily
understand. To bridge this gap, we introduce ExaGPT, an interpretable detection
approach grounded in the human decision-making process for verifying the origin
of a text. ExaGPT identifies a text by checking whether it shares more similar
spans with human-written vs. with LLM-generated texts from a datastore. This
approach can provide similar span examples that contribute to the decision for
each span in the text as evidence. Our human evaluation demonstrates that
providing similar span examples contributes more effectively to judging the
correctness of the decision than existing interpretable methods. Moreover,
extensive experiments in four domains and three generators show that ExaGPT
massively outperforms prior powerful detectors by up to +40.9 points of
accuracy at a false positive rate of 1%.
67b52de36007d463b988b202
Ryuto Koike
claimed_verified
2025-02-19T09:03:41.013Z
6538e649f940c8a0358aa8b8
https://cdn-avatars.huggingface.co/v1/production/uploads/6538e649f940c8a0358aa8b8/veNw6QJuZu8anWCXtOXxu.jpeg
Ryuto Koike
user
ryuryukke
67b52de36007d463b988b203
Masahiro Kaneko
admin_assigned
2025-02-19T15:46:06.323Z
652bdb25756a15d750071787
https://cdn-avatars.huggingface.co/v1/production/uploads/652bdb25756a15d750071787/qWjkU0_1egP7XYX3sXoSZ.jpeg
Masahiro Kaneko
user
MasahiroKaneko
67b52de36007d463b988b204
Ayana Niwa
67b52de36007d463b988b205
Preslav Nakov
admin_assigned
2025-02-19T15:46:29.303Z
647f7eb25e1bc4753746bf9f
/avatars/cc9c6210fdc822d8a106937e747dff41.svg
Preslav Nakov
user
preslavnakov
67b52de36007d463b988b206
Naoaki Okazaki
admin_assigned
2025-02-19T15:46:36.241Z
630c0a9b09eceb8fafe89cc6
/avatars/6325a9b34da54d5cbddb814c3987a2fe.svg
Naoaki Okazaki
user
nokazaki
ExaGPT: Example-Based Machine-Generated Text Detection for Human
  Interpretability
Detecting texts generated by Large Language Models (LLMs) could cause grave
mistakes due to incorrect decisions, such as undermining student's academic
dignity. LLM text detection thus needs to ensure the interpretability of the
decision, which can help users judge how reliably correct its prediction is.
When humans verify whether a text is human-written or LLM-generated, they
intuitively investigate with which of them it shares more similar spans.
However, existing interpretable detectors are not aligned with the human
decision-making process and fail to offer evidence that users easily
understand. To bridge this gap, we introduce ExaGPT, an interpretable detection
approach grounded in the human decision-making process for verifying the origin
of a text. ExaGPT identifies a text by checking whether it shares more similar
spans with human-written vs. with LLM-generated texts from a datastore. This
approach can provide similar span examples that contribute to the decision for
each span in the text as evidence. Our human evaluation demonstrates that
providing similar span examples contributes more effectively to judging the
correctness of the decision than existing interpretable methods. Moreover,
extensive experiments in four domains and three generators show that ExaGPT
massively outperforms prior powerful detectors by up to +40.9 points of
accuracy at a false positive rate of 1%.
ExaGPT, an interpretable detection method grounded in human decision-making, identifies text origins by comparing similar spans and provides better judgment evidence and accuracy than existing methods.
Large Language Models LLM text detection interpretability human decision-making similar spans interpretable detectors false positive rate