Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal
  LLMs
Can humans identify AI-generated (fake) videos and provide grounded reasons?
While video generation models have advanced rapidly, a critical dimension --
whether humans can detect deepfake traces within a generated video, i.e.,
spatiotemporal grounded visual artifacts that reveal a video as machine
generated -- has been largely overlooked. We introduce DeeptraceReward, the
first fine-grained, spatially- and temporally- aware benchmark that annotates
human-perceived fake traces for video generation reward. The dataset comprises
4.3K detailed annotations across 3.3K high-quality generated videos. Each
annotation provides a natural-language explanation, pinpoints a bounding-box
region containing the perceived trace, and marks precise onset and offset
timestamps. We consolidate these annotations into 9 major categories of
deepfake traces that lead humans to identify a video as AI-generated, and train
multimodal language models (LMs) as reward models to mimic human judgments and
localizations. On DeeptraceReward, our 7B reward model outperforms GPT-5 by
34.7% on average across fake clue identification, grounding, and explanation.
Interestingly, we observe a consistent difficulty gradient: binary fake v.s.
real classification is substantially easier than fine-grained deepfake trace
detection; within the latter, performance degrades from natural language
explanations (easiest), to spatial grounding, to temporal labeling (hardest).
By foregrounding human-perceived deepfake traces, DeeptraceReward provides a
rigorous testbed and training signal for socially aware and trustworthy video
generation.
68dcb64b4159d1f2418f9bf5
Xingyu Fu
68dcb64b4159d1f2418f9bf6
Siyi Liu
68dcb64b4159d1f2418f9bf7
Yinuo Xu
68dcb64b4159d1f2418f9bf8
Pan Lu
claimed_verified
2025-10-09T06:52:20.597Z
60f5f68fa7fd83d025749234
https://cdn-avatars.huggingface.co/v1/production/uploads/60f5f68fa7fd83d025749234/gCeJAZfzaANAcEvI6v5-P.jpeg
Pan Lu
user
lupantech
68dcb64b4159d1f2418f9bf9
Guangqiuse Hu
68dcb64b4159d1f2418f9bfa
Tianbo Yang
68dcb64b4159d1f2418f9bfb
Taran Anantasagar
68dcb64b4159d1f2418f9bfc
Christopher Shen
68dcb64b4159d1f2418f9bfd
Yikai Mao
68dcb64b4159d1f2418f9bfe
Yuanzhe Liu
68dcb64b4159d1f2418f9bff
Keyush Shah
68dcb64b4159d1f2418f9c00
Chung Un Lee
68dcb64b4159d1f2418f9c01
Yejin Choi
68dcb64b4159d1f2418f9c02
James Zou
68dcb64b4159d1f2418f9c03
Dan Roth
68dcb64b4159d1f2418f9c04
Chris Callison-Burch
Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal
  LLMs
Can humans identify AI-generated (fake) videos and provide grounded reasons?
While video generation models have advanced rapidly, a critical dimension --
whether humans can detect deepfake traces within a generated video, i.e.,
spatiotemporal grounded visual artifacts that reveal a video as machine
generated -- has been largely overlooked. We introduce DeeptraceReward, the
first fine-grained, spatially- and temporally- aware benchmark that annotates
human-perceived fake traces for video generation reward. The dataset comprises
4.3K detailed annotations across 3.3K high-quality generated videos. Each
annotation provides a natural-language explanation, pinpoints a bounding-box
region containing the perceived trace, and marks precise onset and offset
timestamps. We consolidate these annotations into 9 major categories of
deepfake traces that lead humans to identify a video as AI-generated, and train
multimodal language models (LMs) as reward models to mimic human judgments and
localizations. On DeeptraceReward, our 7B reward model outperforms GPT-5 by
34.7% on average across fake clue identification, grounding, and explanation.
Interestingly, we observe a consistent difficulty gradient: binary fake v.s.
real classification is substantially easier than fine-grained deepfake trace
detection; within the latter, performance degrades from natural language
explanations (easiest), to spatial grounding, to temporal labeling (hardest).
By foregrounding human-perceived deepfake traces, DeeptraceReward provides a
rigorous testbed and training signal for socially aware and trustworthy video
generation.
DeeptraceReward is a benchmark dataset that annotates human-perceived deepfake traces in videos, used to train multimodal language models for detecting AI-generated videos.
deepfake traces spatiotemporal grounded visual artifacts fine-grained spatially- and temporally- aware benchmark multimodal language models reward models binary fake v.s. real classification natural language explanations spatial grounding temporal labeling