DocReward: A Document Reward Model for Structuring and Stylizing
Recent advances in agentic workflows have enabled the automation of tasks
such as professional document generation. However, they primarily focus on
textual quality, neglecting visual structure and style, which are crucial for
readability and engagement. This gap arises mainly from the absence of suitable
reward models to guide agentic workflows toward producing documents with
stronger structural and stylistic quality. To address this, we propose
DocReward, a document reward model that evaluates documents based on their
structure and style. We construct a multi-domain dataset DocPair of 117K paired
documents, covering 32 domains and 267 document types, each including a high-
and low-professionalism document with identical content but different structure
and style. This enables the model to evaluate professionalism comprehensively,
and in a textual-quality-agnostic way. DocReward is trained using the
Bradley-Terry loss to score documents, penalizing predictions that contradict
the annotated ranking. To assess the performance of reward models, we create a
test dataset containing document bundles ranked by well-educated human
evaluators. Notably, DocReward outperforms GPT-4o and GPT-5 in accuracy by 30.6
and 19.4 percentage points, respectively, demonstrating its superiority over
baselines. In an extrinsic evaluation of document generation, DocReward
achieves a significantly higher win rate of 60.8%, compared to GPT-5's 37.7%
win rate, demonstrating its utility in guiding generation agents toward
producing human-preferred documents.
68edb30ade1fee572713a76d
Junpeng Liu
claimed_verified
2025-10-14T07:30:44.805Z
6512a2e284bedffb5ac5a511
/avatars/e798f2c8633d77aea41fce684d749390.svg
Junpeng Liu
user
jeepliu
68edb30ade1fee572713a76e
Yuzhong Zhao
68edb30ade1fee572713a76f
Bowen Cao
68edb30ade1fee572713a770
Jiayu Ding
claimed_verified
2025-10-15T07:07:56.602Z
667d9460a5d952954ce9f1c4
/avatars/340b2503acad0cd692f46e21b9358bd0.svg
Jiayu Ding
user
JiayuDing
68edb30ade1fee572713a771
Yilin Jia
68edb30ade1fee572713a772
Tengchao Lv
68edb30ade1fee572713a773
Yupan Huang
68edb30ade1fee572713a774
Shaohan Huang
68edb30ade1fee572713a775
Nan Yang
68edb30ade1fee572713a776
Li Dong
68edb30ade1fee572713a777
Lei Cui
claimed_verified
2025-10-15T15:25:23.023Z
5f5a4fbe10b2753d9000c7fc
/avatars/0763b25c13733ef4eb757862ba6d603e.svg
Lei Cui
user
wolfshow
68edb30ade1fee572713a778
Tao Ge
68edb30ade1fee572713a779
Xun Wang
68edb30ade1fee572713a77a
Huitian Jiao
68edb30ade1fee572713a77b
Sun Mao
68edb30ade1fee572713a77c
FNU Kartik
68edb30ade1fee572713a77d
Si-Qing Chen
68edb30ade1fee572713a77e
Wai Lam
68edb30ade1fee572713a77f
Furu Wei
DocReward: A Document Reward Model for Structuring and Stylizing
Recent advances in agentic workflows have enabled the automation of tasks
such as professional document generation. However, they primarily focus on
textual quality, neglecting visual structure and style, which are crucial for
readability and engagement. This gap arises mainly from the absence of suitable
reward models to guide agentic workflows toward producing documents with
stronger structural and stylistic quality. To address this, we propose
DocReward, a document reward model that evaluates documents based on their
structure and style. We construct a multi-domain dataset DocPair of 117K paired
documents, covering 32 domains and 267 document types, each including a high-
and low-professionalism document with identical content but different structure
and style. This enables the model to evaluate professionalism comprehensively,
and in a textual-quality-agnostic way. DocReward is trained using the
Bradley-Terry loss to score documents, penalizing predictions that contradict
the annotated ranking. To assess the performance of reward models, we create a
test dataset containing document bundles ranked by well-educated human
evaluators. Notably, DocReward outperforms GPT-4o and GPT-5 in accuracy by 30.6
and 19.4 percentage points, respectively, demonstrating its superiority over
baselines. In an extrinsic evaluation of document generation, DocReward
achieves a significantly higher win rate of 60.8%, compared to GPT-5's 37.7%
win rate, demonstrating its utility in guiding generation agents toward
producing human-preferred documents.
DocReward, a document reward model, evaluates and enhances the structural and stylistic quality of generated documents, outperforming GPT-4o and GPT-5 in both accuracy and human-preferred document generation.
DocReward Bradley-Terry loss document reward model document generation extrinsic evaluation