WritingBench: A Comprehensive Benchmark for Generative Writing
Recent advancements in large language models (LLMs) have significantly
enhanced text generation capabilities, yet evaluating their performance in
generative writing remains a challenge. Existing benchmarks primarily focus on
generic text generation or limited in writing tasks, failing to capture the
diverse requirements of high-quality written contents across various domains.
To bridge this gap, we present WritingBench, a comprehensive benchmark designed
to evaluate LLMs across 6 core writing domains and 100 subdomains, encompassing
creative, persuasive, informative, and technical writing. We further propose a
query-dependent evaluation framework that empowers LLMs to dynamically generate
instance-specific assessment criteria. This framework is complemented by a
fine-tuned critic model for criteria-aware scoring, enabling evaluations in
style, format and length. The framework's validity is further demonstrated by
its data curation capability, which enables 7B-parameter models to approach
state-of-the-art (SOTA) performance. We open-source the benchmark, along with
evaluation tools and modular framework components, to advance the development
of LLMs in writing.
67cfebe18a4265f3656a50aa
Yuning Wu
claimed_verified
2025-03-11T08:24:29.900Z
642d430a7f9efee76b8713c0
/avatars/4981f166a6df8e2ea60cd4c41c2f44d4.svg
YuningWu
user
AQuarterMile
67cfebe18a4265f3656a50ab
Jiahao Mei
67cfebe18a4265f3656a50ac
Ming Yan
67cfebe18a4265f3656a50ad
Chenliang Li
67cfebe18a4265f3656a50ae
SHaopeng Lai
67cfebe18a4265f3656a50af
Yuran Ren
67cfebe18a4265f3656a50b0
Zijia Wang
67cfebe18a4265f3656a50b1
Ji Zhang
67cfebe18a4265f3656a50b2
Mengyue Wu
67cfebe18a4265f3656a50b3
Qin Jin
67cfebe18a4265f3656a50b4
Fei Huang
WritingBench: A Comprehensive Benchmark for Generative Writing
Recent advancements in large language models (LLMs) have significantly
enhanced text generation capabilities, yet evaluating their performance in
generative writing remains a challenge. Existing benchmarks primarily focus on
generic text generation or limited in writing tasks, failing to capture the
diverse requirements of high-quality written contents across various domains.
To bridge this gap, we present WritingBench, a comprehensive benchmark designed
to evaluate LLMs across 6 core writing domains and 100 subdomains, encompassing
creative, persuasive, informative, and technical writing. We further propose a
query-dependent evaluation framework that empowers LLMs to dynamically generate
instance-specific assessment criteria. This framework is complemented by a
fine-tuned critic model for criteria-aware scoring, enabling evaluations in
style, format and length. The framework's validity is further demonstrated by
its data curation capability, which enables 7B-parameter models to approach
state-of-the-art (SOTA) performance. We open-source the benchmark, along with
evaluation tools and modular framework components, to advance the development
of LLMs in writing.
WritingBench is a comprehensive benchmark that evaluates large language models across diverse writing domains using a query-dependent framework and a fine-tuned critic model for criteria-aware scoring.
large language models WritingBench generative writing query-dependent evaluation fine-tuned critic model state-of-the-art