Self-Reflective Generation at Test Time
Large language models (LLMs) increasingly solve complex reasoning tasks via
long chain-of-thought, but their forward-only autoregressive generation process
is fragile; early token errors can cascade, which creates a clear need for
self-reflection mechanisms. However, existing self-reflection either performs
revisions over full drafts or learns self-correction via expensive training,
both fundamentally reactive and inefficient. To address this, we propose
Self-Reflective Generation at Test Time (SRGen), a lightweight test-time
framework that reflects before generating at uncertain points. During token
generation, SRGen utilizes dynamic entropy thresholding to identify
high-uncertainty tokens. For each identified token, it trains a specific
corrective vector, which fully exploits the already generated context for a
self-reflective generation to correct the token probability distribution. By
retrospectively analyzing the partial output, this self-reflection enables more
trustworthy decisions, thereby significantly reducing the probability of errors
at highly uncertain points. Evaluated on challenging mathematical reasoning
benchmarks and a diverse set of LLMs, SRGen can consistently strengthen model
reasoning: improvements in single-pass quality also translate into stronger
self-consistency voting. Especially, on AIME2024 with
DeepSeek-R1-Distill-Qwen-7B, SRGen yields absolute improvements of +12.0% on
Pass@1 and +13.3% on Cons@5. Moreover, our findings position SRGen as a
plug-and-play method that integrates reflection into the generation process for
reliable LLM reasoning, achieving consistent gains with bounded overhead and
broad composability with other training-time (e.g., RLHF) and test-time (e.g.,
SLOT) techniques.
68e4aac5e4e093a7044e4dec
Jian Mu
claimed_verified
2025-10-10T09:38:26.889Z
6628b0621b4cd5f0ada35ed8
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/TppL-D7GtZPqUlUb6UU-K.png
mj
user
mujianijan
68e4aac5e4e093a7044e4ded
Qixin Zhang
68e4aac5e4e093a7044e4dee
Zhiyong Wang
68e4aac5e4e093a7044e4def
Menglin Yang
68e4aac5e4e093a7044e4df0
Shuang Qiu
68e4aac5e4e093a7044e4df1
Chengwei Qin
68e4aac5e4e093a7044e4df2
Zhongxiang Dai
68e4aac5e4e093a7044e4df3
Yao Shu
Self-Reflective Generation at Test Time
Large language models (LLMs) increasingly solve complex reasoning tasks via
long chain-of-thought, but their forward-only autoregressive generation process
is fragile; early token errors can cascade, which creates a clear need for
self-reflection mechanisms. However, existing self-reflection either performs
revisions over full drafts or learns self-correction via expensive training,
both fundamentally reactive and inefficient. To address this, we propose
Self-Reflective Generation at Test Time (SRGen), a lightweight test-time
framework that reflects before generating at uncertain points. During token
generation, SRGen utilizes dynamic entropy thresholding to identify
high-uncertainty tokens. For each identified token, it trains a specific
corrective vector, which fully exploits the already generated context for a
self-reflective generation to correct the token probability distribution. By
retrospectively analyzing the partial output, this self-reflection enables more
trustworthy decisions, thereby significantly reducing the probability of errors
at highly uncertain points. Evaluated on challenging mathematical reasoning
benchmarks and a diverse set of LLMs, SRGen can consistently strengthen model
reasoning: improvements in single-pass quality also translate into stronger
self-consistency voting. Especially, on AIME2024 with
DeepSeek-R1-Distill-Qwen-7B, SRGen yields absolute improvements of +12.0% on
Pass@1 and +13.3% on Cons@5. Moreover, our findings position SRGen as a
plug-and-play method that integrates reflection into the generation process for
reliable LLM reasoning, achieving consistent gains with bounded overhead and
broad composability with other training-time (e.g., RLHF) and test-time (e.g.,
SLOT) techniques.
SRGen, a lightweight test-time framework, improves LLM reasoning by dynamically identifying and correcting high-uncertainty tokens during generation, leading to better single-pass quality and self-consistency.
Self-Reflective Generation SRGen dynamic entropy thresholding corrective vector self-reflective generation token probability distribution self-consistency voting AIME2024 DeepSeek-R1-Distill-Qwen-7B Pass@1 Cons@5 RLHF SLOT