Information-Preserving Reformulation of Reasoning Traces for
  Antidistillation
Recent advances in Large Language Models (LLMs) show that extending the
length of reasoning chains significantly improves performance on complex tasks.
While revealing these reasoning traces helps users better follow, verify, and
learn from the model's problem-solving process, it also makes them highly
vulnerable to unauthorized distillation. To mitigate this risk, proprietary
model providers often adopt aggressive protection strategies, such as replacing
detailed reasoning with brief summaries, which deprive users of valuable
intermediate information. To address this trade-off, we propose PART, an
information-preserving antidistillation reformulation of reasoning traces.
Motivated by the difference between how humans understand reasoning traces and
how LLMs exploit them for supervised fine-tuning, we design a simple but
effective two-step reformulation: removing self-talk behaviors and reordering
sub-conclusions. A small auxiliary model is trained to perform this
reformulation, incurring minimal computational overhead. Extensive experiments
demonstrate that PART consistently disrupts distillation across student models
of different sizes and types on various reasoning benchmarks. For instance,
when training on reformulated traces, even the performance of a large 32B
student model decreases from 54.17 to 46.88 on AIME 2024, corresponding to a
13.5% degradation.
68ef4a3a486b78128f0e35da
Jiayu Ding
68ef4a3a486b78128f0e35db
Lei Cui
claimed_verified
2025-10-15T15:24:43.386Z
5f5a4fbe10b2753d9000c7fc
/avatars/0763b25c13733ef4eb757862ba6d603e.svg
Lei Cui
user
wolfshow
68ef4a3a486b78128f0e35dc
Li Dong
68ef4a3a486b78128f0e35dd
Nanning Zheng
68ef4a3a486b78128f0e35de
Furu Wei
Information-Preserving Reformulation of Reasoning Traces for
  Antidistillation
Recent advances in Large Language Models (LLMs) show that extending the
length of reasoning chains significantly improves performance on complex tasks.
While revealing these reasoning traces helps users better follow, verify, and
learn from the model's problem-solving process, it also makes them highly
vulnerable to unauthorized distillation. To mitigate this risk, proprietary
model providers often adopt aggressive protection strategies, such as replacing
detailed reasoning with brief summaries, which deprive users of valuable
intermediate information. To address this trade-off, we propose PART, an
information-preserving antidistillation reformulation of reasoning traces.
Motivated by the difference between how humans understand reasoning traces and
how LLMs exploit them for supervised fine-tuning, we design a simple but
effective two-step reformulation: removing self-talk behaviors and reordering
sub-conclusions. A small auxiliary model is trained to perform this
reformulation, incurring minimal computational overhead. Extensive experiments
demonstrate that PART consistently disrupts distillation across student models
of different sizes and types on various reasoning benchmarks. For instance,
when training on reformulated traces, even the performance of a large 32B
student model decreases from 54.17 to 46.88 on AIME 2024, corresponding to a
13.5% degradation.
PART reformulates reasoning traces to preserve information while disrupting unauthorized distillation in Large Language Models.
Large Language Models reasoning chains reasoning traces unauthorized distillation information-preserving antidistillation self-talk behaviors sub-conclusions auxiliary model reasoning benchmarks AIME 2024