PositionID: LLMs can Control Lengths, Copy and Paste with Explicit
  Positional Awareness
Large Language Models (LLMs) demonstrate impressive capabilities across
various domains, including role-playing, creative writing, mathematical
reasoning, and coding. Despite these advancements, LLMs still encounter
challenges with length control, frequently failing to adhere to specific length
constraints due to their token-level operations and insufficient training on
data with strict length limitations. We identify this issue as stemming from a
lack of positional awareness and propose novel approaches--PositionID Prompting
and PositionID Fine-Tuning--to address it. These methods enhance the model's
ability to continuously monitor and manage text length during generation.
Additionally, we introduce PositionID CP Prompting to enable LLMs to perform
copy and paste operations accurately. Furthermore, we develop two benchmarks
for evaluating length control and copy-paste abilities. Our experiments
demonstrate that our methods significantly improve the model's adherence to
length constraints and copy-paste accuracy without compromising response
quality.
6707d46de2deb345cb3bda89
Zekun Wang
6707d46de2deb345cb3bda8a
Feiyu Duan
6707d46de2deb345cb3bda8b
Yibo Zhang
6707d46de2deb345cb3bda8c
Wangchunshu Zhou
admin_assigned
2024-10-14T11:16:28.376Z
628c8598ef14f971b698107f
/avatars/3a4ad87e6b5f9e836a1160d869df1447.svg
Zhou
user
Wangchunshu
6707d46de2deb345cb3bda8d
Ke Xu
6707d46de2deb345cb3bda8e
Wenhao Huang
claimed_verified
2024-10-13T20:16:29.003Z
65bb11cb00a03997849e9e85
/avatars/17022b0254192a837f4fe00d84389cda.svg
Wenhao Huang
user
StephenHuang
6707d46de2deb345cb3bda8f
Jie Fu
PositionID: LLMs can Control Lengths, Copy and Paste with Explicit
  Positional Awareness
Large Language Models (LLMs) demonstrate impressive capabilities across
various domains, including role-playing, creative writing, mathematical
reasoning, and coding. Despite these advancements, LLMs still encounter
challenges with length control, frequently failing to adhere to specific length
constraints due to their token-level operations and insufficient training on
data with strict length limitations. We identify this issue as stemming from a
lack of positional awareness and propose novel approaches--PositionID Prompting
and PositionID Fine-Tuning--to address it. These methods enhance the model's
ability to continuously monitor and manage text length during generation.
Additionally, we introduce PositionID CP Prompting to enable LLMs to perform
copy and paste operations accurately. Furthermore, we develop two benchmarks
for evaluating length control and copy-paste abilities. Our experiments
demonstrate that our methods significantly improve the model's adherence to
length constraints and copy-paste accuracy without compromising response
quality.
PositionID Prompting, PositionID Fine-Tuning, and PositionID CP Prompting improve length control and copy-paste accuracy in LLMs without degrading response quality.
Large Language Models (LLMs) PositionID Prompting PositionID Fine-Tuning PositionID CP Prompting