Constraint Back-translation Improves Complex Instruction Following of
  Large Language Models
Large language models (LLMs) struggle to follow instructions with complex
constraints in format, length, etc. Following the conventional
instruction-tuning practice, previous works conduct post-training on complex
instruction-response pairs generated by feeding complex instructions to
advanced LLMs. However, even advanced LLMs cannot follow complex instructions
well, thus limiting the quality of generated data. In this work, we find that
existing datasets inherently contain implicit complex constraints and propose a
novel data generation technique, constraint back-translation. Specifically, we
take the high-quality instruction-response pairs in existing datasets and only
adopt advanced LLMs to add complex constraints already met by the responses to
the instructions, which naturally reduces costs and data noise. In the
experiments, we adopt Llama3-70B-Instruct to back-translate constraints and
create a high-quality complex instruction-response dataset, named CRAB. We
present that post-training on CRAB improves multiple backbone LLMs' complex
instruction-following ability, evaluated on extensive instruction-following
benchmarks. We further find that constraint back-translation also serves as a
useful auxiliary training objective in post-training. Our code, data, and
models will be released to facilitate future research.
672442ec7b7aa5555e85b437
Yunjia Qi
claimed_verified
2024-11-03T14:22:11.267Z
6556cf2cee35f7d8bcf13bb3
/avatars/0a945054cb4732c2ee7a5502c42628bf.svg
Qi Yunjia
user
Kikkk
672442ec7b7aa5555e85b438
Hao Peng
claimed_verified
2025-03-02T20:19:32.600Z
625a5446f1063e7085d5178a
/avatars/5e78186f13f74b14e01583e06ff6c4dc.svg
Hao Peng
user
Wesleythu
672442ec7b7aa5555e85b439
Xiaozhi Wang
672442ec7b7aa5555e85b43a
Bin Xu
672442ec7b7aa5555e85b43b
Lei Hou
672442ec7b7aa5555e85b43c
Juanzi Li
Constraint Back-translation Improves Complex Instruction Following of
  Large Language Models
Large language models (LLMs) struggle to follow instructions with complex
constraints in format, length, etc. Following the conventional
instruction-tuning practice, previous works conduct post-training on complex
instruction-response pairs generated by feeding complex instructions to
advanced LLMs. However, even advanced LLMs cannot follow complex instructions
well, thus limiting the quality of generated data. In this work, we find that
existing datasets inherently contain implicit complex constraints and propose a
novel data generation technique, constraint back-translation. Specifically, we
take the high-quality instruction-response pairs in existing datasets and only
adopt advanced LLMs to add complex constraints already met by the responses to
the instructions, which naturally reduces costs and data noise. In the
experiments, we adopt Llama3-70B-Instruct to back-translate constraints and
create a high-quality complex instruction-response dataset, named CRAB. We
present that post-training on CRAB improves multiple backbone LLMs' complex
instruction-following ability, evaluated on extensive instruction-following
benchmarks. We further find that constraint back-translation also serves as a
useful auxiliary training objective in post-training. Our code, data, and
models will be released to facilitate future research.
Constraint back-translation enhances large language models' ability to follow complex instructions by leveraging implicit constraints in existing datasets, improving their performance on instruction-following benchmarks.
LLMs instruction-tuning instruction-response pairs constraint back-translation high-quality instruction-response dataset Llama3-70B-Instruct post-training instruction-following benchmarks auxiliary training objective