Minimum Tuning to Unlock Long Output from LLMs with High Quality Data as
  the Key
As large language models rapidly evolve to support longer context, there is a
notable disparity in their capability to generate output at greater lengths.
Recent study suggests that the primary cause for this imbalance may arise from
the lack of data with long-output during alignment training. In light of this
observation, attempts are made to re-align foundation models with data that
fills the gap, which result in models capable of generating lengthy output when
instructed. In this paper, we explore the impact of data-quality in tuning a
model for long output, and the possibility of doing so from the starting points
of human-aligned (instruct or chat) models. With careful data curation, we show
that it possible to achieve similar performance improvement in our tuned
models, with only a small fraction of training data instances and compute. In
addition, we assess the generalizability of such approaches by applying our
tuning-recipes to several models. our findings suggest that, while capacities
for generating long output vary across different models out-of-the-box, our
approach to tune them with high-quality data using lite compute, consistently
yields notable improvement across all models we experimented on. We have made
public our curated dataset for tuning long-writing capability, the
implementations of model tuning and evaluation, as well as the fine-tuned
models, all of which can be openly-accessed.
67122a1054c8572e476f32a3
Yingda Chen
claimed_verified
2024-10-18T09:54:53.711Z
6215ca5692c0ecfba9186921
https://cdn-avatars.huggingface.co/v1/production/uploads/1645595695434-6215ca5692c0ecfba9186921.png
Yingda Chen
user
Yingda
67122a1054c8572e476f32a4
Xingjun Wang
claimed_verified
2024-10-18T09:54:56.405Z
62f612e22e53c2efd33943c6
https://cdn-avatars.huggingface.co/v1/production/uploads/1679249371815-62f612e22e53c2efd33943c6.jpeg
xingjun wang
user
wangxingjun778
67122a1054c8572e476f32a5
Jintao Huang
admin_assigned
2024-10-18T13:03:40.531Z
6624e6b111772517e5db916c
/avatars/cd7f14216e63689a3ede587224e35ead.svg
Jintao Huang
user
feifeiyu-hjt
67122a1054c8572e476f32a6
Yunlin Mao
claimed_verified
2024-12-02T09:40:59.105Z
64b8c458d6c468ac7525e473
/avatars/b3000da1d230387fa0e450f098803ab4.svg
Lin
user
Yunxz
67122a1054c8572e476f32a7
Daoze Zhang
claimed_verified
2024-10-18T09:54:59.082Z
664dabaade412305ff2c984e
/avatars/9dd55d8078f0534f2bdb8cfb84333b8e.svg
Zhang
user
Daoze
67122a1054c8572e476f32a8
Yuze Zhao
Minimum Tuning to Unlock Long Output from LLMs with High Quality Data as
  the Key
As large language models rapidly evolve to support longer context, there is a
notable disparity in their capability to generate output at greater lengths.
Recent study suggests that the primary cause for this imbalance may arise from
the lack of data with long-output during alignment training. In light of this
observation, attempts are made to re-align foundation models with data that
fills the gap, which result in models capable of generating lengthy output when
instructed. In this paper, we explore the impact of data-quality in tuning a
model for long output, and the possibility of doing so from the starting points
of human-aligned (instruct or chat) models. With careful data curation, we show
that it possible to achieve similar performance improvement in our tuned
models, with only a small fraction of training data instances and compute. In
addition, we assess the generalizability of such approaches by applying our
tuning-recipes to several models. our findings suggest that, while capacities
for generating long output vary across different models out-of-the-box, our
approach to tune them with high-quality data using lite compute, consistently
yields notable improvement across all models we experimented on. We have made
public our curated dataset for tuning long-writing capability, the
implementations of model tuning and evaluation, as well as the fine-tuned
models, all of which can be openly-accessed.
Data quality significantly impacts performance when tuning models for generating long outputs, and high-quality datasets with minimal compute can improve long-output generation across various models.
data-quality long output alignment training human-aligned models model tuning generalizability fine-tuning long-writing capability