Towards Building the Federated GPT: Federated Instruction Tuning
While ``instruction-tuned" generative large language models (LLMs) have
demonstrated an impressive ability to generalize to new tasks, the training
phases heavily rely on large amounts of diverse and high-quality instruction
data (such as ChatGPT and GPT-4). Unfortunately, acquiring high-quality data,
especially when it comes to human-written data, can pose significant challenges
both in terms of cost and accessibility. Moreover, concerns related to privacy
can further limit access to such data, making the process of obtaining it a
complex and nuanced undertaking. Consequently, this hinders the generality of
the tuned models and may restrict their effectiveness in certain contexts. To
tackle this issue, our study introduces a new approach called Federated
Instruction Tuning (FedIT), which leverages federated learning (FL) as the
learning framework for the instruction tuning of LLMs. This marks the first
exploration of FL-based instruction tuning for LLMs. This is especially
important since text data is predominantly generated by end users. Therefore,
it is imperative to design and adapt FL approaches to effectively leverage
these users' diverse instructions stored on local devices, while preserving
privacy and ensuring data security. In the current paper, by conducting widely
used GPT-4 auto-evaluation, we demonstrate that by exploiting the heterogeneous
and diverse sets of instructions on the client's end with the proposed
framework FedIT, we improved the performance of LLMs compared to centralized
training with only limited local instructions. Further, in this paper, we
developed a Github repository named Shepherd. This repository offers a
foundational framework for exploring federated fine-tuning of LLMs using
heterogeneous instructions across diverse categories.
645aedca1a242120814700ab
Jianyi Zhang
645aedca1a242120814700ac
Saeed Vahidian
645aedca1a242120814700ad
Martin Kuo
645aedca1a242120814700ae
Chunyuan Li
admin_assigned
2023-06-02T14:27:42.468Z
62aba526cae4462c0c6caa0f
/avatars/430560ec2c2547f819225769ab432f30.svg
Chunyuan Li
user
Chunyuan24
645aedca1a242120814700af
Ruiyi Zhang
admin_assigned
2023-07-03T09:45:35.776Z
64668f982da1abc242355cbb
/avatars/c9a248b8d70b1a8f7b78265c98690570.svg
Ruiyi Zhang
user
zhangry868
645aedca1a242120814700b0
Guoyin Wang
claimed_verified
2024-12-03T11:18:17.737Z
6490d4ba1afdee3acd1147f6
/avatars/ae13c7b21fe9ced7541dcd664d1b94ed.svg
Guoyin Wang
user
guoyinwang
645aedca1a242120814700b1
Yiran Chen
Towards Building the Federated GPT: Federated Instruction Tuning
While ``instruction-tuned" generative large language models (LLMs) have
demonstrated an impressive ability to generalize to new tasks, the training
phases heavily rely on large amounts of diverse and high-quality instruction
data (such as ChatGPT and GPT-4). Unfortunately, acquiring high-quality data,
especially when it comes to human-written data, can pose significant challenges
both in terms of cost and accessibility. Moreover, concerns related to privacy
can further limit access to such data, making the process of obtaining it a
complex and nuanced undertaking. Consequently, this hinders the generality of
the tuned models and may restrict their effectiveness in certain contexts. To
tackle this issue, our study introduces a new approach called Federated
Instruction Tuning (FedIT), which leverages federated learning (FL) as the
learning framework for the instruction tuning of LLMs. This marks the first
exploration of FL-based instruction tuning for LLMs. This is especially
important since text data is predominantly generated by end users. Therefore,
it is imperative to design and adapt FL approaches to effectively leverage
these users' diverse instructions stored on local devices, while preserving
privacy and ensuring data security. In the current paper, by conducting widely
used GPT-4 auto-evaluation, we demonstrate that by exploiting the heterogeneous
and diverse sets of instructions on the client's end with the proposed
framework FedIT, we improved the performance of LLMs compared to centralized
training with only limited local instructions. Further, in this paper, we
developed a Github repository named Shepherd. This repository offers a
foundational framework for exploring federated fine-tuning of LLMs using
heterogeneous instructions across diverse categories.
Federated Instruction Tuning (FedIT) leverages federated learning to improve generative large language models with diverse local instructions while maintaining privacy.
generative large language models (LLMs) instruction-tuned federated learning (FL) Federated Instruction Tuning (FedIT) GPT-4 auto-evaluation heterogeneous client's end federated fine-tuning