I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative
  Self-Enhancement Paradigm
Large Language Models (LLMs) have achieved significant advancements, however,
the common learning paradigm treats LLMs as passive information repositories,
neglecting their potential for active learning and alignment. Some approaches
train LLMs using their own generated synthetic data, exploring the possibility
of active alignment. However, there is still a huge gap between these one-time
alignment methods and the continuous automatic alignment of humans. In this
paper, we introduce I-SHEEP, an Iterative
Self-EnHancEmEnt Paradigm.This
human-like paradigm enables LLMs to continuously self-align from
scratch with nothing. Compared to the one-time alignment method Dromedary
sun2023principledriven, which refers to the first iteration in this
paper, I-SHEEP can significantly enhance capacities on both Qwen and Llama
models. I-SHEEP achieves a maximum relative improvement of 78.2\% in the Alpaca
Eval, 24.0\% in the MT Bench, and an absolute increase of 8.88\% in the IFEval
accuracy over subsequent iterations in Qwen-1.5 72B model. Additionally,
I-SHEEP surpasses the base model in various standard benchmark generation
tasks, achieving an average improvement of 24.77\% in code generation tasks,
12.04\% in TrivialQA, and 20.29\% in SQuAD. We also provide new insights based
on the experiment results. Our codes, datasets, and models are available at
https://anonymous.4open.science/r/I-SHEEP.
66bedfa3ac74db25de04bd12
Yiming Liang
admin_assigned
2024-08-16T09:20:26.552Z
6555e8d8a0c34cd61a6b9ce3
/avatars/71dc562cef4bd42f6b762f036357c800.svg
yimingliang
user
yimingliang
66bedfa3ac74db25de04bd13
Ge Zhang
extracted_confirmed
2024-08-21T07:16:48.215Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
66bedfa3ac74db25de04bd14
Xingwei Qu
66bedfa3ac74db25de04bd15
Tianyu Zheng
claimed_verified
2024-10-21T07:53:45.446Z
64ab99dcb76bfd863eba64c1
https://cdn-avatars.huggingface.co/v1/production/uploads/64ab99dcb76bfd863eba64c1/UBXwDPx17X-gl-SzBPvrc.jpeg
TY.Zheng
user
aaabiao
66bedfa3ac74db25de04bd16
Jiawei Guo
66bedfa3ac74db25de04bd17
Xinrun Du
claimed_verified
2024-11-03T14:24:53.513Z
654907a4a1faff97850c4eff
/avatars/458c90151614bc7f116943b6e67d6b8a.svg
du
user
dododododo
66bedfa3ac74db25de04bd18
Zhenzhu Yang
66bedfa3ac74db25de04bd19
Jiaheng Liu
claimed_verified
2025-01-05T23:03:36.525Z
65377c30e48353201e6fdda0
/avatars/a8f803b6f2e598eaee9c52c0d2ddfc16.svg
Jiaheng Liu
user
CheeryLJH
66bedfa3ac74db25de04bd1a
Chenghua Lin
66bedfa3ac74db25de04bd1b
Lei Ma
66bedfa3ac74db25de04bd1c
Wenhao Huang
extracted_pending
2024-08-16T05:12:04.738Z
635c855dbb96f709230e8acd
/avatars/8b580863425f6d5cfa9ce833eb4c6d34.svg
Rubio Huang
user
Rubio41
66bedfa3ac74db25de04bd1d
Jiajun Zhang
I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative
  Self-Enhancement Paradigm
Large Language Models (LLMs) have achieved significant advancements, however,
the common learning paradigm treats LLMs as passive information repositories,
neglecting their potential for active learning and alignment. Some approaches
train LLMs using their own generated synthetic data, exploring the possibility
of active alignment. However, there is still a huge gap between these one-time
alignment methods and the continuous automatic alignment of humans. In this
paper, we introduce I-SHEEP, an Iterative
Self-EnHancEmEnt Paradigm.This
human-like paradigm enables LLMs to continuously self-align from
scratch with nothing. Compared to the one-time alignment method Dromedary
sun2023principledriven, which refers to the first iteration in this
paper, I-SHEEP can significantly enhance capacities on both Qwen and Llama
models. I-SHEEP achieves a maximum relative improvement of 78.2\% in the Alpaca
Eval, 24.0\% in the MT Bench, and an absolute increase of 8.88\% in the IFEval
accuracy over subsequent iterations in Qwen-1.5 72B model. Additionally,
I-SHEEP surpasses the base model in various standard benchmark generation
tasks, achieving an average improvement of 24.77\% in code generation tasks,
12.04\% in TrivialQA, and 20.29\% in SQuAD. We also provide new insights based
on the experiment results. Our codes, datasets, and models are available at
https://anonymous.4open.science/r/I-SHEEP.
I-SHEEP, an iterative self-enhancement paradigm, enables LLMs to continuously self-align from scratch, achieving significant improvements across various benchmarks compared to one-time alignment methods.
I-SHEEP active learning alignment LLMs synthetic data continuous self-align Qwen Llama Alpaca Eval MT Bench IFEval code generation TrivialQA SQuAD