Can LLMs Learn by Teaching? A Preliminary Study
Teaching to improve student models (e.g., knowledge distillation) is an
extensively studied methodology in LLMs. However, for humans, teaching not only
improves students but also improves teachers. We ask: Can LLMs also learn by
teaching (LbT)? If yes, we can potentially unlock the possibility of
continuously advancing the models without solely relying on human-produced data
or stronger models. In this paper, we provide a preliminary exploration of this
ambitious agenda. We show that LbT ideas can be incorporated into existing LLM
training/prompting pipelines and provide noticeable improvements. Specifically,
we design three methods, each mimicking one of the three levels of LbT in
humans: observing students' feedback, learning from the feedback, and learning
iteratively, with the goals of improving answer accuracy without training and
improving models' inherent capability with fine-tuning. The findings are
encouraging. For example, similar to LbT in human, we see that: (1) LbT can
induce weak-to-strong generalization: strong models can improve themselves by
teaching other weak models; (2) Diversity in students might help: teaching
multiple students could be better than teaching one student or the teacher
itself. We hope that this early promise can inspire future research on LbT and
more broadly adopting the advanced techniques in education to improve LLMs. The
code is available at https://github.com/imagination-research/lbt.
667e2a17039ad1fcb8d2e79f
Xuefei Ning
claimed_verified
2024-06-28T08:02:47.626Z
641031b1a78453b8d96b8420
https://cdn-avatars.huggingface.co/v1/production/uploads/1678782881444-noauth.jpeg
Xuefei Ning
user
Foxfi
667e2a17039ad1fcb8d2e7a0
Zifu Wang
claimed_verified
2024-06-28T14:58:10.478Z
64f5c7cb65a4b1acb20ffc15
/avatars/67c42a6e26bbeddc4bf706eb8f1a75b1.svg
Zifu Wang
user
wangzifu
667e2a17039ad1fcb8d2e7a1
Shiyao Li
admin_assigned
2024-06-28T13:09:58.037Z
64cb1a9fb704b851478bcd5b
/avatars/b35ec1320adc058bff871f14971485ba.svg
Shiyao Li
user
LSY-noya
667e2a17039ad1fcb8d2e7a2
Zinan Lin
extracted_confirmed
2024-06-29T00:28:55.856Z
64c832a8c547ed5243d29630
https://cdn-avatars.huggingface.co/v1/production/uploads/64c832a8c547ed5243d29630/a46V0xOVRVknM9mUjKkTf.jpeg
Zinan Lin
user
fjxmlzn
667e2a17039ad1fcb8d2e7a3
Peiran Yao
admin_assigned
2024-06-28T13:10:03.887Z
64ed53351d42449431e61c60
/avatars/2d0c70d5a6d38a080eaa939acc9f0f09.svg
Peiran Yao
user
peiran-yao
667e2a17039ad1fcb8d2e7a4
Tianyu Fu
claimed_verified
2024-06-28T09:12:52.141Z
6445fd9ba56444c355dcbcba
https://cdn-avatars.huggingface.co/v1/production/uploads/6445fd9ba56444c355dcbcba/NCyRCD-MK-yA0_qY6I2y0.png
Tianyu Fu
user
fuvty
667e2a17039ad1fcb8d2e7a5
Matthew B. Blaschko
667e2a17039ad1fcb8d2e7a6
Guohao Dai
667e2a17039ad1fcb8d2e7a7
Huazhong Yang
667e2a17039ad1fcb8d2e7a8
Yu Wang
Can LLMs Learn by Teaching? A Preliminary Study
Teaching to improve student models (e.g., knowledge distillation) is an
extensively studied methodology in LLMs. However, for humans, teaching not only
improves students but also improves teachers. We ask: Can LLMs also learn by
teaching (LbT)? If yes, we can potentially unlock the possibility of
continuously advancing the models without solely relying on human-produced data
or stronger models. In this paper, we provide a preliminary exploration of this
ambitious agenda. We show that LbT ideas can be incorporated into existing LLM
training/prompting pipelines and provide noticeable improvements. Specifically,
we design three methods, each mimicking one of the three levels of LbT in
humans: observing students' feedback, learning from the feedback, and learning
iteratively, with the goals of improving answer accuracy without training and
improving models' inherent capability with fine-tuning. The findings are
encouraging. For example, similar to LbT in human, we see that: (1) LbT can
induce weak-to-strong generalization: strong models can improve themselves by
teaching other weak models; (2) Diversity in students might help: teaching
multiple students could be better than teaching one student or the teacher
itself. We hope that this early promise can inspire future research on LbT and
more broadly adopting the advanced techniques in education to improve LLMs. The
code is available at https://github.com/imagination-research/lbt.
LLMs can improve through a learning-by-teaching approach, potentially leading to self-improvement without relying solely on human data or more powerful models.
knowledge distillation LLMs learning by teaching LbT prompting pipelines weak-to-strong generalization diversity in students