Learning to Retrieve In-Context Examples for Large Language Models
Large language models (LLMs) have demonstrated their ability to learn
in-context, allowing them to perform various tasks based on a few input-output
examples. However, the effectiveness of in-context learning is heavily reliant
on the quality of the selected examples. In this paper, we propose a novel
framework to iteratively train dense retrievers that can identify high-quality
in-context examples for LLMs. Our framework initially trains a reward model
based on LLM feedback to evaluate the quality of candidate examples, followed
by knowledge distillation to train a bi-encoder based dense retriever. Our
experiments on a suite of 30 tasks demonstrate that our framework significantly
enhances in-context learning performance. Furthermore, we show the
generalization ability of our framework to unseen tasks during training. An
in-depth analysis reveals that our model improves performance by retrieving
examples with similar patterns, and the gains are consistent across LLMs of
varying sizes.
64b4aa31966b28317e594464
Liang Wang
extracted_confirmed
2023-07-17T02:47:46.883Z
62343594c63d91cec1ca4a8d
/avatars/5a1ee74c2dbe349a6ec9843a1599d281.svg
Liang Wang
user
intfloat
64b4aa31966b28317e594465
Nan Yang
claimed_verified
2024-05-13T07:51:13.112Z
6341342a9948f573f36d77da
/avatars/813717cb180b2ce566421b43dadabb43.svg
nyanyanya
user
nyanyanya
64b4aa31966b28317e594466
Furu Wei
admin_assigned
2023-07-17T10:39:39.872Z
6368c512fbfe97c16a40baba
/avatars/1c23bc7c0b6d9225699ce27647623d7a.svg
Furu Wei
user
thegenerality
Learning to Retrieve In-Context Examples for Large Language Models
Large language models (LLMs) have demonstrated their ability to learn
in-context, allowing them to perform various tasks based on a few input-output
examples. However, the effectiveness of in-context learning is heavily reliant
on the quality of the selected examples. In this paper, we propose a novel
framework to iteratively train dense retrievers that can identify high-quality
in-context examples for LLMs. Our framework initially trains a reward model
based on LLM feedback to evaluate the quality of candidate examples, followed
by knowledge distillation to train a bi-encoder based dense retriever. Our
experiments on a suite of 30 tasks demonstrate that our framework significantly
enhances in-context learning performance. Furthermore, we show the
generalization ability of our framework to unseen tasks during training. An
in-depth analysis reveals that our model improves performance by retrieving
examples with similar patterns, and the gains are consistent across LLMs of
varying sizes.
A framework is proposed for training dense retrievers to select high-quality in-context examples for large language models, enhancing their performance across various tasks.
LLMs in-context learning dense retrievers reward model bi-encoder knowledge distillation