Learning to Decode Collaboratively with Multiple Language Models
We propose a method to teach multiple large language models (LLM) to
collaborate by interleaving their generations at the token level. We model the
decision of which LLM generates the next token as a latent variable. By
optimizing the marginal likelihood of a training set under our latent variable
model, the base LLM automatically learns when to generate itself and when to
call on one of the ``assistant'' language models to generate, all without
direct supervision. Token-level collaboration during decoding allows for a
fusion of each model's expertise in a manner tailored to the specific task at
hand. Our collaborative decoding is especially useful in cross-domain settings
where a generalist base LLM learns to invoke domain expert models. On
instruction-following, domain-specific QA, and reasoning tasks, we show that
the performance of the joint system exceeds that of the individual models.
Through qualitative analysis of the learned latent decisions, we show models
trained with our method exhibit several interesting collaboration patterns,
e.g., template-filling. Our code is available at
https://github.com/clinicalml/co-llm.
65e95204014f4be34f6ce7ed
Shannon Zejiang Shen
admin_assigned
2024-03-07T09:59:44.246Z
613f897ffbfd59f147a88c81
https://cdn-avatars.huggingface.co/v1/production/uploads/1659455519119-613f897ffbfd59f147a88c81.jpeg
Shannon Shen
user
shannons
65e95204014f4be34f6ce7ee
Hunter Lang
admin_assigned
2024-03-07T09:59:51.040Z
62cdc3a5342b1d5dab9012d4
/avatars/2ec0afa7ef65a014f8926407da26ab17.svg
Hunter Lang
user
hjl
65e95204014f4be34f6ce7ef
Bailin Wang
65e95204014f4be34f6ce7f0
Yoon Kim
65e95204014f4be34f6ce7f1
David Sontag
admin_assigned
2024-03-07T10:00:48.683Z
634ffb851d81beb8e244a4bb
/avatars/16fff0e1a8cc8466240aa084eee467f3.svg
David Sontag
user
dsontag
Learning to Decode Collaboratively with Multiple Language Models
We propose a method to teach multiple large language models (LLM) to
collaborate by interleaving their generations at the token level. We model the
decision of which LLM generates the next token as a latent variable. By
optimizing the marginal likelihood of a training set under our latent variable
model, the base LLM automatically learns when to generate itself and when to
call on one of the ``assistant'' language models to generate, all without
direct supervision. Token-level collaboration during decoding allows for a
fusion of each model's expertise in a manner tailored to the specific task at
hand. Our collaborative decoding is especially useful in cross-domain settings
where a generalist base LLM learns to invoke domain expert models. On
instruction-following, domain-specific QA, and reasoning tasks, we show that
the performance of the joint system exceeds that of the individual models.
Through qualitative analysis of the learned latent decisions, we show models
trained with our method exhibit several interesting collaboration patterns,
e.g., template-filling. Our code is available at
https://github.com/clinicalml/co-llm.
Multiple large language models collaborate by interleaving token-level generation decisions, optimizing performance across tasks without direct supervision.
large language models LLM token-level collaboration latent variable marginal likelihood joint system instruction-following domain-specific QA reasoning tasks latent decisions template-filling