OmniPred: Language Models as Universal Regressors
Over the broad landscape of experimental design, regression has been a
powerful tool to accurately predict the outcome metrics of a system or model
given a set of parameters, but has been traditionally restricted to methods
which are only applicable to a specific task. In this paper, we propose
OmniPred, a framework for training language models as universal end-to-end
regressors over (x,y) evaluation data from diverse real world experiments.
Using data sourced from Google Vizier, one of the largest blackbox optimization
databases in the world, our extensive experiments demonstrate that through only
textual representations of mathematical parameters and values, language models
are capable of very precise numerical regression, and if given the opportunity
to train over multiple tasks, can significantly outperform traditional
regression models.
65d8116a5959cf8b839af381
Xingyou Song
admin_assigned
2024-02-23T12:39:45.864Z
64c93c8c4524c2aea71d2ec6
/avatars/1961fb6209182bf4954778dc3d97dde5.svg
Xingyou Song
user
Srizzle
65d8116a5959cf8b839af382
Oscar Li
65d8116a5959cf8b839af383
Chansoo Lee
65d8116a5959cf8b839af384
Bangding
65d8116a5959cf8b839af385
Yang
65d8116a5959cf8b839af386
Daiyi Peng
65d8116a5959cf8b839af387
Sagi Perel
admin_assigned
2024-02-23T12:40:17.934Z
63b0e82243aaf9276fed1d7f
/avatars/ed6df77f106aabd7fa82e0602264d1df.svg
Sagi Perel
user
sagipe
65d8116a5959cf8b839af388
Yutian Chen
OmniPred: Language Models as Universal Regressors
Over the broad landscape of experimental design, regression has been a
powerful tool to accurately predict the outcome metrics of a system or model
given a set of parameters, but has been traditionally restricted to methods
which are only applicable to a specific task. In this paper, we propose
OmniPred, a framework for training language models as universal end-to-end
regressors over (x,y) evaluation data from diverse real world experiments.
Using data sourced from Google Vizier, one of the largest blackbox optimization
databases in the world, our extensive experiments demonstrate that through only
textual representations of mathematical parameters and values, language models
are capable of very precise numerical regression, and if given the opportunity
to train over multiple tasks, can significantly outperform traditional
regression models.
OmniPred, a framework training language models as universal end-to-end regressors, outperforms traditional regression models using textual representations of parameters and values from diverse experimental data.
OmniPred language models end-to-end regressors Google Vizier blackbox optimization numerical regression