Table-GPT: Table-tuned GPT for Diverse Table Tasks
Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable
abilities to follow diverse human instructions and perform a wide range of
tasks. However, when probing language models using a range of basic
table-understanding tasks, we observe that today's language models are still
sub-optimal in many table-related tasks, likely because they are pre-trained
predominantly on one-dimensional natural-language texts, whereas
relational tables are two-dimensional objects.
  In this work, we propose a new "table-tuning" paradigm, where we
continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using
diverse table-tasks synthesized from real tables as training data, with the
goal of enhancing language models' ability to understand tables and perform
table tasks. We show that our resulting Table-GPT models demonstrate (1) better
table-understanding capabilities, by consistently outperforming the
vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout
unseen tasks, and (2) strong generalizability, in its ability to respond
to diverse human instructions to perform new table-tasks, in a manner similar
to GPT-3.5 and ChatGPT.
652c9b55a2d97e682b62b270
Peng Li
652c9b55a2d97e682b62b271
Yeye He
652c9b55a2d97e682b62b272
Dror Yashar
652c9b55a2d97e682b62b273
Weiwei Cui
652c9b55a2d97e682b62b274
Song Ge
652c9b55a2d97e682b62b275
Haidong Zhang
652c9b55a2d97e682b62b276
Danielle Rifinski Fainman
652c9b55a2d97e682b62b277
Dongmei Zhang
652c9b55a2d97e682b62b278
Surajit Chaudhuri
Table-GPT: Table-tuned GPT for Diverse Table Tasks
Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable
abilities to follow diverse human instructions and perform a wide range of
tasks. However, when probing language models using a range of basic
table-understanding tasks, we observe that today's language models are still
sub-optimal in many table-related tasks, likely because they are pre-trained
predominantly on one-dimensional natural-language texts, whereas
relational tables are two-dimensional objects.
  In this work, we propose a new "table-tuning" paradigm, where we
continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using
diverse table-tasks synthesized from real tables as training data, with the
goal of enhancing language models' ability to understand tables and perform
table tasks. We show that our resulting Table-GPT models demonstrate (1) better
table-understanding capabilities, by consistently outperforming the
vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout
unseen tasks, and (2) strong generalizability, in its ability to respond
to diverse human instructions to perform new table-tasks, in a manner similar
to GPT-3.5 and ChatGPT.
A new table-tuning paradigm improves language models' understanding and generalization in table-related tasks by fine-tuning them with synthesized table data, resulting in enhanced performance.
table-tuning table-understanding generalizability Table-GPT GPT-3.5 ChatGPT table-tasks