Source2Synth: Synthetic Data Generation and Curation Grounded in Real
  Data Sources
Large Language Models still struggle in challenging scenarios that leverage
structured data, complex reasoning, or tool usage. In this paper, we propose
Source2Synth: a new method that can be used for teaching LLMs new skills
without relying on costly human annotations. Source2Synth takes as input a
custom data source and produces synthetic data points with intermediate
reasoning steps grounded in real-world sources. Source2Synth improves the
dataset quality by discarding low-quality generations based on their
answerability. We demonstrate the generality of this approach by applying it to
two challenging domains: we test reasoning abilities in multi-hop question
answering (MHQA), and tool usage in tabular question answering (TQA). Our
method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on
HotPotQA compared to the fine-tuned baselines.
66e39c75444055a570a17890
Alisia Lupidi
66e39c75444055a570a17891
Carlos Gemmell
admin_assigned
2024-09-13T08:01:56.124Z
654ce9abce215f3cd11f8df7
/avatars/4139d3956cf8b1bb2ebd820e9b031123.svg
Carlos Gemmell
user
carlos-gemmell-malt-ai
66e39c75444055a570a17892
Nicola Cancedda
admin_assigned
2024-09-13T08:02:02.257Z
65d8ee3a2b0037ec192f7c20
/avatars/900fe214eb32279b5dfb1a8ff0ef4be1.svg
Nicola Cancedda
user
ncan
66e39c75444055a570a17893
Jane Dwivedi-Yu
66e39c75444055a570a17894
Jason Weston
admin_assigned
2024-09-13T08:02:17.791Z
62f023a36a027498eaa2f9cc
/avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
Jason Weston
user
spermwhale
66e39c75444055a570a17895
Jakob Foerster
66e39c75444055a570a17896
Roberta Raileanu
admin_assigned
2024-09-13T08:02:41.162Z
633e94793a17ab61de8e2b9c
/avatars/5f2f58ddeed211393660ada6b135f0d5.svg
Roberta Raileanu
user
rraileanu
66e39c75444055a570a17897
Maria Lomeli
admin_assigned
2024-09-13T08:02:54.467Z
64f86587e7584abc62545b3e
/avatars/57296279592ad5f77c39d174f6289420.svg
Maria Lomeli
user
marialomeli
Source2Synth: Synthetic Data Generation and Curation Grounded in Real
  Data Sources
Large Language Models still struggle in challenging scenarios that leverage
structured data, complex reasoning, or tool usage. In this paper, we propose
Source2Synth: a new method that can be used for teaching LLMs new skills
without relying on costly human annotations. Source2Synth takes as input a
custom data source and produces synthetic data points with intermediate
reasoning steps grounded in real-world sources. Source2Synth improves the
dataset quality by discarding low-quality generations based on their
answerability. We demonstrate the generality of this approach by applying it to
two challenging domains: we test reasoning abilities in multi-hop question
answering (MHQA), and tool usage in tabular question answering (TQA). Our
method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on
HotPotQA compared to the fine-tuned baselines.
Source2Synth improves LLM performance in structured reasoning and tool usage scenarios by generating high-quality synthetic data points and filtering out low-quality ones.
Large Language Models multi-hop question answering (MHQA) tabular question answering (TQA) synthetic data answerability filtering Source2Synth WikiSQL HotPotQA