Synthesizing Text-to-SQL Data from Weak and Strong LLMs
The capability gap between open-source and closed-source large language
models (LLMs) remains a challenge in text-to-SQL tasks. In this paper, we
introduce a synthetic data approach that combines data produced by larger, more
powerful models (strong models) with error information data generated by
smaller, not well-aligned models (weak models). The method not only enhances
the domain generalization of text-to-SQL models but also explores the potential
of error data supervision through preference learning. Furthermore, we employ
the synthetic data approach for instruction tuning on open-source LLMs,
resulting SENSE, a specialized text-to-SQL model. The effectiveness of SENSE is
demonstrated through state-of-the-art results on the SPIDER and BIRD
benchmarks, bridging the performance gap between open-source models and methods
prompted by closed-source models.
66b322a8cbe0b82f9573baa4
Jiaxi Yang
66b322a8cbe0b82f9573baa5
Binyuan Hui
admin_assigned
2024-08-14T13:06:06.668Z
61e4c4ca1ab24785ac11ba69
https://cdn-avatars.huggingface.co/v1/production/uploads/61e4c4ca1ab24785ac11ba69/1Q1zhhyGSJ9RJG9MzwxVv.jpeg
Binyuan Hui
user
huybery
66b322a8cbe0b82f9573baa6
Min Yang
66b322a8cbe0b82f9573baa7
Jian Yang
claimed_verified
2024-10-14T19:05:11.130Z
64ccb9bfead94891d12aef42
/avatars/c54809d43d93d3f0766bd2555cecc4e3.svg
Yang Jian
user
CSJianYang
66b322a8cbe0b82f9573baa8
Junyang Lin
admin_assigned
2024-08-14T13:06:14.095Z
620760a26e3b7210c2ff1943
https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/VC-rKqimF6yxGESNVlPoR.jpeg
Junyang Lin
user
JustinLin610
66b322a8cbe0b82f9573baa9
Chang Zhou
admin_assigned
2024-08-14T13:06:34.661Z
622892774323cef93a956a4a
/avatars/e57ef5c3b0c4289988ccd42f14e54336.svg
chang zhou
user
jiemizc
Synthesizing Text-to-SQL Data from Weak and Strong LLMs
The capability gap between open-source and closed-source large language
models (LLMs) remains a challenge in text-to-SQL tasks. In this paper, we
introduce a synthetic data approach that combines data produced by larger, more
powerful models (strong models) with error information data generated by
smaller, not well-aligned models (weak models). The method not only enhances
the domain generalization of text-to-SQL models but also explores the potential
of error data supervision through preference learning. Furthermore, we employ
the synthetic data approach for instruction tuning on open-source LLMs,
resulting SENSE, a specialized text-to-SQL model. The effectiveness of SENSE is
demonstrated through state-of-the-art results on the SPIDER and BIRD
benchmarks, bridging the performance gap between open-source models and methods
prompted by closed-source models.
A synthetic data approach combining strong and weak models improves open-source text-to-SQL models' performance and domain generalization.
large language models LLMs text-to-SQL synthetic data error information error data supervision preference learning instruction tuning SPIDER benchmark BIRD benchmark