ReGAL: Refactoring Programs to Discover Generalizable Abstractions
While large language models (LLMs) are increasingly being used for program
synthesis, they lack the global view needed to develop useful abstractions;
they generally predict programs one at a time, often repeating the same
functionality. Generating redundant code from scratch is both inefficient and
error-prone. To address this, we propose Refactoring for Generalizable
Abstraction Learning (ReGAL), a gradient-free method for learning a library of
reusable functions via code refactorization, i.e. restructuring code without
changing its execution output. ReGAL learns from a small set of existing
programs, iteratively verifying and refining its abstractions via execution. We
find that the shared function libraries discovered by ReGAL make programs
easier to predict across diverse domains. On three datasets (LOGO graphics
generation, Date reasoning, and TextCraft, a Minecraft-based text game), both
open-source and proprietary LLMs improve in accuracy when predicting programs
with ReGAL functions. For CodeLlama-13B, ReGAL results in absolute accuracy
increases of 11.5% on graphics, 26.1% on date understanding, and 8.1% on
TextCraft, outperforming GPT-3.5 in two of three domains. Our analysis reveals
ReGAL's abstractions encapsulate frequently-used subroutines as well as
environment dynamics.
65b9b2401d610bfed4889bc8
Elias Stengel-Eskin
claimed_verified
2024-01-31T14:00:51.048Z
61781c4caf41befe8ff060e8
/avatars/8871d7b046fc28cbc8638228da8e9737.svg
Elias Stengel-Eskin
user
esteng
65b9b2401d610bfed4889bc9
Archiki Prasad
claimed_verified
2024-01-31T08:58:43.025Z
607aeae5d2cd8c150e6ae074
/avatars/a087743b98b6fe2181283a9610db4ec4.svg
Archiki Prasad
user
archiki
65b9b2401d610bfed4889bca
Mohit Bansal
ReGAL: Refactoring Programs to Discover Generalizable Abstractions
While large language models (LLMs) are increasingly being used for program
synthesis, they lack the global view needed to develop useful abstractions;
they generally predict programs one at a time, often repeating the same
functionality. Generating redundant code from scratch is both inefficient and
error-prone. To address this, we propose Refactoring for Generalizable
Abstraction Learning (ReGAL), a gradient-free method for learning a library of
reusable functions via code refactorization, i.e. restructuring code without
changing its execution output. ReGAL learns from a small set of existing
programs, iteratively verifying and refining its abstractions via execution. We
find that the shared function libraries discovered by ReGAL make programs
easier to predict across diverse domains. On three datasets (LOGO graphics
generation, Date reasoning, and TextCraft, a Minecraft-based text game), both
open-source and proprietary LLMs improve in accuracy when predicting programs
with ReGAL functions. For CodeLlama-13B, ReGAL results in absolute accuracy
increases of 11.5% on graphics, 26.1% on date understanding, and 8.1% on
TextCraft, outperforming GPT-3.5 in two of three domains. Our analysis reveals
ReGAL's abstractions encapsulate frequently-used subroutines as well as
environment dynamics.
Refactoring for Generalizable Abstraction Learning (ReGAL) enhances program synthesis by creating a library of reusable functions, improving the accuracy of large language models across diverse tasks.
large language models program synthesis abstractions code refactorization function libraries execution CodeLlama-13B GPT-3.5