Priority Sampling of Large Language Models for Compilers
Large language models show great potential in generating and optimizing code.
Widely used sampling methods such as Nucleus Sampling increase the diversity of
generation but often produce repeated samples for low temperatures and
incoherent samples for high temperatures. Furthermore, the temperature
coefficient has to be tuned for each task, limiting its usability. We present
Priority Sampling, a simple and deterministic sampling technique that produces
unique samples ordered by the model's confidence. Each new sample expands the
unexpanded token with the highest probability in the augmented search tree.
Additionally, Priority Sampling supports generation based on regular expression
that provides a controllable and structured exploration process. Priority
Sampling outperforms Nucleus Sampling for any number of samples, boosting the
performance of the original model from 2.87% to 5% improvement over -Oz.
Moreover, it outperforms the autotuner used for the generation of labels for
the training of the original model in just 30 samples.
65e15c5d66d097f80adcf21d
Dejan Grubisic
admin_assigned
2024-03-01T12:58:21.882Z
65d8b5a3132f10d591f3c00f
/avatars/3939f9582d882128dc2e64b0220f3844.svg
Dejan Grubisic
user
dejangrubisic
65e15c5d66d097f80adcf21e
Chris Cummins
admin_assigned
2024-03-01T12:58:29.377Z
63d32301c20657d2434d2fcb
https://cdn-avatars.huggingface.co/v1/production/uploads/1674781395584-noauth.jpeg
Chris Cummins
user
ChrisCummins
65e15c5d66d097f80adcf21f
Volker Seeker
65e15c5d66d097f80adcf220
Hugh Leather
Priority Sampling of Large Language Models for Compilers
Large language models show great potential in generating and optimizing code.
Widely used sampling methods such as Nucleus Sampling increase the diversity of
generation but often produce repeated samples for low temperatures and
incoherent samples for high temperatures. Furthermore, the temperature
coefficient has to be tuned for each task, limiting its usability. We present
Priority Sampling, a simple and deterministic sampling technique that produces
unique samples ordered by the model's confidence. Each new sample expands the
unexpanded token with the highest probability in the augmented search tree.
Additionally, Priority Sampling supports generation based on regular expression
that provides a controllable and structured exploration process. Priority
Sampling outperforms Nucleus Sampling for any number of samples, boosting the
performance of the original model from 2.87% to 5% improvement over -Oz.
Moreover, it outperforms the autotuner used for the generation of labels for
the training of the original model in just 30 samples.
Priority Sampling enhances code generation by producing unique, high-confidence samples and improving performance over Nucleus Sampling.
Priority Sampling Nucleus Sampling regular expression search tree autotuner