Add support for greedy decoding

by adityastomar - opened 6 days ago

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

-0

adityastomar

6 days ago

The current implementation of sampling only uses torch.multinomial and does not support greedy decoding when temperature is 0.0 / top-k is 0 / top-p is 1.0. This PR adds support for greedy decoding.

Add support for greedy decoding0c28d1a3

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Ready to merge

This branch is ready to get merged automatically.

· Sign up or log in to comment