xu-song's picture
add more tokenizer
c75633b
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('google/byt5-small')