mistralai

/

Mistral-Small-3.2-24B-Instruct-2506

like 493

Follow

Mistral AI_ 12.8k

Hopefully this saves others some time:

llm = LLM(
    model='mistralai/Mistral-Small-3.2-24B-Instruct-2506',
    tokenizer_mode="mistral",
    config_format="mistral",
    load_format="mistral",
)

"llm = LLM(
model='mistralai/Mistral-Small-3.2-24B-Instruct-2506',
tokenizer_mode="mistral",
config_format="mistral",
load_format="mistral",
)"

Is this the implementation in vLLM?
So we have to accept that downloading from HuggingFace does not work like this:

tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.save_pretrained(local_path)
model = AutoModelForCausalLM.from_pretrained( model_id)
model.save_pretrained(local_path, safe_serialization=False)

Error: KeyError: <class 'transformers.models.mistral3.configuration_mistral3.Mistral3Config'>

I'm currently struggling with this and not really sure what to do. I put the config files from 2503 in as suggested by the OP but that doesn't seem to help. My model folder currently looks like this:

~/.cache/huggingface/hub/models--mistralai--Mistral-Small-3.2-24B-Instruct-2506/snapshots/cb17b97769b0305ddc717ede4a4ef6fd54ef8371# ls -lh
total 17M
lrwxrwxrwx 1 root root   52 Oct  4 05:56 config.json -> ../../blobs/786d10ba172eb033ce576f199e13cd65c3a8d905
lrwxrwxrwx 1 root root   76 Oct  4 05:58 consolidated.safetensors -> ../../blobs/2cc4cc283a4bf3820f77ddb73bef3128b7159057c240b94a72b663d97af86f89
lrwxrwxrwx 1 root root   52 Oct  4 05:57 generation_config.json -> ../../blobs/8e89f4a98a0588586cddf4ea551cd27629907a32
lrwxrwxrwx 1 root root   52 Oct  4 05:56 model.safetensors.index.json -> ../../blobs/c8d12958a1c6d6de8f6539883802e6e1a7bcac6a
lrwxrwxrwx 1 root root   52 Oct  4 05:56 params.json -> ../../blobs/ba80d5d1ebf33f2b8e7ad064f7a0c05057af7ab0
-rw-r--r-- 1 root root  634 Oct 11 14:16 preprocessor_config.json
-rw-r--r-- 1 root root  189 Oct 11 14:15 processor_config.json
lrwxrwxrwx 1 root root   76 Oct  4 05:57 tekken.json -> ../../blobs/6e2501687ccd0e1f30f36319eaf2b46958b897811e246cd8eb5d385b9e3de7d1
-rw-r--r-- 1 root root  17M Oct 11 14:16 tokenizer.json
-rw-r--r-- 1 root root 194K Oct 11 14:17 tokenizer_config.json

But I'm still getting this error:

[rank6]: Traceback (most recent call last):
[rank6]:   File "/root/minihf/agent/trainer/trainer.py", line 139, in <module>
[rank6]:     main()
[rank6]:   File "/root/minihf/agent/trainer/trainer.py", line 72, in main
[rank6]:     tokenizer = AutoTokenizer.from_pretrained(args.model)
[rank6]:   File "/root/tune_comma/env_comma/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 1156, in from_pretrained
[rank6]:     tokenizer_class_py, tokenizer_class_fast = TOKENIZER_MAPPING[type(config)]
[rank6]:   File "/root/tune_comma/env_comma/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 815, in __getitem__
[rank6]:     raise KeyError(key)
[rank6]: KeyError: <class 'transformers.models.mistral3.configuration_mistral3.Mistral3Config'>

I don't have a solution yet, but I'm leaving this here for now in case either someone knows or I figure it out and can report back.

UPDATE: So far I am mitigating this issue by using the mistralai/Mistral-Small-3.1-24B-Instruct-2503HF repo to get my tokenizer. Not great.