mistralai

/

Mistral-Small-3.2-24B-Instruct-2506

like 514

Follow

Mistral AI_ 13.2k

Simply loading the model from_pretrained and saving without changes throws an error with transformers latest versions, because we validate strictly that generation params are consistent. Setting a sampling flag solves the issue

Current woraround is to manually change the value after loading the model

from transformers import Mistral3ForConditionalGeneration
import torch

text_encoder = Mistral3ForConditionalGeneration.from_pretrained(
    "mistralai/Mistral-Small-3.2-24B-Instruct-2506", 
    dtype=torch.bfloat16
).to("cuda")
model.generation_config.do_sample = True # change the value
model.save_pretrained(output_dir) # SUCCESS!