Code for training
It's impressive how the model responds using such a small amount of data. Could you share the code for training?
Hello @cybertruck32489 ,
You can use this notebook for the larger Qwen 3 8B model here. Just change the model_name to this Liontix/Qwen3-4B-Thinking-2507-Gemini-2.5-Pro-Distill
I will upload the safetensor repo for that model shortly.
No, that's not what I mean. I'm talking about the code that allows you to achieve such changes in the model's responses with so few examples.
I used the same notebook, just with a normal Qwen3 safetensor base model. That unsloth notebook handles the supervised fine-tuning and selects/loads the parameters that are going to be updated for the fine-tuning progress. For a good, fine-tuned model, I aim for at least 30 epoch runs on the chosen dataset. The used dataset really makes the difference here in my opinion, as it is very diverse considering its size.
wow i would suggest lowering the learning rate !!!
to at least 2e6
this way you dont get over fit like that notebook !
if your model has 0.01 its enough its better to be at 0.3 etc but what your looking for in training is the point when the model begins to flat line ..here perhaps at .2