train_wsc_456_1760356427

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3275
• Num Input Tokens Seen: 485152

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 456
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.9258	0.504	63	0.3463	24704
0.4236	1.008	126	0.3752	48688
0.3432	1.512	189	0.3334	73456
0.3609	2.016	252	0.3276	97568
0.3639	2.52	315	0.3736	121888
0.3418	3.024	378	0.3296	146336
0.3506	3.528	441	0.3275	172480
0.377	4.032	504	0.3651	196240
0.3479	4.536	567	0.3568	221136
0.3271	5.04	630	0.3355	244736
0.3529	5.5440	693	0.3348	268480
0.3433	6.048	756	0.3435	293424
0.354	6.552	819	0.3450	317840
0.348	7.056	882	0.3411	342384
0.3469	7.5600	945	0.3402	366288
0.342	8.064	1008	0.3410	391840
0.3469	8.568	1071	0.3438	416320
0.3467	9.072	1134	0.3437	440048
0.3497	9.576	1197	0.3431	464688

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1