train_wsc_101112_1760351840

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wsc dataset. It achieves the following results on the evaluation set:

• Loss: 0.3449
• Num Input Tokens Seen: 488816

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.03
• train_batch_size: 4
• eval_batch_size: 4
• seed: 101112
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.4762	0.504	63	0.4047	24608
0.346	1.008	126	0.3474	49296
0.5915	1.512	189	0.3491	74672
0.36	2.016	252	0.3449	98816
0.6543	2.52	315	0.3493	123680
0.3336	3.024	378	0.4224	147776
0.3359	3.528	441	0.3620	173312
0.386	4.032	504	0.3593	197728
0.3477	4.536	567	0.3515	222560
0.3466	5.04	630	0.3489	246848
0.3566	5.5440	693	0.3491	271008
0.3451	6.048	756	0.3467	295984
0.3555	6.552	819	0.3466	320080
0.3478	7.056	882	0.3523	345136
0.3544	7.5600	945	0.3477	370416
0.3546	8.064	1008	0.3495	394688
0.3436	8.568	1071	0.3511	418880
0.3359	9.072	1134	0.3496	444304
0.3421	9.576	1197	0.3495	469328

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1