train_copa_1752870512

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 1.8603
• Num Input Tokens Seen: 281856

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
8.9228	0.5	45	8.9923	14016
8.0092	1.0	90	7.8041	28096
6.3355	1.5	135	5.9985	42144
4.8584	2.0	180	4.7799	56128
4.0149	2.5	225	4.1103	70272
3.6163	3.0	270	3.6867	84352
3.2255	3.5	315	3.3251	98464
3.128	4.0	360	3.0117	112576
2.7234	4.5	405	2.7678	126624
2.525	5.0	450	2.5423	140832
2.2499	5.5	495	2.3521	154976
2.202	6.0	540	2.2062	169056
2.0048	6.5	585	2.0968	183200
1.9417	7.0	630	2.0127	197344
1.9274	7.5	675	1.9512	211392
1.9917	8.0	720	1.9081	225536
1.87	8.5	765	1.8837	239680
1.915	9.0	810	1.8719	253696
1.8226	9.5	855	1.8603	267840
1.9312	10.0	900	1.8612	281856

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1