train_copa_42_1757596066

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the copa dataset. It achieves the following results on the evaluation set:

• Loss: 0.0101
• Num Input Tokens Seen: 282368

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.0468	0.5	45	0.0860	14176
0.1056	1.0	90	0.0588	28256
0.1713	1.5	135	0.0494	42240
0.0889	2.0	180	0.0286	56480
0.1651	2.5	225	0.0291	70688
0.284	3.0	270	0.0386	84736
0.0054	3.5	315	0.0101	98784
0.0874	4.0	360	0.0165	113024
0.0043	4.5	405	0.0343	127264
0.0933	5.0	450	0.0423	141440
0.1026	5.5	495	0.0233	155488
0.001	6.0	540	0.0476	169600
0.0014	6.5	585	0.0284	183648
0.0734	7.0	630	0.0492	197792
0.1057	7.5	675	0.0459	211936
0.0552	8.0	720	0.0465	225984
0.1727	8.5	765	0.0532	239936
0.0779	9.0	810	0.0471	254112
0.0012	9.5	855	0.0479	268256
0.0003	10.0	900	0.0452	282368

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1