README.md

metadata

library_name: peft
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
tags:
  - llama-factory
  - generated_from_trainer
model-index:
  - name: train_winogrande_1754507494
    results: []

train_winogrande_1754507494

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1324
• Num Input Tokens Seen: 30830624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2219	0.5	4545	0.1793	1541600
0.117	1.0	9090	0.1384	3081600
0.1759	1.5	13635	0.1386	4623680
0.0729	2.0	18180	0.1352	6165104
0.3377	2.5	22725	0.1298	7706064
0.1626	3.0	27270	0.1327	9248016
0.0735	3.5	31815	0.1381	10789584
0.1049	4.0	36360	0.1307	12330800
0.0654	4.5	40905	0.1334	13871920
0.001	5.0	45450	0.1381	15413776
0.2929	5.5	49995	0.1352	16954320
0.2065	6.0	54540	0.1269	18496992
0.1853	6.5	59085	0.1326	20039264
0.2249	7.0	63630	0.1306	21579792
0.135	7.5	68175	0.1313	23122160
0.0009	8.0	72720	0.1314	24664400
0.005	8.5	77265	0.1320	26207280
0.0321	9.0	81810	0.1328	27747856
0.2766	9.5	86355	0.1317	29287888
0.0005	10.0	90900	0.1324	30830624

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1