cwe-parent-vulnerability-classification-roberta-base-roberta-base

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3755
• Accuracy: 0.6603
• F1 Macro: 0.4616

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Macro
2.9549	1.0	238	2.9056	0.0948	0.0729
2.2865	2.0	476	1.9760	0.4946	0.3041
1.8517	3.0	714	1.7010	0.5114	0.3522
1.6439	4.0	952	1.5457	0.6074	0.3826
1.3475	5.0	1190	1.5154	0.5894	0.3608
1.1372	6.0	1428	1.4379	0.6327	0.4183
1.0323	7.0	1666	1.3955	0.6411	0.4184
0.8662	8.0	1904	1.3755	0.6603	0.4616
0.8135	9.0	2142	1.4626	0.6807	0.4703
0.632	10.0	2380	1.4197	0.6999	0.4439
0.5727	11.0	2618	1.4083	0.6795	0.4878
0.5429	12.0	2856	1.5234	0.6651	0.4823
0.3597	13.0	3094	1.5866	0.7107	0.4995
0.3076	14.0	3332	1.6262	0.7191	0.5243
0.2458	15.0	3570	1.7271	0.6963	0.5259
0.2052	16.0	3808	1.7799	0.7011	0.4556
0.1801	17.0	4046	1.7717	0.7179	0.4983
0.187	18.0	4284	2.0014	0.7239	0.5273
0.1473	19.0	4522	1.9999	0.7419	0.5388
0.1198	20.0	4760	1.9328	0.7275	0.5336
0.152	21.0	4998	2.0637	0.7407	0.4759
0.0692	22.0	5236	2.2153	0.7647	0.5553
0.0632	23.0	5474	2.1253	0.7431	0.5381
0.069	24.0	5712	2.2856	0.7587	0.5443
0.0472	25.0	5950	2.3607	0.7611	0.5286
0.0452	26.0	6188	2.4693	0.7539	0.5191
0.0388	27.0	6426	2.4699	0.7587	0.5550
0.0412	28.0	6664	2.5062	0.7659	0.5332
0.0419	29.0	6902	2.4443	0.7551	0.5488
0.0238	30.0	7140	2.5642	0.7479	0.5487
0.0616	31.0	7378	2.5451	0.7623	0.5511
0.0163	32.0	7616	2.6758	0.7599	0.5450
0.028	33.0	7854	2.6806	0.7671	0.5432
0.0147	34.0	8092	2.6815	0.7647	0.5518
0.0251	35.0	8330	2.7046	0.7611	0.5470
0.0151	36.0	8568	2.6610	0.7527	0.5440
0.0128	37.0	8806	2.7269	0.7551	0.5426
0.0421	38.0	9044	2.7759	0.7515	0.5437
0.0259	39.0	9282	2.7239	0.7587	0.5444
0.0046	40.0	9520	2.7196	0.7599	0.5448

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.1+cu128
• Datasets 4.4.1
• Tokenizers 0.22.1