drug-classifier

This model is a fine-tuned version of distilbert-base-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6099
• Accuracy: 0.8120
• F1: 0.8037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.5696	0.1492	500	1.4351	0.6050	0.5430
1.0091	0.2985	1000	0.9559	0.7178	0.6886
0.8572	0.4477	1500	0.8307	0.7450	0.7253
0.801	0.5970	2000	0.7775	0.7585	0.7422
0.7474	0.7462	2500	0.7413	0.7682	0.7525
0.6923	0.8955	3000	0.7179	0.7751	0.7643
0.6461	1.0445	3500	0.7038	0.7750	0.7643
0.6352	1.1937	4000	0.6927	0.7831	0.7697
0.6474	1.3430	4500	0.6766	0.7868	0.7750
0.5981	1.4922	5000	0.6601	0.7909	0.7800
0.5903	1.6414	5500	0.6669	0.7896	0.7802
0.6172	1.7907	6000	0.6364	0.7975	0.7869
0.6075	1.9399	6500	0.6318	0.7984	0.7884
0.4876	2.0889	7000	0.6348	0.7996	0.7920
0.5172	2.2382	7500	0.6311	0.8017	0.7943
0.5008	2.3874	8000	0.6216	0.8054	0.7972
0.4782	2.5367	8500	0.6181	0.8094	0.8016
0.506	2.6859	9000	0.6150	0.8095	0.8008
0.4969	2.8352	9500	0.6116	0.8093	0.8025
0.4844	2.9844	10000	0.6099	0.8120	0.8037

Framework versions

• Transformers 4.56.1
• Pytorch 2.8.0+cu126
• Datasets 4.0.0
• Tokenizers 0.22.0