cc522f2f2219b6467260677999c26a6b

This model is a fine-tuned version of FacebookAI/roberta-large on the nyu-mll/glue [qqp] dataset. It achieves the following results on the evaluation set:

• Loss: 0.8585
• Data Size: 0.25
• Epoch Runtime: 445.1982
• Accuracy: 0.3680
• F1 Macro: 0.2690
• Rouge1: 0.3682
• Rouge2: 0.0
• Rougel: 0.3681
• Rougelsum: 0.3683

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Accuracy	F1 Macro	Rouge1	Rouge2	Rougel	Rougelsum
No log	0	0	0.6540	0	52.7420	0.6320	0.3872	0.6318	0.0	0.6319	0.6317
0.6451	1	11370	0.5819	0.0078	66.7814	0.7987	0.7821	0.7988	0.0	0.7987	0.7987
0.5842	2	22740	0.5633	0.0156	78.9004	0.7674	0.7504	0.7674	0.0	0.7674	0.7673
0.6603	3	34110	0.6254	0.0312	103.6159	0.6527	0.6307	0.6527	0.0	0.6527	0.6526
0.6767	4	45480	0.6581	0.0625	151.4573	0.6320	0.3872	0.6318	0.0	0.6319	0.6317
0.6693	5	56850	0.6608	0.125	250.6893	0.6320	0.3872	0.6318	0.0	0.6319	0.6317
0.6585	6	68220	0.8585	0.25	445.1982	0.3680	0.2690	0.3682	0.0	0.3681	0.3683

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1