--- base_model: allenai/longformer-base-4096 tags: - generated_from_trainer datasets: - essays_su_g metrics: - accuracy model-index: - name: longformer-full_labels results: - task: name: Token Classification type: token-classification dataset: name: essays_su_g type: essays_su_g config: full_labels split: train[80%:100%] args: full_labels metrics: - name: Accuracy type: accuracy value: 0.8354393714471922 --- # longformer-full_labels This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset. It achieves the following results on the evaluation set: - Loss: 0.4449 - B-claim: {'precision': 0.5258620689655172, 'recall': 0.45018450184501846, 'f1-score': 0.485089463220676, 'support': 271.0} - B-majorclaim: {'precision': 0.7142857142857143, 'recall': 0.07194244604316546, 'f1-score': 0.13071895424836602, 'support': 139.0} - B-premise: {'precision': 0.7081604426002767, 'recall': 0.8088467614533965, 'f1-score': 0.7551622418879057, 'support': 633.0} - I-claim: {'precision': 0.622454448017149, 'recall': 0.580604848787803, 'f1-score': 0.6008017586964955, 'support': 4001.0} - I-majorclaim: {'precision': 0.6968287526427062, 'recall': 0.8186785891703925, 'f1-score': 0.7528551850159891, 'support': 2013.0} - I-premise: {'precision': 0.8654449817595656, 'recall': 0.8998764996471419, 'f1-score': 0.8823249578341911, 'support': 11336.0} - O: {'precision': 0.9420488250057039, 'recall': 0.8950791242141773, 'f1-score': 0.9179635393508226, 'support': 9226.0} - Accuracy: 0.8354 - Macro avg: {'precision': 0.725012176182376, 'recall': 0.6464589673087279, 'f1-score': 0.6464165857506352, 'support': 27619.0} - Weighted avg: {'precision': 0.8358464532914583, 'recall': 0.8354393714471922, 'f1-score': 0.8334161098638371, 'support': 27619.0} ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 5 ### Training results | Training Loss | Epoch | Step | Validation Loss | B-claim | B-majorclaim | B-premise | I-claim | I-majorclaim | I-premise | O | Accuracy | Macro avg | Weighted avg | |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------:|:----------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:| | No log | 1.0 | 41 | 0.6799 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 271.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 139.0} | {'precision': 0.875, 'recall': 0.044233807266982623, 'f1-score': 0.08421052631578947, 'support': 633.0} | {'precision': 0.44683080146673654, 'recall': 0.2131967008247938, 'f1-score': 0.28866328257191204, 'support': 4001.0} | {'precision': 0.592, 'recall': 0.36761053154495776, 'f1-score': 0.4535703340484217, 'support': 2013.0} | {'precision': 0.7292961700421094, 'recall': 0.9625088214537756, 'f1-score': 0.8298284975472487, 'support': 11336.0} | {'precision': 0.8543361149255307, 'recall': 0.8766529373509646, 'f1-score': 0.8653506660247152, 'support': 9226.0} | 0.7466 | {'precision': 0.49963758377633954, 'recall': 0.35202897120592486, 'f1-score': 0.36023190092972673, 'support': 27619.0} | {'precision': 0.7126524282765021, 'recall': 0.7465874941163692, 'f1-score': 0.706468200590435, 'support': 27619.0} | | No log | 2.0 | 82 | 0.5045 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 271.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 139.0} | {'precision': 0.5941676792223572, 'recall': 0.7725118483412322, 'f1-score': 0.6717032967032966, 'support': 633.0} | {'precision': 0.5916276346604216, 'recall': 0.5051237190702325, 'f1-score': 0.5449642712687071, 'support': 4001.0} | {'precision': 0.65738555922605, 'recall': 0.6920019870839543, 'f1-score': 0.6742497579864472, 'support': 2013.0} | {'precision': 0.8346545866364666, 'recall': 0.910197600564573, 'f1-score': 0.8707907840324077, 'support': 11336.0} | {'precision': 0.9139132389300967, 'recall': 0.8814220680685021, 'f1-score': 0.8973736482012802, 'support': 9226.0} | 0.8093 | {'precision': 0.5131069569536274, 'recall': 0.5373224604469277, 'f1-score': 0.5227259654560198, 'support': 27619.0} | {'precision': 0.7951024792507403, 'recall': 0.8093341540244035, 'f1-score': 0.800655657521358, 'support': 27619.0} | | No log | 3.0 | 123 | 0.4710 | {'precision': 0.5217391304347826, 'recall': 0.17712177121771217, 'f1-score': 0.2644628099173554, 'support': 271.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 139.0} | {'precision': 0.6571798188874515, 'recall': 0.8025276461295419, 'f1-score': 0.7226173541963017, 'support': 633.0} | {'precision': 0.6227746053073564, 'recall': 0.4633841539615096, 'f1-score': 0.531384350816853, 'support': 4001.0} | {'precision': 0.6513105639396346, 'recall': 0.8147044212617983, 'f1-score': 0.7239020083866696, 'support': 2013.0} | {'precision': 0.8291489025738197, 'recall': 0.9264290755116443, 'f1-score': 0.8750937421881511, 'support': 11336.0} | {'precision': 0.9421622250669149, 'recall': 0.8775200520268805, 'f1-score': 0.9086929681800325, 'support': 9226.0} | 0.8200 | {'precision': 0.6034736066014228, 'recall': 0.5802410171584409, 'f1-score': 0.5751647476693377, 'support': 27619.0} | {'precision': 0.8129119859079974, 'recall': 0.8200152069227705, 'f1-score': 0.8116164134497381, 'support': 27619.0} | | No log | 4.0 | 164 | 0.4437 | {'precision': 0.4723618090452261, 'recall': 0.34686346863468637, 'f1-score': 0.4, 'support': 271.0} | {'precision': 0.8571428571428571, 'recall': 0.04316546762589928, 'f1-score': 0.08219178082191782, 'support': 139.0} | {'precision': 0.6771653543307087, 'recall': 0.8151658767772512, 'f1-score': 0.739784946236559, 'support': 633.0} | {'precision': 0.6176223776223776, 'recall': 0.5518620344913772, 'f1-score': 0.5828933474128827, 'support': 4001.0} | {'precision': 0.7292452830188679, 'recall': 0.7680079483358172, 'f1-score': 0.7481248487781272, 'support': 2013.0} | {'precision': 0.8598264678628591, 'recall': 0.9004057868736768, 'f1-score': 0.879648381953721, 'support': 11336.0} | {'precision': 0.9251513483764446, 'recall': 0.9110123563841318, 'f1-score': 0.9180274152148981, 'support': 9226.0} | 0.8321 | {'precision': 0.7340736424856201, 'recall': 0.6194975627318342, 'f1-score': 0.621524388631158, 'support': 27619.0} | {'precision': 0.8290421682205733, 'recall': 0.8321083312212607, 'f1-score': 0.8279682509392567, 'support': 27619.0} | | No log | 5.0 | 205 | 0.4449 | {'precision': 0.5258620689655172, 'recall': 0.45018450184501846, 'f1-score': 0.485089463220676, 'support': 271.0} | {'precision': 0.7142857142857143, 'recall': 0.07194244604316546, 'f1-score': 0.13071895424836602, 'support': 139.0} | {'precision': 0.7081604426002767, 'recall': 0.8088467614533965, 'f1-score': 0.7551622418879057, 'support': 633.0} | {'precision': 0.622454448017149, 'recall': 0.580604848787803, 'f1-score': 0.6008017586964955, 'support': 4001.0} | {'precision': 0.6968287526427062, 'recall': 0.8186785891703925, 'f1-score': 0.7528551850159891, 'support': 2013.0} | {'precision': 0.8654449817595656, 'recall': 0.8998764996471419, 'f1-score': 0.8823249578341911, 'support': 11336.0} | {'precision': 0.9420488250057039, 'recall': 0.8950791242141773, 'f1-score': 0.9179635393508226, 'support': 9226.0} | 0.8354 | {'precision': 0.725012176182376, 'recall': 0.6464589673087279, 'f1-score': 0.6464165857506352, 'support': 27619.0} | {'precision': 0.8358464532914583, 'recall': 0.8354393714471922, 'f1-score': 0.8334161098638371, 'support': 27619.0} | ### Framework versions - Transformers 4.37.2 - Pytorch 2.2.0+cu121 - Datasets 2.17.0 - Tokenizers 0.15.2