Update README.md
Browse files
README.md
CHANGED
|
@@ -7,7 +7,7 @@ datasets:
|
|
| 7 |
metrics:
|
| 8 |
- rouge
|
| 9 |
model-index:
|
| 10 |
-
- name:
|
| 11 |
results:
|
| 12 |
- task:
|
| 13 |
name: Summarization
|
|
@@ -22,33 +22,21 @@ model-index:
|
|
| 22 |
value: 47.4222
|
| 23 |
---
|
| 24 |
|
| 25 |
-
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
| 26 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 27 |
|
| 28 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
-
This model is a fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) on the mlsum tu dataset.
|
| 31 |
It achieves the following results on the evaluation set:
|
| 32 |
-
|
| 33 |
- Rouge1: 47.4222
|
| 34 |
- Rouge2: 34.8624
|
| 35 |
- Rougel: 42.2487
|
| 36 |
- Rougelsum: 43.9494
|
| 37 |
-
- Gen Len: 51.3525
|
| 38 |
-
|
| 39 |
-
## Model description
|
| 40 |
-
|
| 41 |
-
More information needed
|
| 42 |
-
|
| 43 |
-
## Intended uses & limitations
|
| 44 |
-
|
| 45 |
-
More information needed
|
| 46 |
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
More information needed
|
| 50 |
-
|
| 51 |
-
## Training procedure
|
| 52 |
|
| 53 |
### Training hyperparameters
|
| 54 |
|
|
@@ -67,25 +55,22 @@ The following hyperparameters were used during training:
|
|
| 67 |
- num_epochs: 10.0
|
| 68 |
- label_smoothing_factor: 0.1
|
| 69 |
|
| 70 |
-
### Training results
|
| 71 |
-
|
| 72 |
-
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|
| 73 |
-
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
|
| 74 |
-
| 3.084 | 1.0 | 3895 | 2.9282 | 31.6872 | 22.1113 | 29.2851 | 29.7608 | 18.9861 |
|
| 75 |
-
| 2.9162 | 2.0 | 7790 | 2.8552 | 32.1716 | 22.5001 | 29.6845 | 30.1887 | 18.9938 |
|
| 76 |
-
| 2.8149 | 3.0 | 11685 | 2.8089 | 32.5681 | 22.689 | 30.0409 | 30.5507 | 18.9959 |
|
| 77 |
-
| 2.7325 | 4.0 | 15580 | 2.7948 | 33.1236 | 23.1775 | 30.5156 | 31.0461 | 18.9958 |
|
| 78 |
-
| 2.6679 | 5.0 | 19475 | 2.7810 | 33.1766 | 23.162 | 30.4802 | 31.0527 | 18.9967 |
|
| 79 |
-
| 2.6237 | 6.0 | 23370 | 2.7790 | 33.1118 | 23.2043 | 30.5064 | 31.0096 | 18.9978 |
|
| 80 |
-
| 2.5711 | 7.0 | 27265 | 2.7801 | 33.2033 | 23.2957 | 30.59 | 31.1504 | 18.9979 |
|
| 81 |
-
| 2.538 | 8.0 | 31160 | 2.7777 | 33.0256 | 23.0621 | 30.3818 | 30.978 | 18.998 |
|
| 82 |
-
| 2.5 | 9.0 | 35055 | 2.7839 | 33.2288 | 23.2361 | 30.5421 | 31.1573 | 18.998 |
|
| 83 |
-
| 2.4719 | 10.0 | 38950 | 2.7832 | 33.2098 | 23.2274 | 30.5164 | 31.1094 | 18.9981 |
|
| 84 |
-
|
| 85 |
-
|
| 86 |
### Framework versions
|
| 87 |
|
| 88 |
- Transformers 4.11.3
|
| 89 |
- Pytorch 1.8.2+cu111
|
| 90 |
- Datasets 1.14.0
|
| 91 |
- Tokenizers 0.10.3
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
metrics:
|
| 8 |
- rouge
|
| 9 |
model-index:
|
| 10 |
+
- name: mt5-base-turkish-sum
|
| 11 |
results:
|
| 12 |
- task:
|
| 13 |
name: Summarization
|
|
|
|
| 22 |
value: 47.4222
|
| 23 |
---
|
| 24 |
|
|
|
|
|
|
|
| 25 |
|
| 26 |
+
# [Mukayese: Turkish NLP Strikes Back](https://arxiv.org/abs/2203.01215)
|
| 27 |
+
|
| 28 |
+
## Summarization: mukayese/mbart-large-turkish-sum
|
| 29 |
+
|
| 30 |
+
This model is a fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) on the mlsum/tu dataset.
|
| 31 |
|
|
|
|
| 32 |
It achieves the following results on the evaluation set:
|
| 33 |
+
|
| 34 |
- Rouge1: 47.4222
|
| 35 |
- Rouge2: 34.8624
|
| 36 |
- Rougel: 42.2487
|
| 37 |
- Rougelsum: 43.9494
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
+
Check [this](https://arxiv.org/abs/2203.01215) paper for more details on the model and the dataset.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
### Training hyperparameters
|
| 42 |
|
|
|
|
| 55 |
- num_epochs: 10.0
|
| 56 |
- label_smoothing_factor: 0.1
|
| 57 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
### Framework versions
|
| 59 |
|
| 60 |
- Transformers 4.11.3
|
| 61 |
- Pytorch 1.8.2+cu111
|
| 62 |
- Datasets 1.14.0
|
| 63 |
- Tokenizers 0.10.3
|
| 64 |
+
|
| 65 |
+
### Citation
|
| 66 |
+
|
| 67 |
+
```
|
| 68 |
+
@misc{safaya-etal-2022-mukayese,
|
| 69 |
+
title={Mukayese: Turkish NLP Strikes Back},
|
| 70 |
+
author={Ali Safaya and Emirhan Kurtuluş and Arda Göktoğan and Deniz Yuret},
|
| 71 |
+
year={2022},
|
| 72 |
+
eprint={2203.01215},
|
| 73 |
+
archivePrefix={arXiv},
|
| 74 |
+
primaryClass={cs.CL}
|
| 75 |
+
}
|
| 76 |
+
```
|