add Evaluation

README.md

@@ -8,8 +8,6 @@ tags:
 - transformers
 - setfit
 license: mit
-metrics:
-  - cosine similarity
 datasets:
   - deutsche-telekom/ger-backtrans-paraphrase
 
@@ -21,19 +19,22 @@ It maps sentences & paragraphs (text) into a 1024 dimensional dense vector space
 The model is intended to be used together with [SetFit](https://github.com/huggingface/setfit)
 to improve German few-shot text classification.
 
-## Evaluation Results
-TODO
+This model is based on [deepset/gbert-large](https://huggingface.co/deepset/gbert-large).
+Many thanks to [deepset](https://www.deepset.ai/)!
 
 ## Training
 TODO
 
-## Full Model Architecture
-```
-SentenceTransformer(
-  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
-  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
-)
-```
+## Evaluation Results
+We use the [NLU Few-shot Benchmark - English and German](https://huggingface.co/datasets/deutsche-telekom/NLU-few-shot-benchmark-en-de)
+dataset to evaluate this model in a German few-shot scenario.
+
+**Qualitative results**\
+- multilingual sentence embeddings provide the worst results
+- Electra models also deliver poor results
+- German BERT base size model ([deepset/gbert-base](https://huggingface.co/deepset/gbert-base)) provides good results
+- German BERT large size model ([deepset/gbert-large](https://huggingface.co/deepset/gbert-large)) provides very good results
+- our fine-tuned models (this model and [deutsche-telekom/gbert-large-paraphrase-euclidean](https://huggingface.co/deutsche-telekom/gbert-large-paraphrase-euclidean)) provide best results
 
 ## Licensing
 Copyright (c) 2023 [Philip May](https://may.la/), [Deutsche Telekom AG](https://www.telekom.com/)\