Update README.md
Browse filesAdded disclaimer and removed MTEB comparison. Please refer to the official MTEB page for all results.
README.md
CHANGED
|
@@ -13,9 +13,10 @@ tags:
|
|
| 13 |
We trained the **PwC-Embedding-expr** model on top of the [multilingual-e5-large-instruct](https://huggingface.co/intfloat/multilingual-e5-large-instruct) embedding model.
|
| 14 |
To enhance performance in Korean, we applied our curated augmentation to STS datasets and fine-tuned the E5 model using a carefully balanced ratio across datasets.
|
| 15 |
|
|
|
|
| 16 |
|
| 17 |
### To-do
|
| 18 |
-
- [
|
| 19 |
- [ ] Technical Report
|
| 20 |
|
| 21 |
|
|
@@ -23,15 +24,15 @@ To enhance performance in Korean, we applied our curated augmentation to STS dat
|
|
| 23 |
PwC-Embedding_expr was evaluated on the Korean subset of MTEB.
|
| 24 |
A leaderboard link will be added once it is published.
|
| 25 |
|
| 26 |
-
| Task | PwC-Embedding_expr |
|
| 27 |
-
|
| 28 |
-
| KLUE-STS | 0.88 |
|
| 29 |
-
| KLUE-TC | 0.73 |
|
| 30 |
-
| Ko-StrategyQA | 0.80 |
|
| 31 |
-
| KorSTS | 0.84 |
|
| 32 |
-
| MIRACL-Reranking | 0.72 |
|
| 33 |
-
| MIRACL-Retrieval | 0.65 |
|
| 34 |
-
| **Average** | **0.77** |
|
| 35 |
|
| 36 |
|
| 37 |
## Model
|
|
|
|
| 13 |
We trained the **PwC-Embedding-expr** model on top of the [multilingual-e5-large-instruct](https://huggingface.co/intfloat/multilingual-e5-large-instruct) embedding model.
|
| 14 |
To enhance performance in Korean, we applied our curated augmentation to STS datasets and fine-tuned the E5 model using a carefully balanced ratio across datasets.
|
| 15 |
|
| 16 |
+
> ⚠️ This is an experimental model and is under continuous development.
|
| 17 |
|
| 18 |
### To-do
|
| 19 |
+
- [x] MTEB Leaderboard
|
| 20 |
- [ ] Technical Report
|
| 21 |
|
| 22 |
|
|
|
|
| 24 |
PwC-Embedding_expr was evaluated on the Korean subset of MTEB.
|
| 25 |
A leaderboard link will be added once it is published.
|
| 26 |
|
| 27 |
+
| Task | PwC-Embedding_expr |
|
| 28 |
+
|------------------|--------------------|
|
| 29 |
+
| KLUE-STS | 0.88 |
|
| 30 |
+
| KLUE-TC | 0.73 |
|
| 31 |
+
| Ko-StrategyQA | 0.80 |
|
| 32 |
+
| KorSTS | 0.84 |
|
| 33 |
+
| MIRACL-Reranking | 0.72 |
|
| 34 |
+
| MIRACL-Retrieval | 0.65 |
|
| 35 |
+
| **Average** | **0.77** |
|
| 36 |
|
| 37 |
|
| 38 |
## Model
|