Multilingual-Transfer
Collection
Pretraining models to find what allows multilingual transfer • 26 items • Updated • 2
llama3-7b-en1-en2-codeswitching
LLaMA-3 7B pretrained on English bilingual data with code-switching, 34k steps.
Validation Results
Final checkpoint: step 33,400
| Metric | Value |
|---|---|
| Validation language | en2 |
| Cross-entropy loss (nats) | 2.3586 |
| Perplexity | 10.58 |
Evaluation Results
EEE-format evaluation results are stored under eval_results/eee/ in this repository.
Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).
Citation
Part of the The-CoLab multilingual-transfer collection.
- Downloads last month
- 68
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support