gabrielmbmb
/

Upcycled-Qwen1.5-MoE2.7B

Text Generation

Model card Files Files and versions

gabrielmbmb commited on Mar 29, 2024

Commit

e268af1

·

verified ·

1 Parent(s): d265440

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -11,7 +11,7 @@ tags: []
 ## Model Details
-This my attemp (probably too naive) to reproduce the upcycling process used to initialize [Qwen1.5-MoE-A2.7B](https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B) using [Qwen1.5-1.8B](https://huggingface.co/Qwen/Qwen1.5-1.8B).
 ## Upcycling script

 ## Model Details
+This is my attemp (probably too naive) to reproduce the upcycling process used to initialize [Qwen1.5-MoE-A2.7B](https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B) using [Qwen1.5-1.8B](https://huggingface.co/Qwen/Qwen1.5-1.8B).
 ## Upcycling script