Could you release a 20B‑scale MoE version? Thank you very much.
#27
by
houxiaowei
- opened
A 20‑B‑scale model that can run on edge devices with around 16 GB of memory. These machines make up a very large share of the market; it’s a “sweet‑spot” parameter size that avoids the severe hallucinations that can occur when the model is too small. Please
You can easily run A3B on 16GB