|
|
--- |
|
|
base_model: |
|
|
- Retreatcost/Ollpheist-12B |
|
|
- Vortex5/Shadow-Crystal-12B |
|
|
- Retreatcost/KansenSakura-Radiance-RP-12b |
|
|
- Vortex5/MegaMoon-Karcher-12B |
|
|
library_name: transformers |
|
|
tags: |
|
|
- mergekit |
|
|
- merge |
|
|
- roleplay |
|
|
--- |
|
|
<div align="center" style="background:radial-gradient(circle at top left,#0b0b0d 10%,#000000 60%);padding:40px 25px;border-radius:10px;"> |
|
|
|
|
|
<h1 style="color:#ffe9a9;font-family:Cinzel,serif;font-size:2.6rem;letter-spacing:2px;text-transform:uppercase;text-shadow:0 0 12px rgba(255,255,200,0.4),0 0 25px rgba(255,230,160,0.25);">Luminous-Shadow-12B</h1> |
|
|
|
|
|
<p style="color:#d4a74a;font-family:Cinzel,serif;font-size:1.1rem;">“Within the deepest shadow, the brightest light awaits.”</p> |
|
|
|
|
|
</div> |
|
|
|
|
|
|
|
|
 |
|
|
|
|
|
### ✨ **Overview** |
|
|
<div style="background-color:rgba(10,10,10,0.85);border-left:4px solid #e4b865;padding:15px 20px;border-radius:6px;color:#f5e8c7;font-family:'JetBrains Mono',monospace;"> |
|
|
<b>Luminous-Shadow-12B</b> was merged using the <b><a href="https://arxiv.org/abs/2406.11617" style="color:#ffd47f;">DELLA</a></b> merge method via <a href="https://github.com/arcee-ai/mergekit" style="color:#e4b865;">MergeKit</a>, balancing ethereal creativity and reasoned coherence.<br><br> |
|
|
It draws from the expressive nature of <a href="https://huggingface.co/Vortex5/Shadow-Crystal-12B" style="color:#ffd47f;">Shadow-Crystal</a>, the refined structure of <a href="https://huggingface.co/Retreatcost/KansenSakura-Radiance-RP-12b" style="color:#ffd47f;">KansenSakura-Radiance-RP</a>, and the stylistic artistry of <a href="https://huggingface.co/Retreatcost/Ollpheist-12B" style="color:#ffd47f;">Ollpheist</a>. |
|
|
</div> |
|
|
|
|
|
|
|
|
### 🪶 **Merge Configuration** |
|
|
<details> |
|
|
<summary style="color:#ffd47f;font-family:Cinzel,serif;font-weight:600;letter-spacing:1px;cursor:pointer;">Show Config</summary> |
|
|
|
|
|
```yaml |
|
|
models: |
|
|
- model: Retreatcost/KansenSakura-Radiance-RP-12b |
|
|
parameters: |
|
|
weight: |
|
|
- filter: self_attn |
|
|
value: [0.2, 0.25, 0.35, 0.55, 0.7, 0.8, 0.65, 0.4] |
|
|
- filter: mlp |
|
|
value: [0.25, 0.35, 0.25, 0.44] |
|
|
- filter: norm |
|
|
value: 0.35 |
|
|
- value: 0.40 |
|
|
density: 0.45 |
|
|
epsilon: 0.25 |
|
|
- model: Retreatcost/Ollpheist-12B |
|
|
parameters: |
|
|
weight: |
|
|
- filter: self_attn |
|
|
value: [0.0, 0.1, 0.25, 0.45, 0.55, 0.45, 0.25, 0.1] |
|
|
- filter: mlp |
|
|
value: [0.0, 0.15, 0.3, 0.5, 0.7, 0.55, 0.35, 0.15] |
|
|
- filter: norm |
|
|
value: 0.25 |
|
|
- filter: lm_head |
|
|
value: 0.4 |
|
|
- value: 0.25 |
|
|
density: 0.4 |
|
|
epsilon: 0.35 |
|
|
- model: Vortex5/Shadow-Crystal-12B |
|
|
parameters: |
|
|
weight: |
|
|
- filter: self_attn |
|
|
value: [0.2, 0.2, 0.15, 0.35, 0.55, 0.55, 0.25, 0.6] |
|
|
- filter: mlp |
|
|
value: [0.0, 0.1, 0.25, 0.5, 0.4, 0.4, 0.65, 0.65] |
|
|
- filter: lm_head |
|
|
value: 0.55 |
|
|
- filter: norm |
|
|
value: 0.15 |
|
|
- value: 0.15 |
|
|
density: 0.35 |
|
|
epsilon: 0.25 |
|
|
merge_method: della |
|
|
base_model: Vortex5/MegaMoon-Karcher-12B |
|
|
parameters: |
|
|
lambda: 1.0 |
|
|
normalize: true |
|
|
dtype: bfloat16 |
|
|
tokenizer: |
|
|
source: Retreatcost/KansenSakura-Radiance-RP-12b |
|
|
``` |
|
|
</details> |
|
|
|
|
|
### 🪄**Intended Use** |
|
|
<div style="background-color:rgba(10,10,10,0.85);border-left:4px solid #e4b865;padding:15px 20px;border-radius:6px;color:#f5e8c7;"> |
|
|
🧘 Reflective dialogue • 🖋️ Creative writing • 💞 Character roleplay — blending emotion, intellect, and style into a single expressive voice. |
|
|
</div> |
|
|
|
|
|
<div align="center" style="height:1px;width:100%;background:linear-gradient(90deg,transparent,rgba(255,215,100,0.7),transparent);margin:40px 0;"></div> |
|
|
|
|
|
### ✨ **Acknowledgements** |
|
|
<div style="background-color:rgba(10,10,10,0.85);border-left:4px solid #e4b865;padding:15px 20px;border-radius:6px;color:#c9b078;"> |
|
|
<ul> |
|
|
<li>⚙️ <b>mradermacher</b> — static / imatrix quantization</li> |
|
|
<li>🜛 <b>DeathGodlike</b> — EXL3 quants</li> |
|
|
<li>🌟 <b>All original authors and contributors</b> whose models formed the foundation for this merge</li> |
|
|
</ul> |
|
|
</div> |
