Initial commit: EXL3 6.0bpw quantization with proper LFS tracking

.gitattributes

@@ -0,0 +1,4 @@
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,218 @@
+---
+base_model: [BruhzWater/Sapphira-L3.3-70b-0.1]
+library_name: transformers
+tags:
+- mergekit
+- merge
+
+---
+# Sapphira-L3.3-70b-0.1
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/66ca56e62400073af3ad2972/CPUXeq81a9o0_ClXCcG68.png)
+
+Storytelling and RP model with increased coherence, thanks to cogito-v2-preview-llama-70B.
+
+iMatrix quants: https://huggingface.co/mradermacher/Sapphira-L3.3-70b-0.1-i1-GGUF
+
+Static quants: https://huggingface.co/mradermacher/Sapphira-L3.3-70b-0.1-GGUF
+
+Chat Template:
+-
+Llama3
+
+Instruction Template:
+-
+Deep Cogito
+
+Llama3
+
+Sampler Settings
+-
+
+Starter:
+```
+Temp: 1
+Min_P: 0.02
+Top_P: 1
+```
+
+Experimental 1:
+```
+Temp: .95 - 1.1
+Min_P: .015 - .03
+Top_P: .97 - .99
+XTC_Threshold: .11
+XTC_Probability: .15
+```
+
+Experimental 2:
+```
+Temp: .95 - 1.1
+Min_P: .015 - .03
+Top_P: 1
+Typical_P: .99
+XTC_Threshold: .11
+XTC_Probability: .15
+```
+
+### Merge Method
+
+This model was merged using the [Multi-SLERP](https://goddard.blog/posts/multislerp-wow-what-a-cool-idea) merge method using deepcogito--cogito-v2-preview-llama-70B as a base.
+
+### Models Merged
+
+The following models were included in the merge:
+* BruhzWater--Apocrypha-L3.3-70b-0.3
+* BruhzWater--Serpents-Tongue-L3.3-70b-0.3
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+models:
+  - model: /workspace/cache/models--BruhzWater--Apocrypha-L3.3-70b-0.3/snapshots/3facb4c0a7b953ff34a5caa90976830bf82a84c2
+    parameters:
+      weight: [0.5]
+  - model: /workspace/cache/models--BruhzWater--Serpents-Tongue-L3.3-70b-0.3/snapshots/d007a7bcc7047d712abb2dfb6ad940fe03cd2047
+    parameters:
+      weight: [0.5]
+base_model: /workspace/cache/models--deepcogito--cogito-v2-preview-llama-70B/snapshots/1e1d12e8eaebd6084a8dcf45ecdeaa2f4b8879ce
+merge_method: multislerp
+tokenizer:
+  source: base
+chat_template: llama3
+parameters:
+  normalize_weights: false
+  eps: 1e-9
+pad_to_multiple_of: 8
+int8_mask: true
+dtype: bfloat16
+```
+
+### Instruct Template
+
+Deep Cogito
+
+```
+{{- '<|begin_of_text|>' }}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+{%- if not enable_thinking is defined %}
+    {%- set enable_thinking = false %}
+{%- endif %}
+{#- This block extracts the system message, so we can slot it into the right place. #}
+{%- if messages[0]['role'] == 'system' %}
+    {%- set system_message = messages[0]['content']|trim %}
+    {%- set messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+{%- endif %}
+{#- Set the system message. If enable_thinking is true, add the "Enable deep thinking subroutine." #}
+{%- if enable_thinking %}
+    {%- if system_message != "" %}
+        {%- set system_message = "Enable deep thinking subroutine.
+
+" ~ system_message %}
+    {%- else %}
+        {%- set system_message = "Enable deep thinking subroutine." %}
+    {%- endif %}
+{%- endif %}
+{#- Set the system message. In case there are tools present, add them to the system message. #}
+{%- if tools is not none or system_message != '' %}
+    {{- "<|start_header_id|>system<|end_header_id|>
+
+" }}
+    {{- system_message }}
+    {%- if tools is not none %}
+        {%- if system_message != "" %}
+            {{- "
+
+" }}
+        {%- endif %}
+        {{- "Available Tools:
+" }}
+        {%- for t in tools %}
+            {{- t | tojson(indent=4) }}
+            {{- "
+
+" }}
+        {%- endfor %}
+    {%- endif %}
+    {{- "<|eot_id|>" }}
+{%- endif %}
+
+{#- Rest of the messages #}
+{%- for message in messages %}
+    {#- The special cases are when the message is from a tool (via role ipython/tool/tool_results) or when the message is from the assistant, but has "tool_calls". If not, we add the message directly as usual. #}
+    {#- Case 1 - Usual, non tool related message. #}
+    {%- if not (message.role == "ipython" or message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
+        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>
+
+' }}
+        {%- if message['content'] is string %}
+            {{- message['content'] | trim }}
+        {%- else %}
+            {%- for item in message['content'] %}
+                {%- if item.type == 'text' %}
+                    {{- item.text | trim }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|eot_id|>' }}
+
+    {#- Case 2 - the response is from the assistant, but has a tool call returned. The assistant may also have returned some content along with the tool call. #}
+    {%- elif message.tool_calls is defined and message.tool_calls is not none %}
+        {{- "<|start_header_id|>assistant<|end_header_id|>
+
+" }}
+        {%- if message['content'] is string %}
+            {{- message['content'] | trim }}
+        {%- else %}
+            {%- for item in message['content'] %}
+                {%- if item.type == 'text' %}
+                    {{- item.text | trim }}
+                    {%- if item.text | trim != "" %}
+                        {{- "
+
+" }}
+                    {%- endif %}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- "[" }}
+        {%- for tool_call in message.tool_calls %}
+            {%- set out = tool_call.function|tojson %}
+            {%- if not tool_call.id is defined %}
+                {{- out }}
+            {%- else %}
+                {{- out[:-1] }}
+                {{- ', "id": "' + tool_call.id + '"}' }}
+            {%- endif %}
+            {%- if not loop.last %}
+                {{- ", " }}
+            {%- else %}
+                {{- "]<|eot_id|>" }}
+            {%- endif %}
+        {%- endfor %}
+
+    {#- Case 3 - the response is from a tool call. The tool call may have an id associated with it as well. If it does, we add it to the prompt. #}
+    {%- elif message.role == "ipython" or message["role"] == "tool_results" or message["role"] == "tool" %}
+        {{- "<|start_header_id|>ipython<|end_header_id|>
+
+" }}
+        {%- if message.tool_call_id is defined and message.tool_call_id != '' %}
+            {{- '{"content": ' + (message.content | tojson) + ', "call_id": "' + message.tool_call_id + '"}' }}
+        {%- else %}
+            {{- '{"content": ' + (message.content | tojson) + '}' }}
+        {%- endif %}
+        {{- "<|eot_id|>" }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_header_id|>assistant<|end_header_id|>
+
+' }}
+{%- endif %}
+```

chat_template.jinja

@@ -0,0 +1,7 @@
+{% set loop_messages = messages %}
+{% for message in loop_messages %}
+{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}
+{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}
+{{ content }}
+{% endfor %}
+{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}

config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e6efa44dcc545093b52fa5bd9bbf39590dece6b3d064b47c81cf2273976e6fe
+size 1173

mergekit_config.yml

@@ -0,0 +1,18 @@
+models:
+  - model: /workspace/cache/models--BruhzWater--Apocrypha-L3.3-70b-0.3/snapshots/3facb4c0a7b953ff34a5caa90976830bf82a84c2
+    parameters:
+      weight: [0.5]
+  - model: /workspace/cache/models--BruhzWater--Serpents-Tongue-L3.3-70b-0.3/snapshots/d007a7bcc7047d712abb2dfb6ad940fe03cd2047
+    parameters:
+      weight: [0.5]
+base_model: /workspace/cache/models--deepcogito--cogito-v2-preview-llama-70B/snapshots/1e1d12e8eaebd6084a8dcf45ecdeaa2f4b8879ce
+merge_method: multislerp
+tokenizer:
+  source: base
+chat_template: llama3
+parameters:
+  normalize_weights: false
+  eps: 1e-9
+pad_to_multiple_of: 8
+int8_mask: true
+dtype: bfloat16

model-00001-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7eaa33849ca6a8fee52f475a19e79971adf6c43114e2eff65c36bd91d028db45
+size 8522220432

model-00002-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9edbc8c5e1c9bd6268802804ec511b1f63fa7e53384b6b5cab4ad71ed5f9474d
+size 8347135176

model-00003-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:729621ff1f1b63a7a63c97913ca50da15b005625b65f8388b2339b0a1139fed7
+size 8347135176

model-00004-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b4eb9c9cb2bfd1963464b3ca2f02b6f760f59a2a52a67be34b3f4d1a8ae51e8
+size 8347135176

model-00005-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ac398889101b3cebd99d10ab496688af32ec8e5e54f9600cf631c8774992626
+size 8347135176

model-00006-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:709decc56986c1cece30fe28b120cc467cab004317ee5a68dd3a5926bc8e13e0
+size 8347135176

model-00007-of-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5dddc8bc2bc5b3b014e3685700fca65d83406bf37a524525950d80679209cf1
+size 3998731056

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46b07d76d45fe0859f7175e29113f0afdf1e5b81396e00fb2cd61066c31c8f09
+size 155376

quantization_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc888fd18a7c5e5d2caaf7251c9dd27872c6ed610cfb7f95b40d143d65c4d11f
+size 593665

special_tokens_map.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94e708c3f5e64acf85bbe5ad01467a1248faadb73e83b41793087ecced586e8f
+size 454

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2152033590c3bc3e5d4a66356b91f3ace338a57cf2357b68db871e9bc4337909
+size 50569