Initial commit: 5.0bpw EXL3 quantization of Sapphira-L3.3-70b-0.1

.gitattributes

@@ -0,0 +1,3 @@
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,218 @@
+---
+base_model: []
+library_name: transformers
+tags:
+- mergekit
+- merge
+
+---
+# Sapphira-L3.3-70b-0.1
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/66ca56e62400073af3ad2972/CPUXeq81a9o0_ClXCcG68.png)
+
+Storytelling and RP model with increased coherence, thanks to cogito-v2-preview-llama-70B.
+
+iMatrix quants: https://huggingface.co/mradermacher/Sapphira-L3.3-70b-0.1-i1-GGUF
+
+Static quants: https://huggingface.co/mradermacher/Sapphira-L3.3-70b-0.1-GGUF
+
+Chat Template:
+-
+Llama3
+
+Instruction Template:
+-
+Deep Cogito
+
+Llama3
+
+Sampler Settings
+-
+
+Starter:
+```
+Temp: 1
+Min_P: 0.02
+Top_P: 1
+```
+
+Experimental 1:
+```
+Temp: .95 - 1.1
+Min_P: .015 - .03
+Top_P: .97 - .99
+XTC_Threshold: .11
+XTC_Probability: .15
+```
+
+Experimental 2:
+```
+Temp: .95 - 1.1
+Min_P: .015 - .03
+Top_P: 1
+Typical_P: .99
+XTC_Threshold: .11
+XTC_Probability: .15
+```
+
+### Merge Method
+
+This model was merged using the [Multi-SLERP](https://goddard.blog/posts/multislerp-wow-what-a-cool-idea) merge method using deepcogito--cogito-v2-preview-llama-70B as a base.
+
+### Models Merged
+
+The following models were included in the merge:
+* BruhzWater--Apocrypha-L3.3-70b-0.3
+* BruhzWater--Serpents-Tongue-L3.3-70b-0.3
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+models:
+  - model: /workspace/cache/models--BruhzWater--Apocrypha-L3.3-70b-0.3/snapshots/3facb4c0a7b953ff34a5caa90976830bf82a84c2
+    parameters:
+      weight: [0.5]
+  - model: /workspace/cache/models--BruhzWater--Serpents-Tongue-L3.3-70b-0.3/snapshots/d007a7bcc7047d712abb2dfb6ad940fe03cd2047
+    parameters:
+      weight: [0.5]
+base_model: /workspace/cache/models--deepcogito--cogito-v2-preview-llama-70B/snapshots/1e1d12e8eaebd6084a8dcf45ecdeaa2f4b8879ce
+merge_method: multislerp
+tokenizer:
+  source: base
+chat_template: llama3
+parameters:
+  normalize_weights: false
+  eps: 1e-9
+pad_to_multiple_of: 8
+int8_mask: true
+dtype: bfloat16
+```
+
+### Instruct Template
+
+Deep Cogito
+
+```
+{{- '<|begin_of_text|>' }}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+{%- if not enable_thinking is defined %}
+    {%- set enable_thinking = false %}
+{%- endif %}
+{#- This block extracts the system message, so we can slot it into the right place. #}
+{%- if messages[0]['role'] == 'system' %}
+    {%- set system_message = messages[0]['content']|trim %}
+    {%- set messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+{%- endif %}
+{#- Set the system message. If enable_thinking is true, add the "Enable deep thinking subroutine." #}
+{%- if enable_thinking %}
+    {%- if system_message != "" %}
+        {%- set system_message = "Enable deep thinking subroutine.
+
+" ~ system_message %}
+    {%- else %}
+        {%- set system_message = "Enable deep thinking subroutine." %}
+    {%- endif %}
+{%- endif %}
+{#- Set the system message. In case there are tools present, add them to the system message. #}
+{%- if tools is not none or system_message != '' %}
+    {{- "<|start_header_id|>system<|end_header_id|>
+
+" }}
+    {{- system_message }}
+    {%- if tools is not none %}
+        {%- if system_message != "" %}
+            {{- "
+
+" }}
+        {%- endif %}
+        {{- "Available Tools:
+" }}
+        {%- for t in tools %}
+            {{- t | tojson(indent=4) }}
+            {{- "
+
+" }}
+        {%- endfor %}
+    {%- endif %}
+    {{- "<|eot_id|>" }}
+{%- endif %}
+
+{#- Rest of the messages #}
+{%- for message in messages %}
+    {#- The special cases are when the message is from a tool (via role ipython/tool/tool_results) or when the message is from the assistant, but has "tool_calls". If not, we add the message directly as usual. #}
+    {#- Case 1 - Usual, non tool related message. #}
+    {%- if not (message.role == "ipython" or message.role == "tool" or message.role == "tool_results" or (message.tool_calls is defined and message.tool_calls is not none)) %}
+        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>
+
+' }}
+        {%- if message['content'] is string %}
+            {{- message['content'] | trim }}
+        {%- else %}
+            {%- for item in message['content'] %}
+                {%- if item.type == 'text' %}
+                    {{- item.text | trim }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|eot_id|>' }}
+
+    {#- Case 2 - the response is from the assistant, but has a tool call returned. The assistant may also have returned some content along with the tool call. #}
+    {%- elif message.tool_calls is defined and message.tool_calls is not none %}
+        {{- "<|start_header_id|>assistant<|end_header_id|>
+
+" }}
+        {%- if message['content'] is string %}
+            {{- message['content'] | trim }}
+        {%- else %}
+            {%- for item in message['content'] %}
+                {%- if item.type == 'text' %}
+                    {{- item.text | trim }}
+                    {%- if item.text | trim != "" %}
+                        {{- "
+
+" }}
+                    {%- endif %}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- "[" }}
+        {%- for tool_call in message.tool_calls %}
+            {%- set out = tool_call.function|tojson %}
+            {%- if not tool_call.id is defined %}
+                {{- out }}
+            {%- else %}
+                {{- out[:-1] }}
+                {{- ', "id": "' + tool_call.id + '"}' }}
+            {%- endif %}
+            {%- if not loop.last %}
+                {{- ", " }}
+            {%- else %}
+                {{- "]<|eot_id|>" }}
+            {%- endif %}
+        {%- endfor %}
+
+    {#- Case 3 - the response is from a tool call. The tool call may have an id associated with it as well. If it does, we add it to the prompt. #}
+    {%- elif message.role == "ipython" or message["role"] == "tool_results" or message["role"] == "tool" %}
+        {{- "<|start_header_id|>ipython<|end_header_id|>
+
+" }}
+        {%- if message.tool_call_id is defined and message.tool_call_id != '' %}
+            {{- '{"content": ' + (message.content | tojson) + ', "call_id": "' + message.tool_call_id + '"}' }}
+        {%- else %}
+            {{- '{"content": ' + (message.content | tojson) + '}' }}
+        {%- endif %}
+        {{- "<|eot_id|>" }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_header_id|>assistant<|end_header_id|>
+
+' }}
+{%- endif %}
+```

chat_template.jinja

@@ -0,0 +1,7 @@
+{% set loop_messages = messages %}
+{% for message in loop_messages %}
+{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}
+{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}
+{{ content }}
+{% endfor %}
+{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}

config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59eef339bbb977ca3cfd57e6e4bf48f1af90f045a2d412f77086bfc95a53a86e
+size 1173

mergekit_config.yml

@@ -0,0 +1,18 @@
+models:
+  - model: /workspace/cache/models--BruhzWater--Apocrypha-L3.3-70b-0.3/snapshots/3facb4c0a7b953ff34a5caa90976830bf82a84c2
+    parameters:
+      weight: [0.5]
+  - model: /workspace/cache/models--BruhzWater--Serpents-Tongue-L3.3-70b-0.3/snapshots/d007a7bcc7047d712abb2dfb6ad940fe03cd2047
+    parameters:
+      weight: [0.5]
+base_model: /workspace/cache/models--deepcogito--cogito-v2-preview-llama-70B/snapshots/1e1d12e8eaebd6084a8dcf45ecdeaa2f4b8879ce
+merge_method: multislerp
+tokenizer:
+  source: base
+chat_template: llama3
+parameters:
+  normalize_weights: false
+  eps: 1e-9
+pad_to_multiple_of: 8
+int8_mask: true
+dtype: bfloat16

model-00001-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03b2f6e74d135ea5ba6026e2429e41d17ee2d9824d4a25e1886139981d12f61a
+size 8522938256

model-00002-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51d1a9aaa20fbd1631ab2f9df752cd587df88dcd7ab61182edc440f795774f2d
+size 8562121120

model-00003-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:881f1bd964f8aba51ea83503e7806c557600e05db3db86260448ee5f2a1a3537
+size 8562121120

model-00004-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42921f05804dd418b6df892fffbaaf568d7fefcab2cc524b45b0e410ec8865ab
+size 8562121120

model-00005-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3c605e62b53c2718ab08eaa1bacb8ed34309819401a8b861ec69ada5944ef0f
+size 8562121120

model-00006-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4aea4ae38ae19a0a39e49be24845f374553f6013af38b597d6e3b2f44ac90b8
+size 2928824720

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73cb2e515f909a784b79d373740573006696d88b7d7729e3659d12e6c0fb1a57
+size 155376

quantization_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13417a19df521eabc8ee56b0c0c7c03dcf2ea36598e987e5055781251e49787d
+size 593665

special_tokens_map.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94e708c3f5e64acf85bbe5ad01467a1248faadb73e83b41793087ecced586e8f
+size 454

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2152033590c3bc3e5d4a66356b91f3ace338a57cf2357b68db871e9bc4337909
+size 50569