Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,414 +1,3 @@
----
-license: apache-2.0
-base_model:
-- Qwen/Qwen3-4B
-- enlo/Jan-nano
-- ValiantLabs/Qwen3-4B-Esper3
-- prithivMLmods/Crux-Qwen3_OpenThinking-4B
-- XformAI-india/Qwen3-4B-medicaldataset
-- Tesslate/UIGEN-T3-4B-Preview
-- radm/prophet-qwen3-4b-sft
-- NiuTrans/GRAM-Qwen3-4B-RewardModel
-library_name: transformers
-license_link: https://huggingface.co/Qwen/Qwen3-4B/blob/main/LICENSE
-pipeline_tag: text-generation
-tags:
-- moe
----
-# huihui-ai/Huihui-MoE-23B-A4B
-
-## Model Overview
-Huihui-MoE-23B-A4B is a **Mixture of Experts (MoE)** language model developed by **huihui.ai**, built upon the **[Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B)** base model. It enhances the standard Transformer architecture by replacing MLP layers with MoE layers, each containing 8 experts, to achieve high performance with efficient inference. The model is designed for natural language processing tasks, including text generation, question answering, and conversational applications.
-
-**Note**:  
-The activated expert can handle numbers from 1 to 8, and can complete normal conversations as well. 
-You can change the activation parameters using `/num_experts_per_tok <number>`. After modifying the parameters, the model will be reloaded.
-
- - **Architecture**: Qwen3MoeForCausalLM model with 8 experts per layer (num_experts=8), activating 1-8 expert per token (num_experts_per_tok=1-8).
- - **Total Parameters**: ~23 billion (23B)
- - **Activated Parameters**: ~42 billion (4B) during inference, comparable to Qwen3-4B
- - **Developer**: huihui.ai
- - **Release Date**: June 2025
- - **License**: Inherits the license of the Qwen3 base model (apache-2.0)
-
-## Expert Models:
-  
-### Expert 1:
-[Menlo/Jan-nano](https://huggingface.co/Menlo/Jan-nano)
-
-### Expert 2:
-[ValiantLabs/Qwen3-4B-Esper3](https://huggingface.co/ValiantLabs/Qwen3-4B-Esper3)
-
-### Expert 3:
-[prithivMLmods/Crux-Qwen3_OpenThinking-4B](https://huggingface.co/prithivMLmods/Crux-Qwen3_OpenThinking-4B)
-
-### Expert 4:
-[XformAI-india/Qwen3-4B-medicaldataset](https://huggingface.co/XformAI-india/Qwen3-4B-medicaldataset)
-
-### Expert 5:
-[Tesslate/UIGEN-T3-4B-Preview](https://huggingface.co/Tesslate/UIGEN-T3-4B-Preview)
-
-### Expert 6:
-[radm/prophet-qwen3-4b-sft](https://huggingface.co/radm/prophet-qwen3-4b-sft)
-
-### Expert 7:
-[NiuTrans/GRAM-Qwen3-4B-RewardModel](https://huggingface.co/NiuTrans/GRAM-Qwen3-4B-RewardModel)
-
-### Expert 8:
-[Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B)
-
-### Instruction Following:
-[Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B)
-
-`Qwen/Qwen3-4B` model was directly used for this expert, no fine-tune was applied.
-
-## Training
-
- - **Base Model**: Qwen3-4B, pre-trained by the Qwen team.
- - **Conversion**: The model copies embeddings, self-attention, and normalization weights from Qwen3-4B, replacing MLP layers with MoE layers (8 experts). Gating weights are randomly initialized.
- - **Fine-Tuning**: Not fine-tuned; users are recommended to fine-tune for specific tasks to optimize expert routing.
-## ollama
-
-You can use [huihui_ai/huihui-moe:23b](https://ollama.com/huihui_ai/huihui-moe:23b) directly, 
-Switch the thinking toggle using /set think and /set nothink
-```
-ollama run huihui_ai/huihui-moe:23b
-```
-
-## Usage
-
-```
-from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig, BitsAndBytesConfig, TextStreamer
-import torch
-import os
-import signal
-import random
-import numpy as np
-import time
-from collections import Counter
-
-cpu_count = os.cpu_count()
-print(f"Number of CPU cores in the system: {cpu_count}")
-half_cpu_count = cpu_count // 2
-os.environ["MKL_NUM_THREADS"] = str(half_cpu_count)
-os.environ["OMP_NUM_THREADS"] = str(half_cpu_count)
-torch.set_num_threads(half_cpu_count)
-
-print(f"PyTorch threads: {torch.get_num_threads()}")
-print(f"MKL threads: {os.getenv('MKL_NUM_THREADS')}")
-print(f"OMP threads: {os.getenv('OMP_NUM_THREADS')}")
-
-# Load the model and tokenizer
-NEW_MODEL_ID = "huihui-ai/Huihui-MoE-23B-A4B"
-print(f"Load Model {NEW_MODEL_ID} ... ")
-quant_config_4 = BitsAndBytesConfig(
-    load_in_4bit=True,
-    bnb_4bit_compute_dtype=torch.bfloat16,
-    bnb_4bit_use_double_quant=True,
-    llm_int8_enable_fp32_cpu_offload=True,
-)
-
-config = AutoConfig.from_pretrained(
-    NEW_MODEL_ID,
-    trust_remote_code=True
-)
-
-config.num_experts_per_tok = 1
-print(f"num_experts_per_tok: {config.num_experts_per_tok}")
-
-model = AutoModelForCausalLM.from_pretrained(
-    NEW_MODEL_ID,
-    config=config,
-    device_map="auto",
-    trust_remote_code=True,
-    quantization_config=quant_config_4,
-    torch_dtype=torch.bfloat16
-)
-#print(model.config)
-tokenizer = AutoTokenizer.from_pretrained(NEW_MODEL_ID, trust_remote_code=True)
-if tokenizer.pad_token is None:
-    tokenizer.pad_token = tokenizer.eos_token
-tokenizer.pad_token_id = tokenizer.eos_token_id
-
-tokenizer = AutoTokenizer.from_pretrained(NEW_MODEL_ID, trust_remote_code=True)
-if tokenizer.pad_token is None:
-    tokenizer.pad_token = tokenizer.eos_token
-tokenizer.pad_token_id = tokenizer.eos_token_id
-
-messages = []
-nothink = False
-same_seed = False
-skip_prompt=True
-skip_special_tokens=True
-do_sample = True
-
-def set_random_seed(seed=None):
-    """Set random seed for reproducibility. If seed is None, use int(time.time())."""
-    if seed is None:
-        seed = int(time.time())  # Convert float to int
-    random.seed(seed)
-    np.random.seed(seed)
-    torch.manual_seed(seed)
-    torch.cuda.manual_seed_all(seed)  # If using CUDA
-    torch.backends.cudnn.deterministic = True
-    torch.backends.cudnn.benchmark = False
-    return seed  # Return seed for logging if needed
-
-class CustomTextStreamer(TextStreamer):
-    def __init__(self, tokenizer, skip_prompt=True, skip_special_tokens=True):
-        super().__init__(tokenizer, skip_prompt=skip_prompt, skip_special_tokens=skip_special_tokens)
-        self.generated_text = ""
-        self.stop_flag = False
-        self.init_time = time.time()  # Record initialization time
-        self.end_time = None  # To store end time
-        self.first_token_time = None  # To store first token generation time
-        self.token_count = 0  # To track total tokens
-
-    def on_finalized_text(self, text: str, stream_end: bool = False):
-        if self.first_token_time is None and text.strip():  # Set first token time on first non-empty text
-            self.first_token_time = time.time()
-        self.generated_text += text
-        # Count tokens in the generated text
-        tokens = self.tokenizer.encode(text, add_special_tokens=False)
-        self.token_count += len(tokens)
-        print(text, end="", flush=True)
-        if stream_end:
-            self.end_time = time.time()  # Record end time when streaming ends
-        if self.stop_flag:
-            raise StopIteration
-
-    def stop_generation(self):
-        self.stop_flag = True
-        self.end_time = time.time()  # Record end time when generation is stopped
-
-    def get_metrics(self):
-        """Returns initialization time, first token time, first token latency, end time, total time, total tokens, and tokens per second."""
-        if self.end_time is None:
-            self.end_time = time.time()  # Set end time if not already set
-        total_time = self.end_time - self.init_time  # Total time from init to end
-        tokens_per_second = self.token_count / total_time if total_time > 0 else 0
-        first_token_latency = (self.first_token_time - self.init_time) if self.first_token_time is not None else None
-        metrics = {
-            "init_time": self.init_time,
-            "first_token_time": self.first_token_time,
-            "first_token_latency": first_token_latency,
-            "end_time": self.end_time,
-            "total_time": total_time,  # Total time in seconds
-            "total_tokens": self.token_count,
-            "tokens_per_second": tokens_per_second
-        }
-        return metrics
-        
-def generate_stream(model, tokenizer, messages, nothink, skip_prompt, skip_special_tokens, do_sample, max_new_tokens):
-    input_ids = tokenizer.apply_chat_template(
-        messages,
-        tokenize=True,
-        enable_thinking = not nothink,
-        add_generation_prompt=True,
-        return_tensors="pt"
-    )
-    attention_mask = torch.ones_like(input_ids, dtype=torch.long)
-    tokens = input_ids.to(model.device) 
-    attention_mask = attention_mask.to(model.device)
-
-    streamer = CustomTextStreamer(tokenizer, skip_prompt=skip_prompt, skip_special_tokens=skip_special_tokens)
-
-    def signal_handler(sig, frame):
-        streamer.stop_generation()
-        print("\n[Generation stopped by user with Ctrl+C]")
-
-    signal.signal(signal.SIGINT, signal_handler)
-
-    generate_kwargs = {}
-    if do_sample:
-        generate_kwargs = {
-              "do_sample": do_sample,
-              "max_length": max_new_tokens,
-              "temperature": 0.6,
-              "top_k": 20,
-              "top_p": 0.95,
-              "repetition_penalty": 1.2,
-              "no_repeat_ngram_size": 2
-        }
-    else:
-        generate_kwargs = {
-              "do_sample": do_sample,
-              "max_length": max_new_tokens,
-              "repetition_penalty": 1.2,
-              "no_repeat_ngram_size": 2
-        }
-  
-          
-    print("Response: ", end="", flush=True)
-    try:
-        generated_ids = model.generate(
-            tokens,
-            attention_mask=attention_mask,
-            #use_cache=False,
-            pad_token_id=tokenizer.pad_token_id,
-            streamer=streamer,
-            **generate_kwargs
-        )
-        del generated_ids
-    except StopIteration:
-        print("\n[Stopped by user]")
-
-    del input_ids, attention_mask
-    torch.cuda.empty_cache()
-    signal.signal(signal.SIGINT, signal.SIG_DFL)
-
-    return streamer.generated_text, streamer.stop_flag, streamer.get_metrics()
-
-init_seed = set_random_seed()
-
-activated_experts = []
-
-# Define hook function to capture gate_probs output
-def hook_fn(module, input, output):
-    # output is gate_probs, shape: [batch_size, sequence_length, num_experts]
-    gate_probs = output
-    # Compute top-1 expert indices (since only one expert is activated)
-    _, topk_indices = gate_probs.topk(config.num_experts_per_tok, dim=-1)  # Take top-1
-    # Flatten and store activated expert indices
-    activated_experts.extend(topk_indices.squeeze(-1).view(-1).cpu().tolist())
-
-hooks = []
-for layer in model.model.layers:
-    hooks.append(layer.mlp.gate.register_forward_hook(hook_fn))
-      
-
-while True:
-    # List to store activated expert indices
-    activated_experts = []
-
-    if same_seed:
-        set_random_seed(init_seed)
-    else:
-        init_seed = set_random_seed()
-        
-    print(f"\nnothink: {nothink}")
-    print(f"skip_prompt: {skip_prompt}")
-    print(f"skip_special_tokens: {skip_special_tokens}")
-    print(f"do_sample: {do_sample}")
-    print(f"same_seed: {same_seed}, {init_seed}\n")
-    
-    user_input = input("User: ").strip()
-    if user_input.lower() == "/exit":
-        print("Exiting chat.")
-        break
-    if user_input.lower() == "/clear":
-        messages = []
-        print("Chat history cleared. Starting a new conversation.")
-        continue
-    if user_input.lower() == "/nothink":
-        nothink = not nothink
-        continue
-    if user_input.lower() == "/skip_prompt":
-        skip_prompt = not skip_prompt
-        continue
-    if user_input.lower() == "/skip_special_tokens":
-        skip_special_tokens = not skip_special_tokens
-        continue
-    if user_input.lower().startswith("/same_seed"):
-        parts = user_input.split()
-        if len(parts) == 1:  # /same_seed (no number)
-            same_seed = not same_seed  # Toggle switch
-        elif len(parts) == 2:  # /same_seed <number>
-            try:
-                init_seed = int(parts[1])  # Extract and convert number to int
-                same_seed = True
-            except ValueError:
-                print("Error: Please provide a valid integer after /same_seed")       
-        continue
-    if user_input.lower().startswith("/num_experts_per_tok"):
-        parts = user_input.split()
-        if len(parts) == 1:  # /num_experts_per_tok (no number)
-            config.num_experts_per_tok = 1  # set default 1
-        elif len(parts) == 2:  # /num_experts_per_tok  
-            try:
-                num_experts_per_tok = int(parts[1])  # Extract and convert number to int
-                if num_experts_per_tok < 0 or num_experts_per_tok > 8:
-                    num_experts_per_tok = 1
-                config.num_experts_per_tok = num_experts_per_tok
-                print(f"num_experts_per_tok: {config.num_experts_per_tok}")
-                
-                # Remove all hooks after inference
-                for h in hooks: h.remove()
-                del model
-
-                model = AutoModelForCausalLM.from_pretrained(
-                    NEW_MODEL_ID,
-                    config=config,
-                    device_map="auto",
-                    trust_remote_code=True,
-                    quantization_config=quant_config_4,
-                    torch_dtype=torch.bfloat16
-                )
-                hooks = []
-                for layer in model.model.layers:
-                    hooks.append(layer.mlp.gate.register_forward_hook(hook_fn))
-
-            except ValueError:
-                print("Error: Please provide a valid integer after /same_seed")       
-        continue
-    if user_input.lower() == "/do_sample":
-        do_sample = not do_sample
-        continue
-    if not user_input:
-        print("Input cannot be empty. Please enter something.")
-        continue
-    
-
-    messages.append({"role": "user", "content": user_input})
-    activated_experts = []
-    response, stop_flag, metrics = generate_stream(model, tokenizer, messages, nothink, skip_prompt, skip_special_tokens, do_sample, 40960)
-    print("\n\nMetrics:")
-    for key, value in metrics.items():
-        print(f"  {key}: {value}")
-
-    # Count the frequency of each activated expert
-    expert_counts = Counter(activated_experts)
-
-    # Print activation statistics
-    print("\nActivated Expert Statistics:")
-    for expert_idx, count in sorted(expert_counts.items()):
-        print(f"Expert {expert_idx}: {count} times")
-        
-    print("", flush=True)
-    if stop_flag:
-        continue
-    messages.append({"role": "assistant", "content": response})
-
-# Remove all hooks after inference
-for h in hooks: h.remove()
-```
-
-## Applications
-
- - **Text Generation: Articles**, dialogues, and creative writing.
- - **Question Answering**: Information retrieval and query resolution.
- - **Conversational AI**: Multi-turn dialogues for chatbots.
- - **Research**: Exploration of MoE architectures and efficient model scaling.
-
-## Limitations
-
- - **Fine-Tuning Required**: Randomly initialized gating weights may lead to suboptimal expert utilization without fine-tuning.
- - **Compatibility**: Developed with transformers 4.52.4; ensure matching versions to avoid loading issues.
- - **Inference Speed**: While efficient for an MoE model, performance depends on hardware (GPU recommended).
-
-## Ethical Considerations
-
- - **Bias**: Inherits potential biases from the Qwen3-4B base model; users should evaluate outputs for fairness.
- - **Usage**: Intended for research and responsible applications; avoid generating harmful or misleading content.
-
-## Contact
-
- - **Developer**: huihui.ai
- - **Repository**: huihui-ai/Huihui-MoE-23B-A4B (available locally or on Hugging Face)
- - **Issues**: Report bugs or request features via the repository or please send an email to [email protected]
-
-## Acknowledgments
-
- - Built upon the Qwen3-4B model by the Qwen team.
- - Built upon the Experts model by the Suayptalha team.
- - Powered by the Hugging Face transformers library.
+---
+license: mit
+---

added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,85 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
+        {%- set ns.multi_step_tool = false %}
+        {%- set ns.last_query_index = index %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set content = message.content %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in message.content %}
+                {%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
+                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {%- if loop.last or (not loop.last and reasoning_content) %}
+                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
+            {%- else %}
+                {{- '<|im_start|>' + message.role + '\n' + content }}
+            {%- endif %}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,38 @@
+{
+  "architectures": [
+    "Qwen3MoeForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 151643,
+  "decoder_sparse_step": 1,
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2560,
+  "initializer_range": 0.02,
+  "intermediate_size": 9728,
+  "max_position_embeddings": 40960,
+  "max_window_layers": 36,
+  "mlp_only_layers": [],
+  "model_type": "qwen3_moe",
+  "moe_intermediate_size": 9728,
+  "norm_topk_prob": true,
+  "num_attention_heads": 32,
+  "num_experts": 8,
+  "num_experts_per_tok": 1,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 8,
+  "output_router_logits": false,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "router_aux_loss_coef": 0.001,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.52.4",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 151643,
+  "eos_token_id": 151645,
+  "transformers_version": "4.52.4"
+}

model-00001-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:108558a87411ffdf04ee2ae00f589e40c0264a287a5bf627394e0be55db8fb97
+size 4972426800

model-00002-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82d2f7305d4bcdeb481b89bbc07991ea8121ff8f9f63b07992a76aec6ac6fae2
+size 4991444528

model-00003-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3e689f6cc9d214d357503857dd4c6973ba93a94122327e8aed1f2327041e5ef
+size 4991444584

model-00004-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f4060dcb05cd9f3c30e928c2ec694fef66c7430db4cbd0759418316b3283237
+size 4991444664

model-00005-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e7579e21c2eaabe9abe22fe8136098e9c6cab3d18203ad1104daedf009924310
+size 4991444664

model-00006-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab9f89892b62d88d364ae1fb342ff388c290c75724d87ce251aac59ff70fd82b
+size 4991444664

model-00007-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b1e10317fc026397ee661e4fe83237a0d2ef8ec36731594b107c547eeff1549
+size 4991444664

model-00008-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5835dc2c682d66f48d79b071ec356e253bd85da595a3b37742284ded47d85b7f
+size 4991444664

model-00009-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e944a16a6a4af030af8622f4e21205d8166baa0f6b28e465d2d83c157bc224e2
+size 4991444664

model-00010-of-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02654f8a8de6074a0666d4ccc634ec208e997a56a7f016baf3696658f703b4a5
+size 1574847872

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
+size 11422654

tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}