Image-Text-to-Text
Transformers
Safetensors
English
gemma4
text-generation-inference
unsloth
trl
conversational
Liontix commited on
Commit
ebe0887
Β·
verified Β·
1 Parent(s): fb60ffa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +115 -7
README.md CHANGED
@@ -19,15 +19,123 @@ datasets:
19
 
20
  The following model was trained on claude-code traces, with some chat data provided by the community.
21
 
22
- I recommend using the model with claude-code or pi, though other harnesses should work without issue.
23
 
24
- Reasoning was left un-touched.
25
 
26
- Benchmarks coming soon.
27
-
28
- ---
29
  The data for this model was easily extracted, formatted, and masked for training with [Teich](https://github.com/TeichAI/teich) <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/6837935ac3b7ffe0d2559ce9/-AxyvV4wfUY8uo87kNKkK.png" width="20" height="20" style="display: inline-block; vertical-align: middle; margin: 0 3px;">
30
 
31
- This model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
32
 
33
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  The following model was trained on claude-code traces, with some chat data provided by the community.
21
 
22
+ I recommend using the model with claude-code or pi, though other harnesses should work without issues.
23
 
 
24
 
 
 
 
25
  The data for this model was easily extracted, formatted, and masked for training with [Teich](https://github.com/TeichAI/teich) <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/6837935ac3b7ffe0d2559ce9/-AxyvV4wfUY8uo87kNKkK.png" width="20" height="20" style="display: inline-block; vertical-align: middle; margin: 0 3px;">
26
 
 
27
 
28
+ ## πŸ“‹ Stage Details & Benchmarks
29
+
30
+ *Reasoning was left un-touched*
31
+
32
+ *Benchmarks coming soon*
33
+
34
+ > **Deep Dive Analysis:** For more comprehensive insights regarding the base capabilities of the Gemma 4 architecture, please refer to [this Analysis Document](https://huggingface.co/TeichAI/gemma-4-31B-it-Claude-Opus-Distill/resolve/main/Gemma%204%20Analysis.pdf).
35
+
36
+
37
+ ## 🌟 Core Skills & Capabilities
38
+
39
+ Thanks to its robust base model and high-effort reasoning distillation, this model is highly optimized for the following use cases:
40
+
41
+ 1. **πŸ’» Coding:** Advanced code generation, debugging, and software architecture planning.
42
+ 2. **πŸ”¬ Science:** Deep scientific reasoning, hypothesis evaluation, and analytical problem-solving.
43
+ 3. **πŸ”Ž Deep Research:** Navigating complex, multi-step research queries and synthesizing vast amounts of information.
44
+ 4. **🧠 General Purpose:** Highly capable instruction-following for everyday tasks requiring high logical coherence.
45
+
46
+ ## Getting Started
47
+
48
+ You can use all Gemma 4 models with the latest version of Transformers. To get started, install the necessary dependencies in your environment:
49
+
50
+ `pip install -U transformers torch accelerate`
51
+
52
+ Once you have everything installed, you can proceed to load the model with the code below:
53
+
54
+ ```python
55
+ from transformers import AutoProcessor, AutoModelForCausalLM
56
+
57
+ MODEL_ID = "google/gemma-4-31B-it"
58
+
59
+ # Load model
60
+ processor = AutoProcessor.from_pretrained(MODEL_ID)
61
+ model = AutoModelForCausalLM.from_pretrained(
62
+ MODEL_ID,
63
+ dtype="auto",
64
+ device_map="auto"
65
+ )
66
+ ```
67
+
68
+ Once the model is loaded, you can start generating output:
69
+
70
+ ```python
71
+ # Prompt
72
+ messages = [
73
+ {"role": "system", "content": "You are a helpful assistant."},
74
+ {"role": "user", "content": "Write a short joke about saving RAM."},
75
+ ]
76
+
77
+ # Process input
78
+ text = processor.apply_chat_template(
79
+ messages,
80
+ tokenize=False,
81
+ add_generation_prompt=True,
82
+ enable_thinking=False
83
+ )
84
+ inputs = processor(text=text, return_tensors="pt").to(model.device)
85
+ input_len = inputs["input_ids"].shape[-1]
86
+
87
+ # Generate output
88
+ outputs = model.generate(**inputs, max_new_tokens=1024)
89
+ response = processor.decode(outputs[0][input_len:], skip_special_tokens=False)
90
+
91
+ # Parse output
92
+ processor.parse_response(response)
93
+ ```
94
+
95
+ To enable reasoning, set `enable_thinking=True` and the `parse_response` function will take care of parsing the thinking output.
96
+
97
+ ## **Best Practices**
98
+
99
+ For the best performance, use these configurations and best practices:
100
+
101
+ ### 1. Sampling Parameters
102
+
103
+ Use the following standardized sampling configuration across all use cases:
104
+
105
+ * `temperature=1.0`
106
+ * `top_p=0.95`
107
+ * `top_k=64`
108
+
109
+ ### 2. Thinking Mode Configuration
110
+
111
+ Compared to Gemma 3, the models use standard `system`, `assistant`, and `user` roles. To properly manage the thinking process, use the following control tokens:
112
+
113
+ * **Trigger Thinking:** Thinking is enabled by including the `<|think|>` token at the start of the system prompt. To disable thinking, remove the token.
114
+ * **Standard Generation:** When thinking is enabled, the model will output its internal reasoning followed by the final answer using this structure:
115
+ `<|channel>thought\n`**[Internal reasoning]**`<channel|>`
116
+ * **Disabled Thinking Behavior:** For all models except for the E2B and E4B variants, if thinking is disabled, the model will still generate the tags but with an empty thought block:
117
+ `<|channel>thought\n<channel|>`**[Final answer]**
118
+
119
+ > [!Note]
120
+ > Note that many libraries like Transformers and llama.cpp handle the complexities of the chat template for you.
121
+
122
+ ## πŸ™ Acknowledgements
123
+
124
+ - **Google**: For providing an exceptional open weights model. Read more about Gemma 4 on the [Google Innovation Blog](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/).
125
+ - **Unsloth**: For assembling ready-to-use, cutting-edge fine-tuning environments that make this work possible.
126
+ - **PawanKrd**, **victor** and **armand0e**: For creating and sharing their awesome Fable datasets with the community.
127
+
128
+
129
+ ## πŸ“– Citation
130
+
131
+ If you use this model in your research or projects, please cite:
132
+
133
+ ```bibtex
134
+ @misc{teichai_gemma4_31b_fable_5_agent_distilled,
135
+ title = {TeichAI/Gemma-4-31B-Fable-5-Agent-Distill},
136
+ author = {TeichAI},
137
+ year = {2026},
138
+ publisher = {Hugging Face},
139
+ howpublished = {\url{https://huggingface.co/TeichAI/Gemma-4-31B-Fable-5-Agent-Distill}}
140
+ }
141
+ ```