---
base_model: THUDM/GLM-Z1-9B-0414
language:
- zh
- en
library_name: transformers
license: mit
pipeline_tag: text-generation
tags:
- llama-cpp
- gguf-my-repo
---

# Triangle104/GLM-Z1-9B-0414-Q8_0-GGUF
This model was converted to GGUF format from [`THUDM/GLM-Z1-9B-0414`](https://huggingface.co/THUDM/GLM-Z1-9B-0414) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/THUDM/GLM-Z1-9B-0414) for more details on the model.

---
Introduction
-
The GLM family welcomes a new generation of open-source models, the GLM-4-32B-0414
 series, featuring 32 billion parameters. Its performance is comparable 
to OpenAI's GPT series and DeepSeek's V3/R1 series, and it supports very
 user-friendly local deployment features. GLM-4-32B-Base-0414 was 
pre-trained on 15T of high-quality data, including a large amount of 
reasoning-type synthetic data, laying the foundation for subsequent 
reinforcement learning extensions. In the post-training stage, in 
addition to human preference alignment for dialogue scenarios, we also 
enhanced the model's performance in instruction following, engineering 
code, and function calling using techniques such as rejection sampling 
and reinforcement learning, strengthening the atomic capabilities 
required for agent tasks. GLM-4-32B-0414 achieves good results in areas 
such as engineering code, Artifact generation, function calling, 
search-based Q&A, and report generation. Some benchmarks even rival 
larger models like GPT-4o and DeepSeek-V3-0324 (671B).

GLM-Z1-9B-0414 is a surprise. We employed the 
aforementioned series of techniques to train a 9B small-sized model that
 maintains the open-source tradition. Despite its smaller scale, 
GLM-Z1-9B-0414 still exhibits excellent capabilities in mathematical 
reasoning and general tasks. Its overall performance is already at a 
leading level among open-source models of the same size. Especially in 
resource-constrained scenarios, this model achieves an excellent balance
 between efficiency and effectiveness, providing a powerful option for 
users seeking lightweight deployment

---
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo Triangle104/GLM-Z1-9B-0414-Q8_0-GGUF --hf-file glm-z1-9b-0414-q8_0.gguf -p "The meaning to life and the universe is"
```

### Server:
```bash
llama-server --hf-repo Triangle104/GLM-Z1-9B-0414-Q8_0-GGUF --hf-file glm-z1-9b-0414-q8_0.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Triangle104/GLM-Z1-9B-0414-Q8_0-GGUF --hf-file glm-z1-9b-0414-q8_0.gguf -p "The meaning to life and the universe is"
```
or 
```
./llama-server --hf-repo Triangle104/GLM-Z1-9B-0414-Q8_0-GGUF --hf-file glm-z1-9b-0414-q8_0.gguf -c 2048
```