apepkuss79 commited on
Commit
9fd9272
·
verified ·
1 Parent(s): dffa906

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +62 -0
README.md ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: openai/gpt-oss-120b
3
+ license: apache-2.0
4
+ model_creator: openai
5
+ model_name: gpt-oss-120b
6
+ quantized_by: Second State Inc.
7
+ pipeline_tag: text-generation
8
+ library_name: transformers
9
+ ---
10
+
11
+ <!-- header start -->
12
+ <!-- 200823 -->
13
+ <div style="width: auto; margin-left: auto; margin-right: auto">
14
+ <img src="https://github.com/LlamaEdge/LlamaEdge/raw/dev/assets/logo.svg" style="width: 100%; min-width: 400px; display: block; margin: auto;">
15
+ </div>
16
+ <hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
17
+ <!-- header end -->
18
+
19
+ # gpt-oss-120b-GGUF
20
+
21
+ ## Original Model
22
+
23
+ [openai/gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b)
24
+
25
+ ## Run with LlamaEdge
26
+
27
+ - LlamaEdge version: [v0.25.0](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.25.0) and above
28
+
29
+ - Prompt template
30
+
31
+ - Prompt type: `gpt-oss`
32
+
33
+ - Prompt string
34
+
35
+ ```text
36
+ <|start|>system<|message|>
37
+ You are ChatGPT, a large language model trained by OpenAI.
38
+ Knowledge cutoff: 2024-06
39
+ Current date: 2025-08-06
40
+ Reasoning: medium
41
+ # Valid channels: analysis, commentary, final. Channel must be included for every message.
42
+ <|end|>
43
+
44
+ <|start|>user<|message|>Hello!<|end|>
45
+ <|start|>assistant<|channel|>final<|message|>Hi there!<|end|>
46
+ <|start|>user<|message|>What's your favorite color?<|end|>
47
+ <|start|>assistant
48
+ ```
49
+
50
+ - Context size: `128000`
51
+
52
+ - Run as LlamaEdge service
53
+
54
+ ```bash
55
+ wasmedge --dir .:. --nn-preload default:GGML:AUTO:gpt-oss-120b-MXFP4_MOE.gguf \
56
+ llama-api-server.wasm \
57
+ --model-name gpt-oss-120b \
58
+ --prompt-template gpt-oss \
59
+ --ctx-size 128000
60
+ ```
61
+
62
+ *Quantized with llama.cpp b6115*