Hugging Face's logo

nightmedia
/
gpt-oss-120b-heretic-v2-mxfp4-q8-hi-mlx

Text Generation
MLX
Safetensors
gpt_oss
vllm
heretic
uncensored
decensored
abliterated
mxfp4
conversational
4-bit precision
Model card Files
xet
Community
1
nightmedia commited on
Commit
27a206d
·
verified ·
1 Parent(s): 946f7c6

Add files using upload-large-folder tool

Browse files
Files changed (22) hide show
  1. .gitattributes +1 -0
  2. README.md +42 -0
  3. chat_template.jinja +331 -0
  4. config.json +1838 -0
  5. generation_config.json +11 -0
  6. model-00001-of-00013.safetensors +3 -0
  7. model-00002-of-00013.safetensors +3 -0
  8. model-00003-of-00013.safetensors +3 -0
  9. model-00004-of-00013.safetensors +3 -0
  10. model-00005-of-00013.safetensors +3 -0
  11. model-00006-of-00013.safetensors +3 -0
  12. model-00007-of-00013.safetensors +3 -0
  13. model-00008-of-00013.safetensors +3 -0
  14. model-00009-of-00013.safetensors +3 -0
  15. model-00010-of-00013.safetensors +3 -0
  16. model-00011-of-00013.safetensors +3 -0
  17. model-00012-of-00013.safetensors +3 -0
  18. model-00013-of-00013.safetensors +3 -0
  19. model.safetensors.index.json +0 -0
  20. special_tokens_map.json +23 -0
  21. tokenizer.json +3 -0
  22. tokenizer_config.json +187 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: text-generation
4
+ library_name: mlx
5
+ tags:
6
+ - vllm
7
+ - heretic
8
+ - uncensored
9
+ - decensored
10
+ - abliterated
11
+ - mxfp4
12
+ - mlx
13
+ base_model: kldzj/gpt-oss-120b-heretic-v2
14
+ ---
15
+
16
+ # gpt-oss-120b-heretic-v2-mxfp4-q8-hi-mlx
17
+
18
+ This model [gpt-oss-120b-heretic-v2-mxfp4-q8-hi-mlx](https://huggingface.co/gpt-oss-120b-heretic-v2-mxfp4-q8-hi-mlx) was
19
+ converted to MLX format from [kldzj/gpt-oss-120b-heretic-v2](https://huggingface.co/kldzj/gpt-oss-120b-heretic-v2)
20
+ using mlx-lm version **0.28.4**.
21
+
22
+ ## Use with mlx
23
+
24
+ ```bash
25
+ pip install mlx-lm
26
+ ```
27
+
28
+ ```python
29
+ from mlx_lm import load, generate
30
+
31
+ model, tokenizer = load("gpt-oss-120b-heretic-v2-mxfp4-q8-hi-mlx")
32
+
33
+ prompt = "hello"
34
+
35
+ if tokenizer.chat_template is not None:
36
+ messages = [{"role": "user", "content": prompt}]
37
+ prompt = tokenizer.apply_chat_template(
38
+ messages, add_generation_prompt=True
39
+ )
40
+
41
+ response = generate(model, tokenizer, prompt=prompt, verbose=True)
42
+ ```
chat_template.jinja ADDED
@@ -0,0 +1,331 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {#-
2
+ In addition to the normal inputs of `messages` and `tools`, this template also accepts the
3
+ following kwargs:
4
+ - "builtin_tools": A list, can contain "browser" and/or "python".
5
+ - "model_identity": A string that optionally describes the model identity.
6
+ - "reasoning_effort": A string that describes the reasoning effort, defaults to "medium".
7
+ #}
8
+
9
+ {#- Tool Definition Rendering ============================================== #}
10
+ {%- macro render_typescript_type(param_spec, required_params, is_nullable=false) -%}
11
+ {%- if param_spec.type == "array" -%}
12
+ {%- if param_spec['items'] -%}
13
+ {%- if param_spec['items']['type'] == "string" -%}
14
+ {{- "string[]" }}
15
+ {%- elif param_spec['items']['type'] == "number" -%}
16
+ {{- "number[]" }}
17
+ {%- elif param_spec['items']['type'] == "integer" -%}
18
+ {{- "number[]" }}
19
+ {%- elif param_spec['items']['type'] == "boolean" -%}
20
+ {{- "boolean[]" }}
21
+ {%- else -%}
22
+ {%- set inner_type = render_typescript_type(param_spec['items'], required_params) -%}
23
+ {%- if inner_type == "object | object" or inner_type|length > 50 -%}
24
+ {{- "any[]" }}
25
+ {%- else -%}
26
+ {{- inner_type + "[]" }}
27
+ {%- endif -%}
28
+ {%- endif -%}
29
+ {%- if param_spec.nullable -%}
30
+ {{- " | null" }}
31
+ {%- endif -%}
32
+ {%- else -%}
33
+ {{- "any[]" }}
34
+ {%- if param_spec.nullable -%}
35
+ {{- " | null" }}
36
+ {%- endif -%}
37
+ {%- endif -%}
38
+ {%- elif param_spec.type is defined and param_spec.type is iterable and param_spec.type is not string and param_spec.type is not mapping and param_spec.type[0] is defined -%}
39
+ {#- Handle array of types like ["object", "object"] from Union[dict, list] #}
40
+ {%- if param_spec.type | length > 1 -%}
41
+ {{- param_spec.type | join(" | ") }}
42
+ {%- else -%}
43
+ {{- param_spec.type[0] }}
44
+ {%- endif -%}
45
+ {%- elif param_spec.oneOf -%}
46
+ {#- Handle oneOf schemas - check for complex unions and fallback to any #}
47
+ {%- set has_object_variants = false -%}
48
+ {%- for variant in param_spec.oneOf -%}
49
+ {%- if variant.type == "object" -%}
50
+ {%- set has_object_variants = true -%}
51
+ {%- endif -%}
52
+ {%- endfor -%}
53
+ {%- if has_object_variants and param_spec.oneOf|length > 1 -%}
54
+ {{- "any" }}
55
+ {%- else -%}
56
+ {%- for variant in param_spec.oneOf -%}
57
+ {{- render_typescript_type(variant, required_params) -}}
58
+ {%- if variant.description %}
59
+ {{- "// " + variant.description }}
60
+ {%- endif -%}
61
+ {%- if variant.default is defined %}
62
+ {{ "// default: " + variant.default|tojson }}
63
+ {%- endif -%}
64
+ {%- if not loop.last %}
65
+ {{- " | " }}
66
+ {% endif -%}
67
+ {%- endfor -%}
68
+ {%- endif -%}
69
+ {%- elif param_spec.type == "string" -%}
70
+ {%- if param_spec.enum -%}
71
+ {{- '"' + param_spec.enum|join('" | "') + '"' -}}
72
+ {%- else -%}
73
+ {{- "string" }}
74
+ {%- if param_spec.nullable %}
75
+ {{- " | null" }}
76
+ {%- endif -%}
77
+ {%- endif -%}
78
+ {%- elif param_spec.type == "number" -%}
79
+ {{- "number" }}
80
+ {%- elif param_spec.type == "integer" -%}
81
+ {{- "number" }}
82
+ {%- elif param_spec.type == "boolean" -%}
83
+ {{- "boolean" }}
84
+
85
+ {%- elif param_spec.type == "object" -%}
86
+ {%- if param_spec.properties -%}
87
+ {{- "{\n" }}
88
+ {%- for prop_name, prop_spec in param_spec.properties.items() -%}
89
+ {{- prop_name -}}
90
+ {%- if prop_name not in (param_spec.required or []) -%}
91
+ {{- "?" }}
92
+ {%- endif -%}
93
+ {{- ": " }}
94
+ {{ render_typescript_type(prop_spec, param_spec.required or []) }}
95
+ {%- if not loop.last -%}
96
+ {{-", " }}
97
+ {%- endif -%}
98
+ {%- endfor -%}
99
+ {{- "}" }}
100
+ {%- else -%}
101
+ {{- "object" }}
102
+ {%- endif -%}
103
+ {%- else -%}
104
+ {{- "any" }}
105
+ {%- endif -%}
106
+ {%- endmacro -%}
107
+
108
+ {%- macro render_tool_namespace(namespace_name, tools) -%}
109
+ {{- "## " + namespace_name + "\n\n" }}
110
+ {{- "namespace " + namespace_name + " {\n\n" }}
111
+ {%- for tool in tools %}
112
+ {%- set tool = tool.function %}
113
+ {{- "// " + tool.description + "\n" }}
114
+ {{- "type "+ tool.name + " = " }}
115
+ {%- if tool.parameters and tool.parameters.properties %}
116
+ {{- "(_: {\n" }}
117
+ {%- for param_name, param_spec in tool.parameters.properties.items() %}
118
+ {%- if param_spec.description %}
119
+ {{- "// " + param_spec.description + "\n" }}
120
+ {%- endif %}
121
+ {{- param_name }}
122
+ {%- if param_name not in (tool.parameters.required or []) -%}
123
+ {{- "?" }}
124
+ {%- endif -%}
125
+ {{- ": " }}
126
+ {{- render_typescript_type(param_spec, tool.parameters.required or []) }}
127
+ {%- if param_spec.default is defined -%}
128
+ {%- if param_spec.enum %}
129
+ {{- ", // default: " + param_spec.default }}
130
+ {%- elif param_spec.oneOf %}
131
+ {{- "// default: " + param_spec.default }}
132
+ {%- else %}
133
+ {{- ", // default: " + param_spec.default|tojson }}
134
+ {%- endif -%}
135
+ {%- endif -%}
136
+ {%- if not loop.last %}
137
+ {{- ",\n" }}
138
+ {%- else %}
139
+ {{- ",\n" }}
140
+ {%- endif -%}
141
+ {%- endfor %}
142
+ {{- "}) => any;\n\n" }}
143
+ {%- else -%}
144
+ {{- "() => any;\n\n" }}
145
+ {%- endif -%}
146
+ {%- endfor %}
147
+ {{- "} // namespace " + namespace_name }}
148
+ {%- endmacro -%}
149
+
150
+ {%- macro render_builtin_tools(browser_tool, python_tool) -%}
151
+ {%- if browser_tool %}
152
+ {{- "## browser\n\n" }}
153
+ {{- "// Tool for browsing.\n" }}
154
+ {{- "// The `cursor` appears in brackets before each browsing display: `[{cursor}]`.\n" }}
155
+ {{- "// Cite information from the tool using the following format:\n" }}
156
+ {{- "// `【{cursor}†L{line_start}(-L{line_end})?】`, for example: `【6†L9-L11】` or `【8†L3】`.\n" }}
157
+ {{- "// Do not quote more than 10 words directly from the tool output.\n" }}
158
+ {{- "// sources=web (default: web)\n" }}
159
+ {{- "namespace browser {\n\n" }}
160
+ {{- "// Searches for information related to `query` and displays `topn` results.\n" }}
161
+ {{- "type search = (_: {\n" }}
162
+ {{- "query: string,\n" }}
163
+ {{- "topn?: number, // default: 10\n" }}
164
+ {{- "source?: string,\n" }}
165
+ {{- "}) => any;\n\n" }}
166
+ {{- "// Opens the link `id` from the page indicated by `cursor` starting at line number `loc`, showing `num_lines` lines.\n" }}
167
+ {{- "// Valid link ids are displayed with the formatting: `【{id}†.*】`.\n" }}
168
+ {{- "// If `cursor` is not provided, the most recent page is implied.\n" }}
169
+ {{- "// If `id` is a string, it is treated as a fully qualified URL associated with `source`.\n" }}
170
+ {{- "// If `loc` is not provided, the viewport will be positioned at the beginning of the document or centered on the most relevant passage, if available.\n" }}
171
+ {{- "// Use this function without `id` to scroll to a new location of an opened page.\n" }}
172
+ {{- "type open = (_: {\n" }}
173
+ {{- "id?: number | string, // default: -1\n" }}
174
+ {{- "cursor?: number, // default: -1\n" }}
175
+ {{- "loc?: number, // default: -1\n" }}
176
+ {{- "num_lines?: number, // default: -1\n" }}
177
+ {{- "view_source?: boolean, // default: false\n" }}
178
+ {{- "source?: string,\n" }}
179
+ {{- "}) => any;\n\n" }}
180
+ {{- "// Finds exact matches of `pattern` in the current page, or the page given by `cursor`.\n" }}
181
+ {{- "type find = (_: {\n" }}
182
+ {{- "pattern: string,\n" }}
183
+ {{- "cursor?: number, // default: -1\n" }}
184
+ {{- "}) => any;\n\n" }}
185
+ {{- "} // namespace browser\n\n" }}
186
+ {%- endif -%}
187
+
188
+ {%- if python_tool %}
189
+ {{- "## python\n\n" }}
190
+ {{- "Use this tool to execute Python code in your chain of thought. The code will not be shown to the user. This tool should be used for internal reasoning, but not for code that is intended to be visible to the user (e.g. when creating plots, tables, or files).\n\n" }}
191
+ {{- "When you send a message containing Python code to python, it will be executed in a stateful Jupyter notebook environment. python will respond with the output of the execution or time out after 120.0 seconds. The drive at '/mnt/data' can be used to save and persist user files. Internet access for this session is UNKNOWN. Depends on the cluster.\n\n" }}
192
+ {%- endif -%}
193
+ {%- endmacro -%}
194
+
195
+ {#- System Message Construction ============================================ #}
196
+ {%- macro build_system_message() -%}
197
+ {%- if model_identity is not defined %}
198
+ {%- set model_identity = "You are ChatGPT, a large language model trained by OpenAI." %}
199
+ {%- endif %}
200
+ {{- model_identity + "\n" }}
201
+ {{- "Knowledge cutoff: 2024-06\n" }}
202
+ {{- "Current date: " + strftime_now("%Y-%m-%d") + "\n\n" }}
203
+ {%- if reasoning_effort is not defined %}
204
+ {%- set reasoning_effort = "medium" %}
205
+ {%- endif %}
206
+ {{- "Reasoning: " + reasoning_effort + "\n\n" }}
207
+ {%- if builtin_tools %}
208
+ {{- "# Tools\n\n" }}
209
+ {%- set available_builtin_tools = namespace(browser=false, python=false) %}
210
+ {%- for tool in builtin_tools %}
211
+ {%- if tool == "browser" %}
212
+ {%- set available_builtin_tools.browser = true %}
213
+ {%- elif tool == "python" %}
214
+ {%- set available_builtin_tools.python = true %}
215
+ {%- endif %}
216
+ {%- endfor %}
217
+ {{- render_builtin_tools(available_builtin_tools.browser, available_builtin_tools.python) }}
218
+ {%- endif -%}
219
+ {{- "# Valid channels: analysis, commentary, final. Channel must be included for every message." }}
220
+ {%- if tools -%}
221
+ {{- "\nCalls to these tools must go to the commentary channel: 'functions'." }}
222
+ {%- endif -%}
223
+ {%- endmacro -%}
224
+
225
+ {#- Main Template Logic ================================================= #}
226
+ {#- Set defaults #}
227
+
228
+ {#- Render system message #}
229
+ {{- "<|start|>system<|message|>" }}
230
+ {{- build_system_message() }}
231
+ {{- "<|end|>" }}
232
+
233
+ {#- Extract developer message #}
234
+ {%- if messages[0].role == "developer" or messages[0].role == "system" %}
235
+ {%- set developer_message = messages[0].content %}
236
+ {%- set loop_messages = messages[1:] %}
237
+ {%- else %}
238
+ {%- set developer_message = "" %}
239
+ {%- set loop_messages = messages %}
240
+ {%- endif %}
241
+
242
+ {#- Render developer message #}
243
+ {%- if developer_message or tools %}
244
+ {{- "<|start|>developer<|message|>" }}
245
+ {%- if developer_message %}
246
+ {{- "# Instructions\n\n" }}
247
+ {{- developer_message }}
248
+ {{- "\n\n" }}
249
+ {%- endif %}
250
+ {%- if tools -%}
251
+ {{- "# Tools\n\n" }}
252
+ {{- render_tool_namespace("functions", tools) }}
253
+ {%- endif -%}
254
+ {{- "<|end|>" }}
255
+ {%- endif %}
256
+
257
+ {#- Render messages #}
258
+ {%- set last_tool_call = namespace(name=none) %}
259
+ {%- for message in loop_messages -%}
260
+ {#- At this point only assistant/user/tool messages should remain #}
261
+ {%- if message.role == 'assistant' -%}
262
+ {#- Checks to ensure the messages are being passed in the format we expect #}
263
+ {%- if "content" in message %}
264
+ {%- if "<|channel|>analysis<|message|>" in message.content or "<|channel|>final<|message|>" in message.content %}
265
+ {{- raise_exception("You have passed a message containing <|channel|> tags in the content field. Instead of doing this, you should pass analysis messages (the string between '<|message|>' and '<|end|>') in the 'thinking' field, and final messages (the string between '<|message|>' and '<|end|>') in the 'content' field.") }}
266
+ {%- endif %}
267
+ {%- endif %}
268
+ {%- if "thinking" in message %}
269
+ {%- if "<|channel|>analysis<|message|>" in message.thinking or "<|channel|>final<|message|>" in message.thinking %}
270
+ {{- raise_exception("You have passed a message containing <|channel|> tags in the thinking field. Instead of doing this, you should pass analysis messages (the string between '<|message|>' and '<|end|>') in the 'thinking' field, and final messages (the string between '<|message|>' and '<|end|>') in the 'content' field.") }}
271
+ {%- endif %}
272
+ {%- endif %}
273
+ {%- if "tool_calls" in message %}
274
+ {#- We need very careful handling here - we want to drop the tool call analysis message if the model #}
275
+ {#- has output a later <|final|> message, but otherwise we want to retain it. This is the only case #}
276
+ {#- when we render CoT/analysis messages in inference. #}
277
+ {%- set future_final_message = namespace(found=false) %}
278
+ {%- for future_message in loop_messages[loop.index:] %}
279
+ {%- if future_message.role == 'assistant' and "tool_calls" not in future_message %}
280
+ {%- set future_final_message.found = true %}
281
+ {%- endif %}
282
+ {%- endfor %}
283
+ {#- We assume max 1 tool call per message, and so we infer the tool call name #}
284
+ {#- in "tool" messages from the most recent assistant tool call name #}
285
+ {%- set tool_call = message.tool_calls[0] %}
286
+ {%- if tool_call.function %}
287
+ {%- set tool_call = tool_call.function %}
288
+ {%- endif %}
289
+ {%- if message.content and message.thinking %}
290
+ {{- raise_exception("Cannot pass both content and thinking in an assistant message with tool calls! Put the analysis message in one or the other, but not both.") }}
291
+ {%- elif message.content and not future_final_message.found %}
292
+ {{- "<|start|>assistant<|channel|>analysis<|message|>" + message.content + "<|end|>" }}
293
+ {%- elif message.thinking and not future_final_message.found %}
294
+ {{- "<|start|>assistant<|channel|>analysis<|message|>" + message.thinking + "<|end|>" }}
295
+ {%- endif %}
296
+ {{- "<|start|>assistant to=" }}
297
+ {{- "functions." + tool_call.name + "<|channel|>commentary " }}
298
+ {{- (tool_call.content_type if tool_call.content_type is defined else "json") + "<|message|>" }}
299
+ {{- tool_call.arguments|tojson }}
300
+ {{- "<|call|>" }}
301
+ {%- set last_tool_call.name = tool_call.name %}
302
+ {%- elif loop.last and not add_generation_prompt %}
303
+ {#- Only render the CoT if the final turn is an assistant turn and add_generation_prompt is false #}
304
+ {#- This is a situation that should only occur in training, never in inference. #}
305
+ {%- if "thinking" in message %}
306
+ {{- "<|start|>assistant<|channel|>analysis<|message|>" + message.thinking + "<|end|>" }}
307
+ {%- endif %}
308
+ {#- <|return|> indicates the end of generation, but <|end|> does not #}
309
+ {#- <|return|> should never be an input to the model, but we include it as the final token #}
310
+ {#- when training, so the model learns to emit it. #}
311
+ {{- "<|start|>assistant<|channel|>final<|message|>" + message.content + "<|return|>" }}
312
+ {%- else %}
313
+ {#- CoT is dropped during all previous turns, so we never render it for inference #}
314
+ {{- "<|start|>assistant<|channel|>final<|message|>" + message.content + "<|end|>" }}
315
+ {%- set last_tool_call.name = none %}
316
+ {%- endif %}
317
+ {%- elif message.role == 'tool' -%}
318
+ {%- if last_tool_call.name is none %}
319
+ {{- raise_exception("Message has tool role, but there was no previous assistant message with a tool call!") }}
320
+ {%- endif %}
321
+ {{- "<|start|>functions." + last_tool_call.name }}
322
+ {{- " to=assistant<|channel|>commentary<|message|>" + message.content|tojson + "<|end|>" }}
323
+ {%- elif message.role == 'user' -%}
324
+ {{- "<|start|>user<|message|>" + message.content + "<|end|>" }}
325
+ {%- endif -%}
326
+ {%- endfor -%}
327
+
328
+ {#- Generation prompt #}
329
+ {%- if add_generation_prompt -%}
330
+ <|start|>assistant
331
+ {%- endif -%}
config.json ADDED
@@ -0,0 +1,1838 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "GptOssForCausalLM"
4
+ ],
5
+ "attention_bias": true,
6
+ "attention_dropout": 0.0,
7
+ "dtype": "bfloat16",
8
+ "eos_token_id": 200002,
9
+ "experts_per_token": 4,
10
+ "head_dim": 64,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 2880,
13
+ "initial_context_length": 4096,
14
+ "initializer_range": 0.02,
15
+ "intermediate_size": 2880,
16
+ "layer_types": [
17
+ "sliding_attention",
18
+ "full_attention",
19
+ "sliding_attention",
20
+ "full_attention",
21
+ "sliding_attention",
22
+ "full_attention",
23
+ "sliding_attention",
24
+ "full_attention",
25
+ "sliding_attention",
26
+ "full_attention",
27
+ "sliding_attention",
28
+ "full_attention",
29
+ "sliding_attention",
30
+ "full_attention",
31
+ "sliding_attention",
32
+ "full_attention",
33
+ "sliding_attention",
34
+ "full_attention",
35
+ "sliding_attention",
36
+ "full_attention",
37
+ "sliding_attention",
38
+ "full_attention",
39
+ "sliding_attention",
40
+ "full_attention",
41
+ "sliding_attention",
42
+ "full_attention",
43
+ "sliding_attention",
44
+ "full_attention",
45
+ "sliding_attention",
46
+ "full_attention",
47
+ "sliding_attention",
48
+ "full_attention",
49
+ "sliding_attention",
50
+ "full_attention",
51
+ "sliding_attention",
52
+ "full_attention"
53
+ ],
54
+ "max_position_embeddings": 131072,
55
+ "model_type": "gpt_oss",
56
+ "num_attention_heads": 64,
57
+ "num_experts_per_tok": 4,
58
+ "num_hidden_layers": 36,
59
+ "num_key_value_heads": 8,
60
+ "num_local_experts": 128,
61
+ "output_router_logits": false,
62
+ "pad_token_id": 199999,
63
+ "quantization": {
64
+ "group_size": 32,
65
+ "bits": 4,
66
+ "mode": "mxfp4",
67
+ "model.embed_tokens": {
68
+ "group_size": 32,
69
+ "bits": 8,
70
+ "mode": "affine"
71
+ },
72
+ "model.layers.0.self_attn.q_proj": {
73
+ "group_size": 32,
74
+ "bits": 8,
75
+ "mode": "affine"
76
+ },
77
+ "model.layers.0.self_attn.k_proj": {
78
+ "group_size": 32,
79
+ "bits": 8,
80
+ "mode": "affine"
81
+ },
82
+ "model.layers.0.self_attn.v_proj": {
83
+ "group_size": 32,
84
+ "bits": 8,
85
+ "mode": "affine"
86
+ },
87
+ "model.layers.0.self_attn.o_proj": {
88
+ "group_size": 32,
89
+ "bits": 8,
90
+ "mode": "affine"
91
+ },
92
+ "model.layers.0.mlp.router": {
93
+ "group_size": 64,
94
+ "bits": 8
95
+ },
96
+ "model.layers.1.self_attn.q_proj": {
97
+ "group_size": 32,
98
+ "bits": 8,
99
+ "mode": "affine"
100
+ },
101
+ "model.layers.1.self_attn.k_proj": {
102
+ "group_size": 32,
103
+ "bits": 8,
104
+ "mode": "affine"
105
+ },
106
+ "model.layers.1.self_attn.v_proj": {
107
+ "group_size": 32,
108
+ "bits": 8,
109
+ "mode": "affine"
110
+ },
111
+ "model.layers.1.self_attn.o_proj": {
112
+ "group_size": 32,
113
+ "bits": 8,
114
+ "mode": "affine"
115
+ },
116
+ "model.layers.1.mlp.router": {
117
+ "group_size": 64,
118
+ "bits": 8
119
+ },
120
+ "model.layers.2.self_attn.q_proj": {
121
+ "group_size": 32,
122
+ "bits": 8,
123
+ "mode": "affine"
124
+ },
125
+ "model.layers.2.self_attn.k_proj": {
126
+ "group_size": 32,
127
+ "bits": 8,
128
+ "mode": "affine"
129
+ },
130
+ "model.layers.2.self_attn.v_proj": {
131
+ "group_size": 32,
132
+ "bits": 8,
133
+ "mode": "affine"
134
+ },
135
+ "model.layers.2.self_attn.o_proj": {
136
+ "group_size": 32,
137
+ "bits": 8,
138
+ "mode": "affine"
139
+ },
140
+ "model.layers.2.mlp.router": {
141
+ "group_size": 64,
142
+ "bits": 8
143
+ },
144
+ "model.layers.3.self_attn.q_proj": {
145
+ "group_size": 32,
146
+ "bits": 8,
147
+ "mode": "affine"
148
+ },
149
+ "model.layers.3.self_attn.k_proj": {
150
+ "group_size": 32,
151
+ "bits": 8,
152
+ "mode": "affine"
153
+ },
154
+ "model.layers.3.self_attn.v_proj": {
155
+ "group_size": 32,
156
+ "bits": 8,
157
+ "mode": "affine"
158
+ },
159
+ "model.layers.3.self_attn.o_proj": {
160
+ "group_size": 32,
161
+ "bits": 8,
162
+ "mode": "affine"
163
+ },
164
+ "model.layers.3.mlp.router": {
165
+ "group_size": 64,
166
+ "bits": 8
167
+ },
168
+ "model.layers.4.self_attn.q_proj": {
169
+ "group_size": 32,
170
+ "bits": 8,
171
+ "mode": "affine"
172
+ },
173
+ "model.layers.4.self_attn.k_proj": {
174
+ "group_size": 32,
175
+ "bits": 8,
176
+ "mode": "affine"
177
+ },
178
+ "model.layers.4.self_attn.v_proj": {
179
+ "group_size": 32,
180
+ "bits": 8,
181
+ "mode": "affine"
182
+ },
183
+ "model.layers.4.self_attn.o_proj": {
184
+ "group_size": 32,
185
+ "bits": 8,
186
+ "mode": "affine"
187
+ },
188
+ "model.layers.4.mlp.router": {
189
+ "group_size": 64,
190
+ "bits": 8
191
+ },
192
+ "model.layers.5.self_attn.q_proj": {
193
+ "group_size": 32,
194
+ "bits": 8,
195
+ "mode": "affine"
196
+ },
197
+ "model.layers.5.self_attn.k_proj": {
198
+ "group_size": 32,
199
+ "bits": 8,
200
+ "mode": "affine"
201
+ },
202
+ "model.layers.5.self_attn.v_proj": {
203
+ "group_size": 32,
204
+ "bits": 8,
205
+ "mode": "affine"
206
+ },
207
+ "model.layers.5.self_attn.o_proj": {
208
+ "group_size": 32,
209
+ "bits": 8,
210
+ "mode": "affine"
211
+ },
212
+ "model.layers.5.mlp.router": {
213
+ "group_size": 64,
214
+ "bits": 8
215
+ },
216
+ "model.layers.6.self_attn.q_proj": {
217
+ "group_size": 32,
218
+ "bits": 8,
219
+ "mode": "affine"
220
+ },
221
+ "model.layers.6.self_attn.k_proj": {
222
+ "group_size": 32,
223
+ "bits": 8,
224
+ "mode": "affine"
225
+ },
226
+ "model.layers.6.self_attn.v_proj": {
227
+ "group_size": 32,
228
+ "bits": 8,
229
+ "mode": "affine"
230
+ },
231
+ "model.layers.6.self_attn.o_proj": {
232
+ "group_size": 32,
233
+ "bits": 8,
234
+ "mode": "affine"
235
+ },
236
+ "model.layers.6.mlp.router": {
237
+ "group_size": 64,
238
+ "bits": 8
239
+ },
240
+ "model.layers.7.self_attn.q_proj": {
241
+ "group_size": 32,
242
+ "bits": 8,
243
+ "mode": "affine"
244
+ },
245
+ "model.layers.7.self_attn.k_proj": {
246
+ "group_size": 32,
247
+ "bits": 8,
248
+ "mode": "affine"
249
+ },
250
+ "model.layers.7.self_attn.v_proj": {
251
+ "group_size": 32,
252
+ "bits": 8,
253
+ "mode": "affine"
254
+ },
255
+ "model.layers.7.self_attn.o_proj": {
256
+ "group_size": 32,
257
+ "bits": 8,
258
+ "mode": "affine"
259
+ },
260
+ "model.layers.7.mlp.router": {
261
+ "group_size": 64,
262
+ "bits": 8
263
+ },
264
+ "model.layers.8.self_attn.q_proj": {
265
+ "group_size": 32,
266
+ "bits": 8,
267
+ "mode": "affine"
268
+ },
269
+ "model.layers.8.self_attn.k_proj": {
270
+ "group_size": 32,
271
+ "bits": 8,
272
+ "mode": "affine"
273
+ },
274
+ "model.layers.8.self_attn.v_proj": {
275
+ "group_size": 32,
276
+ "bits": 8,
277
+ "mode": "affine"
278
+ },
279
+ "model.layers.8.self_attn.o_proj": {
280
+ "group_size": 32,
281
+ "bits": 8,
282
+ "mode": "affine"
283
+ },
284
+ "model.layers.8.mlp.router": {
285
+ "group_size": 64,
286
+ "bits": 8
287
+ },
288
+ "model.layers.9.self_attn.q_proj": {
289
+ "group_size": 32,
290
+ "bits": 8,
291
+ "mode": "affine"
292
+ },
293
+ "model.layers.9.self_attn.k_proj": {
294
+ "group_size": 32,
295
+ "bits": 8,
296
+ "mode": "affine"
297
+ },
298
+ "model.layers.9.self_attn.v_proj": {
299
+ "group_size": 32,
300
+ "bits": 8,
301
+ "mode": "affine"
302
+ },
303
+ "model.layers.9.self_attn.o_proj": {
304
+ "group_size": 32,
305
+ "bits": 8,
306
+ "mode": "affine"
307
+ },
308
+ "model.layers.9.mlp.router": {
309
+ "group_size": 64,
310
+ "bits": 8
311
+ },
312
+ "model.layers.10.self_attn.q_proj": {
313
+ "group_size": 32,
314
+ "bits": 8,
315
+ "mode": "affine"
316
+ },
317
+ "model.layers.10.self_attn.k_proj": {
318
+ "group_size": 32,
319
+ "bits": 8,
320
+ "mode": "affine"
321
+ },
322
+ "model.layers.10.self_attn.v_proj": {
323
+ "group_size": 32,
324
+ "bits": 8,
325
+ "mode": "affine"
326
+ },
327
+ "model.layers.10.self_attn.o_proj": {
328
+ "group_size": 32,
329
+ "bits": 8,
330
+ "mode": "affine"
331
+ },
332
+ "model.layers.10.mlp.router": {
333
+ "group_size": 64,
334
+ "bits": 8
335
+ },
336
+ "model.layers.11.self_attn.q_proj": {
337
+ "group_size": 32,
338
+ "bits": 8,
339
+ "mode": "affine"
340
+ },
341
+ "model.layers.11.self_attn.k_proj": {
342
+ "group_size": 32,
343
+ "bits": 8,
344
+ "mode": "affine"
345
+ },
346
+ "model.layers.11.self_attn.v_proj": {
347
+ "group_size": 32,
348
+ "bits": 8,
349
+ "mode": "affine"
350
+ },
351
+ "model.layers.11.self_attn.o_proj": {
352
+ "group_size": 32,
353
+ "bits": 8,
354
+ "mode": "affine"
355
+ },
356
+ "model.layers.11.mlp.router": {
357
+ "group_size": 64,
358
+ "bits": 8
359
+ },
360
+ "model.layers.12.self_attn.q_proj": {
361
+ "group_size": 32,
362
+ "bits": 8,
363
+ "mode": "affine"
364
+ },
365
+ "model.layers.12.self_attn.k_proj": {
366
+ "group_size": 32,
367
+ "bits": 8,
368
+ "mode": "affine"
369
+ },
370
+ "model.layers.12.self_attn.v_proj": {
371
+ "group_size": 32,
372
+ "bits": 8,
373
+ "mode": "affine"
374
+ },
375
+ "model.layers.12.self_attn.o_proj": {
376
+ "group_size": 32,
377
+ "bits": 8,
378
+ "mode": "affine"
379
+ },
380
+ "model.layers.12.mlp.router": {
381
+ "group_size": 64,
382
+ "bits": 8
383
+ },
384
+ "model.layers.13.self_attn.q_proj": {
385
+ "group_size": 32,
386
+ "bits": 8,
387
+ "mode": "affine"
388
+ },
389
+ "model.layers.13.self_attn.k_proj": {
390
+ "group_size": 32,
391
+ "bits": 8,
392
+ "mode": "affine"
393
+ },
394
+ "model.layers.13.self_attn.v_proj": {
395
+ "group_size": 32,
396
+ "bits": 8,
397
+ "mode": "affine"
398
+ },
399
+ "model.layers.13.self_attn.o_proj": {
400
+ "group_size": 32,
401
+ "bits": 8,
402
+ "mode": "affine"
403
+ },
404
+ "model.layers.13.mlp.router": {
405
+ "group_size": 64,
406
+ "bits": 8
407
+ },
408
+ "model.layers.14.self_attn.q_proj": {
409
+ "group_size": 32,
410
+ "bits": 8,
411
+ "mode": "affine"
412
+ },
413
+ "model.layers.14.self_attn.k_proj": {
414
+ "group_size": 32,
415
+ "bits": 8,
416
+ "mode": "affine"
417
+ },
418
+ "model.layers.14.self_attn.v_proj": {
419
+ "group_size": 32,
420
+ "bits": 8,
421
+ "mode": "affine"
422
+ },
423
+ "model.layers.14.self_attn.o_proj": {
424
+ "group_size": 32,
425
+ "bits": 8,
426
+ "mode": "affine"
427
+ },
428
+ "model.layers.14.mlp.router": {
429
+ "group_size": 64,
430
+ "bits": 8
431
+ },
432
+ "model.layers.15.self_attn.q_proj": {
433
+ "group_size": 32,
434
+ "bits": 8,
435
+ "mode": "affine"
436
+ },
437
+ "model.layers.15.self_attn.k_proj": {
438
+ "group_size": 32,
439
+ "bits": 8,
440
+ "mode": "affine"
441
+ },
442
+ "model.layers.15.self_attn.v_proj": {
443
+ "group_size": 32,
444
+ "bits": 8,
445
+ "mode": "affine"
446
+ },
447
+ "model.layers.15.self_attn.o_proj": {
448
+ "group_size": 32,
449
+ "bits": 8,
450
+ "mode": "affine"
451
+ },
452
+ "model.layers.15.mlp.router": {
453
+ "group_size": 64,
454
+ "bits": 8
455
+ },
456
+ "model.layers.16.self_attn.q_proj": {
457
+ "group_size": 32,
458
+ "bits": 8,
459
+ "mode": "affine"
460
+ },
461
+ "model.layers.16.self_attn.k_proj": {
462
+ "group_size": 32,
463
+ "bits": 8,
464
+ "mode": "affine"
465
+ },
466
+ "model.layers.16.self_attn.v_proj": {
467
+ "group_size": 32,
468
+ "bits": 8,
469
+ "mode": "affine"
470
+ },
471
+ "model.layers.16.self_attn.o_proj": {
472
+ "group_size": 32,
473
+ "bits": 8,
474
+ "mode": "affine"
475
+ },
476
+ "model.layers.16.mlp.router": {
477
+ "group_size": 64,
478
+ "bits": 8
479
+ },
480
+ "model.layers.17.self_attn.q_proj": {
481
+ "group_size": 32,
482
+ "bits": 8,
483
+ "mode": "affine"
484
+ },
485
+ "model.layers.17.self_attn.k_proj": {
486
+ "group_size": 32,
487
+ "bits": 8,
488
+ "mode": "affine"
489
+ },
490
+ "model.layers.17.self_attn.v_proj": {
491
+ "group_size": 32,
492
+ "bits": 8,
493
+ "mode": "affine"
494
+ },
495
+ "model.layers.17.self_attn.o_proj": {
496
+ "group_size": 32,
497
+ "bits": 8,
498
+ "mode": "affine"
499
+ },
500
+ "model.layers.17.mlp.router": {
501
+ "group_size": 64,
502
+ "bits": 8
503
+ },
504
+ "model.layers.18.self_attn.q_proj": {
505
+ "group_size": 32,
506
+ "bits": 8,
507
+ "mode": "affine"
508
+ },
509
+ "model.layers.18.self_attn.k_proj": {
510
+ "group_size": 32,
511
+ "bits": 8,
512
+ "mode": "affine"
513
+ },
514
+ "model.layers.18.self_attn.v_proj": {
515
+ "group_size": 32,
516
+ "bits": 8,
517
+ "mode": "affine"
518
+ },
519
+ "model.layers.18.self_attn.o_proj": {
520
+ "group_size": 32,
521
+ "bits": 8,
522
+ "mode": "affine"
523
+ },
524
+ "model.layers.18.mlp.router": {
525
+ "group_size": 64,
526
+ "bits": 8
527
+ },
528
+ "model.layers.19.self_attn.q_proj": {
529
+ "group_size": 32,
530
+ "bits": 8,
531
+ "mode": "affine"
532
+ },
533
+ "model.layers.19.self_attn.k_proj": {
534
+ "group_size": 32,
535
+ "bits": 8,
536
+ "mode": "affine"
537
+ },
538
+ "model.layers.19.self_attn.v_proj": {
539
+ "group_size": 32,
540
+ "bits": 8,
541
+ "mode": "affine"
542
+ },
543
+ "model.layers.19.self_attn.o_proj": {
544
+ "group_size": 32,
545
+ "bits": 8,
546
+ "mode": "affine"
547
+ },
548
+ "model.layers.19.mlp.router": {
549
+ "group_size": 64,
550
+ "bits": 8
551
+ },
552
+ "model.layers.20.self_attn.q_proj": {
553
+ "group_size": 32,
554
+ "bits": 8,
555
+ "mode": "affine"
556
+ },
557
+ "model.layers.20.self_attn.k_proj": {
558
+ "group_size": 32,
559
+ "bits": 8,
560
+ "mode": "affine"
561
+ },
562
+ "model.layers.20.self_attn.v_proj": {
563
+ "group_size": 32,
564
+ "bits": 8,
565
+ "mode": "affine"
566
+ },
567
+ "model.layers.20.self_attn.o_proj": {
568
+ "group_size": 32,
569
+ "bits": 8,
570
+ "mode": "affine"
571
+ },
572
+ "model.layers.20.mlp.router": {
573
+ "group_size": 64,
574
+ "bits": 8
575
+ },
576
+ "model.layers.21.self_attn.q_proj": {
577
+ "group_size": 32,
578
+ "bits": 8,
579
+ "mode": "affine"
580
+ },
581
+ "model.layers.21.self_attn.k_proj": {
582
+ "group_size": 32,
583
+ "bits": 8,
584
+ "mode": "affine"
585
+ },
586
+ "model.layers.21.self_attn.v_proj": {
587
+ "group_size": 32,
588
+ "bits": 8,
589
+ "mode": "affine"
590
+ },
591
+ "model.layers.21.self_attn.o_proj": {
592
+ "group_size": 32,
593
+ "bits": 8,
594
+ "mode": "affine"
595
+ },
596
+ "model.layers.21.mlp.router": {
597
+ "group_size": 64,
598
+ "bits": 8
599
+ },
600
+ "model.layers.22.self_attn.q_proj": {
601
+ "group_size": 32,
602
+ "bits": 8,
603
+ "mode": "affine"
604
+ },
605
+ "model.layers.22.self_attn.k_proj": {
606
+ "group_size": 32,
607
+ "bits": 8,
608
+ "mode": "affine"
609
+ },
610
+ "model.layers.22.self_attn.v_proj": {
611
+ "group_size": 32,
612
+ "bits": 8,
613
+ "mode": "affine"
614
+ },
615
+ "model.layers.22.self_attn.o_proj": {
616
+ "group_size": 32,
617
+ "bits": 8,
618
+ "mode": "affine"
619
+ },
620
+ "model.layers.22.mlp.router": {
621
+ "group_size": 64,
622
+ "bits": 8
623
+ },
624
+ "model.layers.23.self_attn.q_proj": {
625
+ "group_size": 32,
626
+ "bits": 8,
627
+ "mode": "affine"
628
+ },
629
+ "model.layers.23.self_attn.k_proj": {
630
+ "group_size": 32,
631
+ "bits": 8,
632
+ "mode": "affine"
633
+ },
634
+ "model.layers.23.self_attn.v_proj": {
635
+ "group_size": 32,
636
+ "bits": 8,
637
+ "mode": "affine"
638
+ },
639
+ "model.layers.23.self_attn.o_proj": {
640
+ "group_size": 32,
641
+ "bits": 8,
642
+ "mode": "affine"
643
+ },
644
+ "model.layers.23.mlp.router": {
645
+ "group_size": 64,
646
+ "bits": 8
647
+ },
648
+ "model.layers.24.self_attn.q_proj": {
649
+ "group_size": 32,
650
+ "bits": 8,
651
+ "mode": "affine"
652
+ },
653
+ "model.layers.24.self_attn.k_proj": {
654
+ "group_size": 32,
655
+ "bits": 8,
656
+ "mode": "affine"
657
+ },
658
+ "model.layers.24.self_attn.v_proj": {
659
+ "group_size": 32,
660
+ "bits": 8,
661
+ "mode": "affine"
662
+ },
663
+ "model.layers.24.self_attn.o_proj": {
664
+ "group_size": 32,
665
+ "bits": 8,
666
+ "mode": "affine"
667
+ },
668
+ "model.layers.24.mlp.router": {
669
+ "group_size": 64,
670
+ "bits": 8
671
+ },
672
+ "model.layers.25.self_attn.q_proj": {
673
+ "group_size": 32,
674
+ "bits": 8,
675
+ "mode": "affine"
676
+ },
677
+ "model.layers.25.self_attn.k_proj": {
678
+ "group_size": 32,
679
+ "bits": 8,
680
+ "mode": "affine"
681
+ },
682
+ "model.layers.25.self_attn.v_proj": {
683
+ "group_size": 32,
684
+ "bits": 8,
685
+ "mode": "affine"
686
+ },
687
+ "model.layers.25.self_attn.o_proj": {
688
+ "group_size": 32,
689
+ "bits": 8,
690
+ "mode": "affine"
691
+ },
692
+ "model.layers.25.mlp.router": {
693
+ "group_size": 64,
694
+ "bits": 8
695
+ },
696
+ "model.layers.26.self_attn.q_proj": {
697
+ "group_size": 32,
698
+ "bits": 8,
699
+ "mode": "affine"
700
+ },
701
+ "model.layers.26.self_attn.k_proj": {
702
+ "group_size": 32,
703
+ "bits": 8,
704
+ "mode": "affine"
705
+ },
706
+ "model.layers.26.self_attn.v_proj": {
707
+ "group_size": 32,
708
+ "bits": 8,
709
+ "mode": "affine"
710
+ },
711
+ "model.layers.26.self_attn.o_proj": {
712
+ "group_size": 32,
713
+ "bits": 8,
714
+ "mode": "affine"
715
+ },
716
+ "model.layers.26.mlp.router": {
717
+ "group_size": 64,
718
+ "bits": 8
719
+ },
720
+ "model.layers.27.self_attn.q_proj": {
721
+ "group_size": 32,
722
+ "bits": 8,
723
+ "mode": "affine"
724
+ },
725
+ "model.layers.27.self_attn.k_proj": {
726
+ "group_size": 32,
727
+ "bits": 8,
728
+ "mode": "affine"
729
+ },
730
+ "model.layers.27.self_attn.v_proj": {
731
+ "group_size": 32,
732
+ "bits": 8,
733
+ "mode": "affine"
734
+ },
735
+ "model.layers.27.self_attn.o_proj": {
736
+ "group_size": 32,
737
+ "bits": 8,
738
+ "mode": "affine"
739
+ },
740
+ "model.layers.27.mlp.router": {
741
+ "group_size": 64,
742
+ "bits": 8
743
+ },
744
+ "model.layers.28.self_attn.q_proj": {
745
+ "group_size": 32,
746
+ "bits": 8,
747
+ "mode": "affine"
748
+ },
749
+ "model.layers.28.self_attn.k_proj": {
750
+ "group_size": 32,
751
+ "bits": 8,
752
+ "mode": "affine"
753
+ },
754
+ "model.layers.28.self_attn.v_proj": {
755
+ "group_size": 32,
756
+ "bits": 8,
757
+ "mode": "affine"
758
+ },
759
+ "model.layers.28.self_attn.o_proj": {
760
+ "group_size": 32,
761
+ "bits": 8,
762
+ "mode": "affine"
763
+ },
764
+ "model.layers.28.mlp.router": {
765
+ "group_size": 64,
766
+ "bits": 8
767
+ },
768
+ "model.layers.29.self_attn.q_proj": {
769
+ "group_size": 32,
770
+ "bits": 8,
771
+ "mode": "affine"
772
+ },
773
+ "model.layers.29.self_attn.k_proj": {
774
+ "group_size": 32,
775
+ "bits": 8,
776
+ "mode": "affine"
777
+ },
778
+ "model.layers.29.self_attn.v_proj": {
779
+ "group_size": 32,
780
+ "bits": 8,
781
+ "mode": "affine"
782
+ },
783
+ "model.layers.29.self_attn.o_proj": {
784
+ "group_size": 32,
785
+ "bits": 8,
786
+ "mode": "affine"
787
+ },
788
+ "model.layers.29.mlp.router": {
789
+ "group_size": 64,
790
+ "bits": 8
791
+ },
792
+ "model.layers.30.self_attn.q_proj": {
793
+ "group_size": 32,
794
+ "bits": 8,
795
+ "mode": "affine"
796
+ },
797
+ "model.layers.30.self_attn.k_proj": {
798
+ "group_size": 32,
799
+ "bits": 8,
800
+ "mode": "affine"
801
+ },
802
+ "model.layers.30.self_attn.v_proj": {
803
+ "group_size": 32,
804
+ "bits": 8,
805
+ "mode": "affine"
806
+ },
807
+ "model.layers.30.self_attn.o_proj": {
808
+ "group_size": 32,
809
+ "bits": 8,
810
+ "mode": "affine"
811
+ },
812
+ "model.layers.30.mlp.router": {
813
+ "group_size": 64,
814
+ "bits": 8
815
+ },
816
+ "model.layers.31.self_attn.q_proj": {
817
+ "group_size": 32,
818
+ "bits": 8,
819
+ "mode": "affine"
820
+ },
821
+ "model.layers.31.self_attn.k_proj": {
822
+ "group_size": 32,
823
+ "bits": 8,
824
+ "mode": "affine"
825
+ },
826
+ "model.layers.31.self_attn.v_proj": {
827
+ "group_size": 32,
828
+ "bits": 8,
829
+ "mode": "affine"
830
+ },
831
+ "model.layers.31.self_attn.o_proj": {
832
+ "group_size": 32,
833
+ "bits": 8,
834
+ "mode": "affine"
835
+ },
836
+ "model.layers.31.mlp.router": {
837
+ "group_size": 64,
838
+ "bits": 8
839
+ },
840
+ "model.layers.32.self_attn.q_proj": {
841
+ "group_size": 32,
842
+ "bits": 8,
843
+ "mode": "affine"
844
+ },
845
+ "model.layers.32.self_attn.k_proj": {
846
+ "group_size": 32,
847
+ "bits": 8,
848
+ "mode": "affine"
849
+ },
850
+ "model.layers.32.self_attn.v_proj": {
851
+ "group_size": 32,
852
+ "bits": 8,
853
+ "mode": "affine"
854
+ },
855
+ "model.layers.32.self_attn.o_proj": {
856
+ "group_size": 32,
857
+ "bits": 8,
858
+ "mode": "affine"
859
+ },
860
+ "model.layers.32.mlp.router": {
861
+ "group_size": 64,
862
+ "bits": 8
863
+ },
864
+ "model.layers.33.self_attn.q_proj": {
865
+ "group_size": 32,
866
+ "bits": 8,
867
+ "mode": "affine"
868
+ },
869
+ "model.layers.33.self_attn.k_proj": {
870
+ "group_size": 32,
871
+ "bits": 8,
872
+ "mode": "affine"
873
+ },
874
+ "model.layers.33.self_attn.v_proj": {
875
+ "group_size": 32,
876
+ "bits": 8,
877
+ "mode": "affine"
878
+ },
879
+ "model.layers.33.self_attn.o_proj": {
880
+ "group_size": 32,
881
+ "bits": 8,
882
+ "mode": "affine"
883
+ },
884
+ "model.layers.33.mlp.router": {
885
+ "group_size": 64,
886
+ "bits": 8
887
+ },
888
+ "model.layers.34.self_attn.q_proj": {
889
+ "group_size": 32,
890
+ "bits": 8,
891
+ "mode": "affine"
892
+ },
893
+ "model.layers.34.self_attn.k_proj": {
894
+ "group_size": 32,
895
+ "bits": 8,
896
+ "mode": "affine"
897
+ },
898
+ "model.layers.34.self_attn.v_proj": {
899
+ "group_size": 32,
900
+ "bits": 8,
901
+ "mode": "affine"
902
+ },
903
+ "model.layers.34.self_attn.o_proj": {
904
+ "group_size": 32,
905
+ "bits": 8,
906
+ "mode": "affine"
907
+ },
908
+ "model.layers.34.mlp.router": {
909
+ "group_size": 64,
910
+ "bits": 8
911
+ },
912
+ "model.layers.35.self_attn.q_proj": {
913
+ "group_size": 32,
914
+ "bits": 8,
915
+ "mode": "affine"
916
+ },
917
+ "model.layers.35.self_attn.k_proj": {
918
+ "group_size": 32,
919
+ "bits": 8,
920
+ "mode": "affine"
921
+ },
922
+ "model.layers.35.self_attn.v_proj": {
923
+ "group_size": 32,
924
+ "bits": 8,
925
+ "mode": "affine"
926
+ },
927
+ "model.layers.35.self_attn.o_proj": {
928
+ "group_size": 32,
929
+ "bits": 8,
930
+ "mode": "affine"
931
+ },
932
+ "model.layers.35.mlp.router": {
933
+ "group_size": 64,
934
+ "bits": 8
935
+ },
936
+ "lm_head": {
937
+ "group_size": 32,
938
+ "bits": 8,
939
+ "mode": "affine"
940
+ }
941
+ },
942
+ "quantization_config": {
943
+ "group_size": 32,
944
+ "bits": 4,
945
+ "mode": "mxfp4",
946
+ "model.embed_tokens": {
947
+ "group_size": 32,
948
+ "bits": 8,
949
+ "mode": "affine"
950
+ },
951
+ "model.layers.0.self_attn.q_proj": {
952
+ "group_size": 32,
953
+ "bits": 8,
954
+ "mode": "affine"
955
+ },
956
+ "model.layers.0.self_attn.k_proj": {
957
+ "group_size": 32,
958
+ "bits": 8,
959
+ "mode": "affine"
960
+ },
961
+ "model.layers.0.self_attn.v_proj": {
962
+ "group_size": 32,
963
+ "bits": 8,
964
+ "mode": "affine"
965
+ },
966
+ "model.layers.0.self_attn.o_proj": {
967
+ "group_size": 32,
968
+ "bits": 8,
969
+ "mode": "affine"
970
+ },
971
+ "model.layers.0.mlp.router": {
972
+ "group_size": 64,
973
+ "bits": 8
974
+ },
975
+ "model.layers.1.self_attn.q_proj": {
976
+ "group_size": 32,
977
+ "bits": 8,
978
+ "mode": "affine"
979
+ },
980
+ "model.layers.1.self_attn.k_proj": {
981
+ "group_size": 32,
982
+ "bits": 8,
983
+ "mode": "affine"
984
+ },
985
+ "model.layers.1.self_attn.v_proj": {
986
+ "group_size": 32,
987
+ "bits": 8,
988
+ "mode": "affine"
989
+ },
990
+ "model.layers.1.self_attn.o_proj": {
991
+ "group_size": 32,
992
+ "bits": 8,
993
+ "mode": "affine"
994
+ },
995
+ "model.layers.1.mlp.router": {
996
+ "group_size": 64,
997
+ "bits": 8
998
+ },
999
+ "model.layers.2.self_attn.q_proj": {
1000
+ "group_size": 32,
1001
+ "bits": 8,
1002
+ "mode": "affine"
1003
+ },
1004
+ "model.layers.2.self_attn.k_proj": {
1005
+ "group_size": 32,
1006
+ "bits": 8,
1007
+ "mode": "affine"
1008
+ },
1009
+ "model.layers.2.self_attn.v_proj": {
1010
+ "group_size": 32,
1011
+ "bits": 8,
1012
+ "mode": "affine"
1013
+ },
1014
+ "model.layers.2.self_attn.o_proj": {
1015
+ "group_size": 32,
1016
+ "bits": 8,
1017
+ "mode": "affine"
1018
+ },
1019
+ "model.layers.2.mlp.router": {
1020
+ "group_size": 64,
1021
+ "bits": 8
1022
+ },
1023
+ "model.layers.3.self_attn.q_proj": {
1024
+ "group_size": 32,
1025
+ "bits": 8,
1026
+ "mode": "affine"
1027
+ },
1028
+ "model.layers.3.self_attn.k_proj": {
1029
+ "group_size": 32,
1030
+ "bits": 8,
1031
+ "mode": "affine"
1032
+ },
1033
+ "model.layers.3.self_attn.v_proj": {
1034
+ "group_size": 32,
1035
+ "bits": 8,
1036
+ "mode": "affine"
1037
+ },
1038
+ "model.layers.3.self_attn.o_proj": {
1039
+ "group_size": 32,
1040
+ "bits": 8,
1041
+ "mode": "affine"
1042
+ },
1043
+ "model.layers.3.mlp.router": {
1044
+ "group_size": 64,
1045
+ "bits": 8
1046
+ },
1047
+ "model.layers.4.self_attn.q_proj": {
1048
+ "group_size": 32,
1049
+ "bits": 8,
1050
+ "mode": "affine"
1051
+ },
1052
+ "model.layers.4.self_attn.k_proj": {
1053
+ "group_size": 32,
1054
+ "bits": 8,
1055
+ "mode": "affine"
1056
+ },
1057
+ "model.layers.4.self_attn.v_proj": {
1058
+ "group_size": 32,
1059
+ "bits": 8,
1060
+ "mode": "affine"
1061
+ },
1062
+ "model.layers.4.self_attn.o_proj": {
1063
+ "group_size": 32,
1064
+ "bits": 8,
1065
+ "mode": "affine"
1066
+ },
1067
+ "model.layers.4.mlp.router": {
1068
+ "group_size": 64,
1069
+ "bits": 8
1070
+ },
1071
+ "model.layers.5.self_attn.q_proj": {
1072
+ "group_size": 32,
1073
+ "bits": 8,
1074
+ "mode": "affine"
1075
+ },
1076
+ "model.layers.5.self_attn.k_proj": {
1077
+ "group_size": 32,
1078
+ "bits": 8,
1079
+ "mode": "affine"
1080
+ },
1081
+ "model.layers.5.self_attn.v_proj": {
1082
+ "group_size": 32,
1083
+ "bits": 8,
1084
+ "mode": "affine"
1085
+ },
1086
+ "model.layers.5.self_attn.o_proj": {
1087
+ "group_size": 32,
1088
+ "bits": 8,
1089
+ "mode": "affine"
1090
+ },
1091
+ "model.layers.5.mlp.router": {
1092
+ "group_size": 64,
1093
+ "bits": 8
1094
+ },
1095
+ "model.layers.6.self_attn.q_proj": {
1096
+ "group_size": 32,
1097
+ "bits": 8,
1098
+ "mode": "affine"
1099
+ },
1100
+ "model.layers.6.self_attn.k_proj": {
1101
+ "group_size": 32,
1102
+ "bits": 8,
1103
+ "mode": "affine"
1104
+ },
1105
+ "model.layers.6.self_attn.v_proj": {
1106
+ "group_size": 32,
1107
+ "bits": 8,
1108
+ "mode": "affine"
1109
+ },
1110
+ "model.layers.6.self_attn.o_proj": {
1111
+ "group_size": 32,
1112
+ "bits": 8,
1113
+ "mode": "affine"
1114
+ },
1115
+ "model.layers.6.mlp.router": {
1116
+ "group_size": 64,
1117
+ "bits": 8
1118
+ },
1119
+ "model.layers.7.self_attn.q_proj": {
1120
+ "group_size": 32,
1121
+ "bits": 8,
1122
+ "mode": "affine"
1123
+ },
1124
+ "model.layers.7.self_attn.k_proj": {
1125
+ "group_size": 32,
1126
+ "bits": 8,
1127
+ "mode": "affine"
1128
+ },
1129
+ "model.layers.7.self_attn.v_proj": {
1130
+ "group_size": 32,
1131
+ "bits": 8,
1132
+ "mode": "affine"
1133
+ },
1134
+ "model.layers.7.self_attn.o_proj": {
1135
+ "group_size": 32,
1136
+ "bits": 8,
1137
+ "mode": "affine"
1138
+ },
1139
+ "model.layers.7.mlp.router": {
1140
+ "group_size": 64,
1141
+ "bits": 8
1142
+ },
1143
+ "model.layers.8.self_attn.q_proj": {
1144
+ "group_size": 32,
1145
+ "bits": 8,
1146
+ "mode": "affine"
1147
+ },
1148
+ "model.layers.8.self_attn.k_proj": {
1149
+ "group_size": 32,
1150
+ "bits": 8,
1151
+ "mode": "affine"
1152
+ },
1153
+ "model.layers.8.self_attn.v_proj": {
1154
+ "group_size": 32,
1155
+ "bits": 8,
1156
+ "mode": "affine"
1157
+ },
1158
+ "model.layers.8.self_attn.o_proj": {
1159
+ "group_size": 32,
1160
+ "bits": 8,
1161
+ "mode": "affine"
1162
+ },
1163
+ "model.layers.8.mlp.router": {
1164
+ "group_size": 64,
1165
+ "bits": 8
1166
+ },
1167
+ "model.layers.9.self_attn.q_proj": {
1168
+ "group_size": 32,
1169
+ "bits": 8,
1170
+ "mode": "affine"
1171
+ },
1172
+ "model.layers.9.self_attn.k_proj": {
1173
+ "group_size": 32,
1174
+ "bits": 8,
1175
+ "mode": "affine"
1176
+ },
1177
+ "model.layers.9.self_attn.v_proj": {
1178
+ "group_size": 32,
1179
+ "bits": 8,
1180
+ "mode": "affine"
1181
+ },
1182
+ "model.layers.9.self_attn.o_proj": {
1183
+ "group_size": 32,
1184
+ "bits": 8,
1185
+ "mode": "affine"
1186
+ },
1187
+ "model.layers.9.mlp.router": {
1188
+ "group_size": 64,
1189
+ "bits": 8
1190
+ },
1191
+ "model.layers.10.self_attn.q_proj": {
1192
+ "group_size": 32,
1193
+ "bits": 8,
1194
+ "mode": "affine"
1195
+ },
1196
+ "model.layers.10.self_attn.k_proj": {
1197
+ "group_size": 32,
1198
+ "bits": 8,
1199
+ "mode": "affine"
1200
+ },
1201
+ "model.layers.10.self_attn.v_proj": {
1202
+ "group_size": 32,
1203
+ "bits": 8,
1204
+ "mode": "affine"
1205
+ },
1206
+ "model.layers.10.self_attn.o_proj": {
1207
+ "group_size": 32,
1208
+ "bits": 8,
1209
+ "mode": "affine"
1210
+ },
1211
+ "model.layers.10.mlp.router": {
1212
+ "group_size": 64,
1213
+ "bits": 8
1214
+ },
1215
+ "model.layers.11.self_attn.q_proj": {
1216
+ "group_size": 32,
1217
+ "bits": 8,
1218
+ "mode": "affine"
1219
+ },
1220
+ "model.layers.11.self_attn.k_proj": {
1221
+ "group_size": 32,
1222
+ "bits": 8,
1223
+ "mode": "affine"
1224
+ },
1225
+ "model.layers.11.self_attn.v_proj": {
1226
+ "group_size": 32,
1227
+ "bits": 8,
1228
+ "mode": "affine"
1229
+ },
1230
+ "model.layers.11.self_attn.o_proj": {
1231
+ "group_size": 32,
1232
+ "bits": 8,
1233
+ "mode": "affine"
1234
+ },
1235
+ "model.layers.11.mlp.router": {
1236
+ "group_size": 64,
1237
+ "bits": 8
1238
+ },
1239
+ "model.layers.12.self_attn.q_proj": {
1240
+ "group_size": 32,
1241
+ "bits": 8,
1242
+ "mode": "affine"
1243
+ },
1244
+ "model.layers.12.self_attn.k_proj": {
1245
+ "group_size": 32,
1246
+ "bits": 8,
1247
+ "mode": "affine"
1248
+ },
1249
+ "model.layers.12.self_attn.v_proj": {
1250
+ "group_size": 32,
1251
+ "bits": 8,
1252
+ "mode": "affine"
1253
+ },
1254
+ "model.layers.12.self_attn.o_proj": {
1255
+ "group_size": 32,
1256
+ "bits": 8,
1257
+ "mode": "affine"
1258
+ },
1259
+ "model.layers.12.mlp.router": {
1260
+ "group_size": 64,
1261
+ "bits": 8
1262
+ },
1263
+ "model.layers.13.self_attn.q_proj": {
1264
+ "group_size": 32,
1265
+ "bits": 8,
1266
+ "mode": "affine"
1267
+ },
1268
+ "model.layers.13.self_attn.k_proj": {
1269
+ "group_size": 32,
1270
+ "bits": 8,
1271
+ "mode": "affine"
1272
+ },
1273
+ "model.layers.13.self_attn.v_proj": {
1274
+ "group_size": 32,
1275
+ "bits": 8,
1276
+ "mode": "affine"
1277
+ },
1278
+ "model.layers.13.self_attn.o_proj": {
1279
+ "group_size": 32,
1280
+ "bits": 8,
1281
+ "mode": "affine"
1282
+ },
1283
+ "model.layers.13.mlp.router": {
1284
+ "group_size": 64,
1285
+ "bits": 8
1286
+ },
1287
+ "model.layers.14.self_attn.q_proj": {
1288
+ "group_size": 32,
1289
+ "bits": 8,
1290
+ "mode": "affine"
1291
+ },
1292
+ "model.layers.14.self_attn.k_proj": {
1293
+ "group_size": 32,
1294
+ "bits": 8,
1295
+ "mode": "affine"
1296
+ },
1297
+ "model.layers.14.self_attn.v_proj": {
1298
+ "group_size": 32,
1299
+ "bits": 8,
1300
+ "mode": "affine"
1301
+ },
1302
+ "model.layers.14.self_attn.o_proj": {
1303
+ "group_size": 32,
1304
+ "bits": 8,
1305
+ "mode": "affine"
1306
+ },
1307
+ "model.layers.14.mlp.router": {
1308
+ "group_size": 64,
1309
+ "bits": 8
1310
+ },
1311
+ "model.layers.15.self_attn.q_proj": {
1312
+ "group_size": 32,
1313
+ "bits": 8,
1314
+ "mode": "affine"
1315
+ },
1316
+ "model.layers.15.self_attn.k_proj": {
1317
+ "group_size": 32,
1318
+ "bits": 8,
1319
+ "mode": "affine"
1320
+ },
1321
+ "model.layers.15.self_attn.v_proj": {
1322
+ "group_size": 32,
1323
+ "bits": 8,
1324
+ "mode": "affine"
1325
+ },
1326
+ "model.layers.15.self_attn.o_proj": {
1327
+ "group_size": 32,
1328
+ "bits": 8,
1329
+ "mode": "affine"
1330
+ },
1331
+ "model.layers.15.mlp.router": {
1332
+ "group_size": 64,
1333
+ "bits": 8
1334
+ },
1335
+ "model.layers.16.self_attn.q_proj": {
1336
+ "group_size": 32,
1337
+ "bits": 8,
1338
+ "mode": "affine"
1339
+ },
1340
+ "model.layers.16.self_attn.k_proj": {
1341
+ "group_size": 32,
1342
+ "bits": 8,
1343
+ "mode": "affine"
1344
+ },
1345
+ "model.layers.16.self_attn.v_proj": {
1346
+ "group_size": 32,
1347
+ "bits": 8,
1348
+ "mode": "affine"
1349
+ },
1350
+ "model.layers.16.self_attn.o_proj": {
1351
+ "group_size": 32,
1352
+ "bits": 8,
1353
+ "mode": "affine"
1354
+ },
1355
+ "model.layers.16.mlp.router": {
1356
+ "group_size": 64,
1357
+ "bits": 8
1358
+ },
1359
+ "model.layers.17.self_attn.q_proj": {
1360
+ "group_size": 32,
1361
+ "bits": 8,
1362
+ "mode": "affine"
1363
+ },
1364
+ "model.layers.17.self_attn.k_proj": {
1365
+ "group_size": 32,
1366
+ "bits": 8,
1367
+ "mode": "affine"
1368
+ },
1369
+ "model.layers.17.self_attn.v_proj": {
1370
+ "group_size": 32,
1371
+ "bits": 8,
1372
+ "mode": "affine"
1373
+ },
1374
+ "model.layers.17.self_attn.o_proj": {
1375
+ "group_size": 32,
1376
+ "bits": 8,
1377
+ "mode": "affine"
1378
+ },
1379
+ "model.layers.17.mlp.router": {
1380
+ "group_size": 64,
1381
+ "bits": 8
1382
+ },
1383
+ "model.layers.18.self_attn.q_proj": {
1384
+ "group_size": 32,
1385
+ "bits": 8,
1386
+ "mode": "affine"
1387
+ },
1388
+ "model.layers.18.self_attn.k_proj": {
1389
+ "group_size": 32,
1390
+ "bits": 8,
1391
+ "mode": "affine"
1392
+ },
1393
+ "model.layers.18.self_attn.v_proj": {
1394
+ "group_size": 32,
1395
+ "bits": 8,
1396
+ "mode": "affine"
1397
+ },
1398
+ "model.layers.18.self_attn.o_proj": {
1399
+ "group_size": 32,
1400
+ "bits": 8,
1401
+ "mode": "affine"
1402
+ },
1403
+ "model.layers.18.mlp.router": {
1404
+ "group_size": 64,
1405
+ "bits": 8
1406
+ },
1407
+ "model.layers.19.self_attn.q_proj": {
1408
+ "group_size": 32,
1409
+ "bits": 8,
1410
+ "mode": "affine"
1411
+ },
1412
+ "model.layers.19.self_attn.k_proj": {
1413
+ "group_size": 32,
1414
+ "bits": 8,
1415
+ "mode": "affine"
1416
+ },
1417
+ "model.layers.19.self_attn.v_proj": {
1418
+ "group_size": 32,
1419
+ "bits": 8,
1420
+ "mode": "affine"
1421
+ },
1422
+ "model.layers.19.self_attn.o_proj": {
1423
+ "group_size": 32,
1424
+ "bits": 8,
1425
+ "mode": "affine"
1426
+ },
1427
+ "model.layers.19.mlp.router": {
1428
+ "group_size": 64,
1429
+ "bits": 8
1430
+ },
1431
+ "model.layers.20.self_attn.q_proj": {
1432
+ "group_size": 32,
1433
+ "bits": 8,
1434
+ "mode": "affine"
1435
+ },
1436
+ "model.layers.20.self_attn.k_proj": {
1437
+ "group_size": 32,
1438
+ "bits": 8,
1439
+ "mode": "affine"
1440
+ },
1441
+ "model.layers.20.self_attn.v_proj": {
1442
+ "group_size": 32,
1443
+ "bits": 8,
1444
+ "mode": "affine"
1445
+ },
1446
+ "model.layers.20.self_attn.o_proj": {
1447
+ "group_size": 32,
1448
+ "bits": 8,
1449
+ "mode": "affine"
1450
+ },
1451
+ "model.layers.20.mlp.router": {
1452
+ "group_size": 64,
1453
+ "bits": 8
1454
+ },
1455
+ "model.layers.21.self_attn.q_proj": {
1456
+ "group_size": 32,
1457
+ "bits": 8,
1458
+ "mode": "affine"
1459
+ },
1460
+ "model.layers.21.self_attn.k_proj": {
1461
+ "group_size": 32,
1462
+ "bits": 8,
1463
+ "mode": "affine"
1464
+ },
1465
+ "model.layers.21.self_attn.v_proj": {
1466
+ "group_size": 32,
1467
+ "bits": 8,
1468
+ "mode": "affine"
1469
+ },
1470
+ "model.layers.21.self_attn.o_proj": {
1471
+ "group_size": 32,
1472
+ "bits": 8,
1473
+ "mode": "affine"
1474
+ },
1475
+ "model.layers.21.mlp.router": {
1476
+ "group_size": 64,
1477
+ "bits": 8
1478
+ },
1479
+ "model.layers.22.self_attn.q_proj": {
1480
+ "group_size": 32,
1481
+ "bits": 8,
1482
+ "mode": "affine"
1483
+ },
1484
+ "model.layers.22.self_attn.k_proj": {
1485
+ "group_size": 32,
1486
+ "bits": 8,
1487
+ "mode": "affine"
1488
+ },
1489
+ "model.layers.22.self_attn.v_proj": {
1490
+ "group_size": 32,
1491
+ "bits": 8,
1492
+ "mode": "affine"
1493
+ },
1494
+ "model.layers.22.self_attn.o_proj": {
1495
+ "group_size": 32,
1496
+ "bits": 8,
1497
+ "mode": "affine"
1498
+ },
1499
+ "model.layers.22.mlp.router": {
1500
+ "group_size": 64,
1501
+ "bits": 8
1502
+ },
1503
+ "model.layers.23.self_attn.q_proj": {
1504
+ "group_size": 32,
1505
+ "bits": 8,
1506
+ "mode": "affine"
1507
+ },
1508
+ "model.layers.23.self_attn.k_proj": {
1509
+ "group_size": 32,
1510
+ "bits": 8,
1511
+ "mode": "affine"
1512
+ },
1513
+ "model.layers.23.self_attn.v_proj": {
1514
+ "group_size": 32,
1515
+ "bits": 8,
1516
+ "mode": "affine"
1517
+ },
1518
+ "model.layers.23.self_attn.o_proj": {
1519
+ "group_size": 32,
1520
+ "bits": 8,
1521
+ "mode": "affine"
1522
+ },
1523
+ "model.layers.23.mlp.router": {
1524
+ "group_size": 64,
1525
+ "bits": 8
1526
+ },
1527
+ "model.layers.24.self_attn.q_proj": {
1528
+ "group_size": 32,
1529
+ "bits": 8,
1530
+ "mode": "affine"
1531
+ },
1532
+ "model.layers.24.self_attn.k_proj": {
1533
+ "group_size": 32,
1534
+ "bits": 8,
1535
+ "mode": "affine"
1536
+ },
1537
+ "model.layers.24.self_attn.v_proj": {
1538
+ "group_size": 32,
1539
+ "bits": 8,
1540
+ "mode": "affine"
1541
+ },
1542
+ "model.layers.24.self_attn.o_proj": {
1543
+ "group_size": 32,
1544
+ "bits": 8,
1545
+ "mode": "affine"
1546
+ },
1547
+ "model.layers.24.mlp.router": {
1548
+ "group_size": 64,
1549
+ "bits": 8
1550
+ },
1551
+ "model.layers.25.self_attn.q_proj": {
1552
+ "group_size": 32,
1553
+ "bits": 8,
1554
+ "mode": "affine"
1555
+ },
1556
+ "model.layers.25.self_attn.k_proj": {
1557
+ "group_size": 32,
1558
+ "bits": 8,
1559
+ "mode": "affine"
1560
+ },
1561
+ "model.layers.25.self_attn.v_proj": {
1562
+ "group_size": 32,
1563
+ "bits": 8,
1564
+ "mode": "affine"
1565
+ },
1566
+ "model.layers.25.self_attn.o_proj": {
1567
+ "group_size": 32,
1568
+ "bits": 8,
1569
+ "mode": "affine"
1570
+ },
1571
+ "model.layers.25.mlp.router": {
1572
+ "group_size": 64,
1573
+ "bits": 8
1574
+ },
1575
+ "model.layers.26.self_attn.q_proj": {
1576
+ "group_size": 32,
1577
+ "bits": 8,
1578
+ "mode": "affine"
1579
+ },
1580
+ "model.layers.26.self_attn.k_proj": {
1581
+ "group_size": 32,
1582
+ "bits": 8,
1583
+ "mode": "affine"
1584
+ },
1585
+ "model.layers.26.self_attn.v_proj": {
1586
+ "group_size": 32,
1587
+ "bits": 8,
1588
+ "mode": "affine"
1589
+ },
1590
+ "model.layers.26.self_attn.o_proj": {
1591
+ "group_size": 32,
1592
+ "bits": 8,
1593
+ "mode": "affine"
1594
+ },
1595
+ "model.layers.26.mlp.router": {
1596
+ "group_size": 64,
1597
+ "bits": 8
1598
+ },
1599
+ "model.layers.27.self_attn.q_proj": {
1600
+ "group_size": 32,
1601
+ "bits": 8,
1602
+ "mode": "affine"
1603
+ },
1604
+ "model.layers.27.self_attn.k_proj": {
1605
+ "group_size": 32,
1606
+ "bits": 8,
1607
+ "mode": "affine"
1608
+ },
1609
+ "model.layers.27.self_attn.v_proj": {
1610
+ "group_size": 32,
1611
+ "bits": 8,
1612
+ "mode": "affine"
1613
+ },
1614
+ "model.layers.27.self_attn.o_proj": {
1615
+ "group_size": 32,
1616
+ "bits": 8,
1617
+ "mode": "affine"
1618
+ },
1619
+ "model.layers.27.mlp.router": {
1620
+ "group_size": 64,
1621
+ "bits": 8
1622
+ },
1623
+ "model.layers.28.self_attn.q_proj": {
1624
+ "group_size": 32,
1625
+ "bits": 8,
1626
+ "mode": "affine"
1627
+ },
1628
+ "model.layers.28.self_attn.k_proj": {
1629
+ "group_size": 32,
1630
+ "bits": 8,
1631
+ "mode": "affine"
1632
+ },
1633
+ "model.layers.28.self_attn.v_proj": {
1634
+ "group_size": 32,
1635
+ "bits": 8,
1636
+ "mode": "affine"
1637
+ },
1638
+ "model.layers.28.self_attn.o_proj": {
1639
+ "group_size": 32,
1640
+ "bits": 8,
1641
+ "mode": "affine"
1642
+ },
1643
+ "model.layers.28.mlp.router": {
1644
+ "group_size": 64,
1645
+ "bits": 8
1646
+ },
1647
+ "model.layers.29.self_attn.q_proj": {
1648
+ "group_size": 32,
1649
+ "bits": 8,
1650
+ "mode": "affine"
1651
+ },
1652
+ "model.layers.29.self_attn.k_proj": {
1653
+ "group_size": 32,
1654
+ "bits": 8,
1655
+ "mode": "affine"
1656
+ },
1657
+ "model.layers.29.self_attn.v_proj": {
1658
+ "group_size": 32,
1659
+ "bits": 8,
1660
+ "mode": "affine"
1661
+ },
1662
+ "model.layers.29.self_attn.o_proj": {
1663
+ "group_size": 32,
1664
+ "bits": 8,
1665
+ "mode": "affine"
1666
+ },
1667
+ "model.layers.29.mlp.router": {
1668
+ "group_size": 64,
1669
+ "bits": 8
1670
+ },
1671
+ "model.layers.30.self_attn.q_proj": {
1672
+ "group_size": 32,
1673
+ "bits": 8,
1674
+ "mode": "affine"
1675
+ },
1676
+ "model.layers.30.self_attn.k_proj": {
1677
+ "group_size": 32,
1678
+ "bits": 8,
1679
+ "mode": "affine"
1680
+ },
1681
+ "model.layers.30.self_attn.v_proj": {
1682
+ "group_size": 32,
1683
+ "bits": 8,
1684
+ "mode": "affine"
1685
+ },
1686
+ "model.layers.30.self_attn.o_proj": {
1687
+ "group_size": 32,
1688
+ "bits": 8,
1689
+ "mode": "affine"
1690
+ },
1691
+ "model.layers.30.mlp.router": {
1692
+ "group_size": 64,
1693
+ "bits": 8
1694
+ },
1695
+ "model.layers.31.self_attn.q_proj": {
1696
+ "group_size": 32,
1697
+ "bits": 8,
1698
+ "mode": "affine"
1699
+ },
1700
+ "model.layers.31.self_attn.k_proj": {
1701
+ "group_size": 32,
1702
+ "bits": 8,
1703
+ "mode": "affine"
1704
+ },
1705
+ "model.layers.31.self_attn.v_proj": {
1706
+ "group_size": 32,
1707
+ "bits": 8,
1708
+ "mode": "affine"
1709
+ },
1710
+ "model.layers.31.self_attn.o_proj": {
1711
+ "group_size": 32,
1712
+ "bits": 8,
1713
+ "mode": "affine"
1714
+ },
1715
+ "model.layers.31.mlp.router": {
1716
+ "group_size": 64,
1717
+ "bits": 8
1718
+ },
1719
+ "model.layers.32.self_attn.q_proj": {
1720
+ "group_size": 32,
1721
+ "bits": 8,
1722
+ "mode": "affine"
1723
+ },
1724
+ "model.layers.32.self_attn.k_proj": {
1725
+ "group_size": 32,
1726
+ "bits": 8,
1727
+ "mode": "affine"
1728
+ },
1729
+ "model.layers.32.self_attn.v_proj": {
1730
+ "group_size": 32,
1731
+ "bits": 8,
1732
+ "mode": "affine"
1733
+ },
1734
+ "model.layers.32.self_attn.o_proj": {
1735
+ "group_size": 32,
1736
+ "bits": 8,
1737
+ "mode": "affine"
1738
+ },
1739
+ "model.layers.32.mlp.router": {
1740
+ "group_size": 64,
1741
+ "bits": 8
1742
+ },
1743
+ "model.layers.33.self_attn.q_proj": {
1744
+ "group_size": 32,
1745
+ "bits": 8,
1746
+ "mode": "affine"
1747
+ },
1748
+ "model.layers.33.self_attn.k_proj": {
1749
+ "group_size": 32,
1750
+ "bits": 8,
1751
+ "mode": "affine"
1752
+ },
1753
+ "model.layers.33.self_attn.v_proj": {
1754
+ "group_size": 32,
1755
+ "bits": 8,
1756
+ "mode": "affine"
1757
+ },
1758
+ "model.layers.33.self_attn.o_proj": {
1759
+ "group_size": 32,
1760
+ "bits": 8,
1761
+ "mode": "affine"
1762
+ },
1763
+ "model.layers.33.mlp.router": {
1764
+ "group_size": 64,
1765
+ "bits": 8
1766
+ },
1767
+ "model.layers.34.self_attn.q_proj": {
1768
+ "group_size": 32,
1769
+ "bits": 8,
1770
+ "mode": "affine"
1771
+ },
1772
+ "model.layers.34.self_attn.k_proj": {
1773
+ "group_size": 32,
1774
+ "bits": 8,
1775
+ "mode": "affine"
1776
+ },
1777
+ "model.layers.34.self_attn.v_proj": {
1778
+ "group_size": 32,
1779
+ "bits": 8,
1780
+ "mode": "affine"
1781
+ },
1782
+ "model.layers.34.self_attn.o_proj": {
1783
+ "group_size": 32,
1784
+ "bits": 8,
1785
+ "mode": "affine"
1786
+ },
1787
+ "model.layers.34.mlp.router": {
1788
+ "group_size": 64,
1789
+ "bits": 8
1790
+ },
1791
+ "model.layers.35.self_attn.q_proj": {
1792
+ "group_size": 32,
1793
+ "bits": 8,
1794
+ "mode": "affine"
1795
+ },
1796
+ "model.layers.35.self_attn.k_proj": {
1797
+ "group_size": 32,
1798
+ "bits": 8,
1799
+ "mode": "affine"
1800
+ },
1801
+ "model.layers.35.self_attn.v_proj": {
1802
+ "group_size": 32,
1803
+ "bits": 8,
1804
+ "mode": "affine"
1805
+ },
1806
+ "model.layers.35.self_attn.o_proj": {
1807
+ "group_size": 32,
1808
+ "bits": 8,
1809
+ "mode": "affine"
1810
+ },
1811
+ "model.layers.35.mlp.router": {
1812
+ "group_size": 64,
1813
+ "bits": 8
1814
+ },
1815
+ "lm_head": {
1816
+ "group_size": 32,
1817
+ "bits": 8,
1818
+ "mode": "affine"
1819
+ }
1820
+ },
1821
+ "rms_norm_eps": 1e-05,
1822
+ "rope_scaling": {
1823
+ "beta_fast": 32.0,
1824
+ "beta_slow": 1.0,
1825
+ "factor": 32.0,
1826
+ "original_max_position_embeddings": 4096,
1827
+ "rope_type": "yarn",
1828
+ "truncate": false
1829
+ },
1830
+ "rope_theta": 150000,
1831
+ "router_aux_loss_coef": 0.9,
1832
+ "sliding_window": 128,
1833
+ "swiglu_limit": 7.0,
1834
+ "tie_word_embeddings": false,
1835
+ "transformers_version": "4.57.1",
1836
+ "use_cache": true,
1837
+ "vocab_size": 201088
1838
+ }
generation_config.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 199998,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 200002,
6
+ 199999,
7
+ 200012
8
+ ],
9
+ "pad_token_id": 199999,
10
+ "transformers_version": "4.57.1"
11
+ }
model-00001-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:485df3e856d3888025af1dee9a797b7928a5777560cf450af64c91de4a913685
3
+ size 5260027926
model-00002-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8b4c2ff75367ee5fa92e713a2b8bfceba9d9b0c2d107404b63b1ec87da72255
3
+ size 5173657574
model-00003-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85ad668218aacac862f3cdaf68a968bdf21c68d78e454a01227d56ba59b7bd42
3
+ size 5173657586
model-00004-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:873203f789194f34ddb17c0bcbbde69f0202b2da53bd9492330fb2511f116d23
3
+ size 5173657575
model-00005-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1833624051e30607c65326eedb039b2db9456bad8e8e1f23e7b69d147f050c0f
3
+ size 5173657706
model-00006-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:914a84806dbd2ea5a4ba981820beb420573652cbcde31c64cb19eb5545183531
3
+ size 5173657626
model-00007-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:00dc06a246466c6cd7207e8a82e5de76b8bed387f9acdfa6da9b0a505c32d466
3
+ size 5173657702
model-00008-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:001225b5e3594e68ee91bb179e15846fa8f6200b1ed762a8068c20f238b70353
3
+ size 5173657676
model-00009-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18f1355a4328aa569ad6f83b278eff83bdb29f1b5d1b8b34fca624ddd4a9288f
3
+ size 5173657674
model-00010-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a016b91c091c230efa63a7481188e67f912b701548a397884b6ae8c81742a3fa
3
+ size 5173657690
model-00011-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72591832296491866d0ca488c2f01d87a4743646c774087c9b04be48922369f4
3
+ size 5173657706
model-00012-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2da2467146083197f6d9b3e4f54d6649992ff400bd2f41112ecff878f4b0b597
3
+ size 5173657690
model-00013-of-00013.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d07ea0ae7a61e7f1fd7fa4aba5aa54485bca920cd2c79021f62a3237b1c3686
3
+ size 1216686375
model.safetensors.index.json ADDED
The diff for this file is too large to render. See raw diff
 
special_tokens_map.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<|startoftext|>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<|return|>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<|endoftext|>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ }
23
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1fdf8862a49ebd1e3019573ab20e936090460cbc552939af4f5fd5291696305
3
+ size 27868342
tokenizer_config.json ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "199998": {
4
+ "content": "<|startoftext|>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "199999": {
12
+ "content": "<|endoftext|>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "200000": {
20
+ "content": "<|reserved_200000|>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "200001": {
28
+ "content": "<|reserved_200001|>",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "200002": {
36
+ "content": "<|return|>",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ },
43
+ "200003": {
44
+ "content": "<|constrain|>",
45
+ "lstrip": false,
46
+ "normalized": false,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": true
50
+ },
51
+ "200004": {
52
+ "content": "<|reserved_200004|>",
53
+ "lstrip": false,
54
+ "normalized": false,
55
+ "rstrip": false,
56
+ "single_word": false,
57
+ "special": true
58
+ },
59
+ "200005": {
60
+ "content": "<|channel|>",
61
+ "lstrip": false,
62
+ "normalized": false,
63
+ "rstrip": false,
64
+ "single_word": false,
65
+ "special": true
66
+ },
67
+ "200006": {
68
+ "content": "<|start|>",
69
+ "lstrip": false,
70
+ "normalized": false,
71
+ "rstrip": false,
72
+ "single_word": false,
73
+ "special": true
74
+ },
75
+ "200007": {
76
+ "content": "<|end|>",
77
+ "lstrip": false,
78
+ "normalized": false,
79
+ "rstrip": false,
80
+ "single_word": false,
81
+ "special": true
82
+ },
83
+ "200008": {
84
+ "content": "<|message|>",
85
+ "lstrip": false,
86
+ "normalized": false,
87
+ "rstrip": false,
88
+ "single_word": false,
89
+ "special": true
90
+ },
91
+ "200009": {
92
+ "content": "<|reserved_200009|>",
93
+ "lstrip": false,
94
+ "normalized": false,
95
+ "rstrip": false,
96
+ "single_word": false,
97
+ "special": true
98
+ },
99
+ "200010": {
100
+ "content": "<|reserved_200010|>",
101
+ "lstrip": false,
102
+ "normalized": false,
103
+ "rstrip": false,
104
+ "single_word": false,
105
+ "special": true
106
+ },
107
+ "200011": {
108
+ "content": "<|reserved_200011|>",
109
+ "lstrip": false,
110
+ "normalized": false,
111
+ "rstrip": false,
112
+ "single_word": false,
113
+ "special": true
114
+ },
115
+ "200012": {
116
+ "content": "<|call|>",
117
+ "lstrip": false,
118
+ "normalized": false,
119
+ "rstrip": false,
120
+ "single_word": false,
121
+ "special": true
122
+ },
123
+ "200013": {
124
+ "content": "<|reserved_200013|>",
125
+ "lstrip": false,
126
+ "normalized": false,
127
+ "rstrip": false,
128
+ "single_word": false,
129
+ "special": true
130
+ },
131
+ "200014": {
132
+ "content": "<|reserved_200014|>",
133
+ "lstrip": false,
134
+ "normalized": false,
135
+ "rstrip": false,
136
+ "single_word": false,
137
+ "special": true
138
+ },
139
+ "200015": {
140
+ "content": "<|reserved_200015|>",
141
+ "lstrip": false,
142
+ "normalized": false,
143
+ "rstrip": false,
144
+ "single_word": false,
145
+ "special": true
146
+ },
147
+ "200016": {
148
+ "content": "<|reserved_200016|>",
149
+ "lstrip": false,
150
+ "normalized": false,
151
+ "rstrip": false,
152
+ "single_word": false,
153
+ "special": true
154
+ },
155
+ "200017": {
156
+ "content": "<|reserved_200017|>",
157
+ "lstrip": false,
158
+ "normalized": false,
159
+ "rstrip": false,
160
+ "single_word": false,
161
+ "special": true
162
+ },
163
+ "200018": {
164
+ "content": "<|endofprompt|>",
165
+ "lstrip": false,
166
+ "normalized": false,
167
+ "rstrip": false,
168
+ "single_word": false,
169
+ "special": true
170
+ }
171
+ },
172
+ "bos_token": "<|startoftext|>",
173
+ "clean_up_tokenization_spaces": false,
174
+ "eos_token": "<|return|>",
175
+ "extra_special_tokens": {},
176
+ "max_length": null,
177
+ "model_input_names": [
178
+ "input_ids",
179
+ "attention_mask"
180
+ ],
181
+ "model_max_length": 1000000000000000019884624838656,
182
+ "pad_to_multiple_of": null,
183
+ "pad_token": "<|endoftext|>",
184
+ "pad_token_type_id": 0,
185
+ "padding_side": "right",
186
+ "tokenizer_class": "PreTrainedTokenizerFast"
187
+ }