James commited on
Commit
f57ca4b
·
verified ·
1 Parent(s): f41edb8

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +2 -0
  2. README.md +166 -0
  3. added_tokens.json +28 -0
  4. chat_template.jinja +131 -0
  5. config.json +130 -0
  6. generation_config.json +13 -0
  7. hf_quant_config.json +76 -0
  8. merges.txt +0 -0
  9. model-00001-of-00055.safetensors +3 -0
  10. model-00002-of-00055.safetensors +3 -0
  11. model-00003-of-00055.safetensors +3 -0
  12. model-00004-of-00055.safetensors +3 -0
  13. model-00005-of-00055.safetensors +3 -0
  14. model-00006-of-00055.safetensors +3 -0
  15. model-00007-of-00055.safetensors +3 -0
  16. model-00008-of-00055.safetensors +3 -0
  17. model-00009-of-00055.safetensors +3 -0
  18. model-00010-of-00055.safetensors +3 -0
  19. model-00011-of-00055.safetensors +3 -0
  20. model-00012-of-00055.safetensors +3 -0
  21. model-00013-of-00055.safetensors +3 -0
  22. model-00014-of-00055.safetensors +3 -0
  23. model-00015-of-00055.safetensors +3 -0
  24. model-00016-of-00055.safetensors +3 -0
  25. model-00017-of-00055.safetensors +3 -0
  26. model-00018-of-00055.safetensors +3 -0
  27. model-00019-of-00055.safetensors +3 -0
  28. model-00020-of-00055.safetensors +3 -0
  29. model-00021-of-00055.safetensors +3 -0
  30. model-00022-of-00055.safetensors +3 -0
  31. model-00023-of-00055.safetensors +3 -0
  32. model-00024-of-00055.safetensors +3 -0
  33. model-00025-of-00055.safetensors +3 -0
  34. model-00026-of-00055.safetensors +3 -0
  35. model-00027-of-00055.safetensors +3 -0
  36. model-00028-of-00055.safetensors +3 -0
  37. model-00029-of-00055.safetensors +3 -0
  38. model-00030-of-00055.safetensors +3 -0
  39. model-00031-of-00055.safetensors +3 -0
  40. model-00032-of-00055.safetensors +3 -0
  41. model-00033-of-00055.safetensors +3 -0
  42. model-00034-of-00055.safetensors +3 -0
  43. model-00035-of-00055.safetensors +3 -0
  44. model-00036-of-00055.safetensors +3 -0
  45. model-00037-of-00055.safetensors +3 -0
  46. model-00038-of-00055.safetensors +3 -0
  47. model-00039-of-00055.safetensors +3 -0
  48. model-00040-of-00055.safetensors +3 -0
  49. model-00041-of-00055.safetensors +3 -0
  50. model-00042-of-00055.safetensors +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
37
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,166 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ license_link: https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct/blob/main/LICENSE
5
+ pipeline_tag: text-generation
6
+ ---
7
+
8
+ # Qwen3-Coder-480B-A35B-Instruct
9
+ <a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
10
+ <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
11
+ </a>
12
+
13
+ ## Highlights
14
+
15
+ Today, we're announcing **Qwen3-Coder**, our most agentic code model to date. **Qwen3-Coder** is available in multiple sizes, but we're excited to introduce its most powerful variant first: **Qwen3-Coder-480B-A35B-Instruct**. featuring the following key enhancements:
16
+
17
+ - **Significant Performance** among open models on **Agentic Coding**, **Agentic Browser-Use**, and other foundational coding tasks, achieving results comparable to Claude Sonnet.
18
+ - **Long-context Capabilities** with native support for **256K** tokens, extendable up to **1M** tokens using Yarn, optimized for repository-scale understanding.
19
+ - **Agentic Coding** supporting for most platform such as **Qwen Code**, **CLINE**, featuring a specially designed function call format.
20
+
21
+ ![image/jpeg](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Coder/qwen3-coder-main.jpg)
22
+
23
+ ## Model Overview
24
+
25
+ **Qwen3-480B-A35B-Instruct** has the following features:
26
+ - Type: Causal Language Models
27
+ - Training Stage: Pretraining & Post-training
28
+ - Number of Parameters: 480B in total and 35B activated
29
+ - Number of Layers: 62
30
+ - Number of Attention Heads (GQA): 96 for Q and 8 for KV
31
+ - Number of Experts: 160
32
+ - Number of Activated Experts: 8
33
+ - Context Length: **262,144 natively**.
34
+
35
+ **NOTE: This model supports only non-thinking mode and does not generate ``<think></think>`` blocks in its output. Meanwhile, specifying `enable_thinking=False` is no longer required.**
36
+
37
+ For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3-coder/), [GitHub](https://github.com/QwenLM/Qwen3-Coder), and [Documentation](https://qwen.readthedocs.io/en/latest/).
38
+
39
+
40
+ ## Quickstart
41
+
42
+ We advise you to use the latest version of `transformers`.
43
+
44
+ With `transformers<4.51.0`, you will encounter the following error:
45
+ ```
46
+ KeyError: 'qwen3_moe'
47
+ ```
48
+
49
+ The following contains a code snippet illustrating how to use the model generate content based on given inputs.
50
+ ```python
51
+ from transformers import AutoModelForCausalLM, AutoTokenizer
52
+
53
+ model_name = "Qwen/Qwen3-480B-A35B-Instruct"
54
+
55
+ # load the tokenizer and the model
56
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
57
+ model = AutoModelForCausalLM.from_pretrained(
58
+ model_name,
59
+ torch_dtype="auto",
60
+ device_map="auto"
61
+ )
62
+
63
+ # prepare the model input
64
+ prompt = "Write a quick sort algorithm."
65
+ messages = [
66
+ {"role": "user", "content": prompt}
67
+ ]
68
+ text = tokenizer.apply_chat_template(
69
+ messages,
70
+ tokenize=False,
71
+ add_generation_prompt=True,
72
+ )
73
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
74
+
75
+ # conduct text completion
76
+ generated_ids = model.generate(
77
+ **model_inputs,
78
+ max_new_tokens=65536
79
+ )
80
+ output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
81
+
82
+ content = tokenizer.decode(output_ids, skip_special_tokens=True)
83
+
84
+ print("content:", content)
85
+ ```
86
+
87
+ **Note: If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as `32,768`.**
88
+
89
+ For local use, applications such as Ollama, LMStudio, MLX-LM, llama.cpp, and KTransformers have also supported Qwen3.
90
+
91
+ ## Agentic Coding
92
+
93
+ Qwen3-Coder excels in tool calling capabilities.
94
+
95
+ You can simply define or use any tools as following example.
96
+ ```python
97
+ # Your tool implementation
98
+ def square_the_number(num: float) -> dict:
99
+ return num ** 2
100
+
101
+ # Define Tools
102
+ tools=[
103
+ {
104
+ "type":"function",
105
+ "function":{
106
+ "name": "square_the_number",
107
+ "description": "output the square of the number.",
108
+ "parameters": {
109
+ "type": "object",
110
+ "required": ["input_num"],
111
+ "properties": {
112
+ 'input_num': {
113
+ 'type': 'number',
114
+ 'description': 'input_num is a number that will be squared'
115
+ }
116
+ },
117
+ }
118
+ }
119
+ }
120
+ ]
121
+
122
+ import OpenAI
123
+ # Define LLM
124
+ client = OpenAI(
125
+ # Use a custom endpoint compatible with OpenAI API
126
+ base_url='http://localhost:8000/v1', # api_base
127
+ api_key="EMPTY"
128
+ )
129
+
130
+ messages = [{'role': 'user', 'content': 'square the number 1024'}]
131
+
132
+ completion = client.chat.completions.create(
133
+ messages=messages,
134
+ model="Qwen3-480B-A35B-Instruct",
135
+ max_tokens=65536,
136
+ tools=tools,
137
+ )
138
+
139
+ print(completion.choice[0])
140
+ ```
141
+
142
+ ## Best Practices
143
+
144
+ To achieve optimal performance, we recommend the following settings:
145
+
146
+ 1. **Sampling Parameters**:
147
+ - We suggest using `temperature=0.7`, `top_p=0.8`, `top_k=20`, `repetition_penalty=1.05`.
148
+
149
+ 2. **Adequate Output Length**: We recommend using an output length of 65,536 tokens for most queries, which is adequate for instruct models.
150
+
151
+
152
+ ### Citation
153
+
154
+ If you find our work helpful, feel free to give us a cite.
155
+
156
+ ```
157
+ @misc{qwen3technicalreport,
158
+ title={Qwen3 Technical Report},
159
+ author={Qwen Team},
160
+ year={2025},
161
+ eprint={2505.09388},
162
+ archivePrefix={arXiv},
163
+ primaryClass={cs.CL},
164
+ url={https://arxiv.org/abs/2505.09388},
165
+ }
166
+ ```
added_tokens.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "</think>": 151668,
3
+ "</tool_call>": 151658,
4
+ "</tool_response>": 151666,
5
+ "<think>": 151667,
6
+ "<tool_call>": 151657,
7
+ "<tool_response>": 151665,
8
+ "<|box_end|>": 151649,
9
+ "<|box_start|>": 151648,
10
+ "<|endoftext|>": 151643,
11
+ "<|file_sep|>": 151664,
12
+ "<|fim_middle|>": 151660,
13
+ "<|fim_pad|>": 151662,
14
+ "<|fim_prefix|>": 151659,
15
+ "<|fim_suffix|>": 151661,
16
+ "<|im_end|>": 151645,
17
+ "<|im_start|>": 151644,
18
+ "<|image_pad|>": 151655,
19
+ "<|object_ref_end|>": 151647,
20
+ "<|object_ref_start|>": 151646,
21
+ "<|quad_end|>": 151651,
22
+ "<|quad_start|>": 151650,
23
+ "<|repo_name|>": 151663,
24
+ "<|video_pad|>": 151656,
25
+ "<|vision_end|>": 151653,
26
+ "<|vision_pad|>": 151654,
27
+ "<|vision_start|>": 151652
28
+ }
chat_template.jinja ADDED
@@ -0,0 +1,131 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {% macro render_item_list(item_list, tag_name='required') %}
2
+ {%- if item_list is defined and item_list is iterable and item_list | length > 0 %}
3
+ {%- if tag_name %}{{- '\n<' ~ tag_name ~ '>' -}}{% endif %}
4
+ {{- '[' }}
5
+ {%- for item in item_list -%}
6
+ {%- if loop.index > 1 %}{{- ", "}}{% endif -%}
7
+ {%- if item is string -%}
8
+ {{ "`" ~ item ~ "`" }}
9
+ {%- else -%}
10
+ {{ item }}
11
+ {%- endif -%}
12
+ {%- endfor -%}
13
+ {{- ']' }}
14
+ {%- if tag_name %}{{- '</' ~ tag_name ~ '>' -}}{% endif %}
15
+ {%- endif %}
16
+ {% endmacro %}
17
+
18
+ {%- if messages[0]["role"] == "system" %}
19
+ {%- set system_message = messages[0]["content"] %}
20
+ {%- set loop_messages = messages[1:] %}
21
+ {%- else %}
22
+ {%- set loop_messages = messages %}
23
+ {%- endif %}
24
+
25
+ {%- if not tools is defined %}
26
+ {%- set tools = [] %}
27
+ {%- endif %}
28
+
29
+ {%- if system_message is defined %}
30
+ {{- "<|im_start|>system\n" + system_message }}
31
+ {%- else %}
32
+ {%- if tools is iterable and tools | length > 0 %}
33
+ {{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }}
34
+ {%- endif %}
35
+ {%- endif %}
36
+ {%- if tools is iterable and tools | length > 0 %}
37
+ {{- "\n\nYou have access to the following functions:\n\n" }}
38
+ {{- "<tools>" }}
39
+ {%- for tool in tools %}
40
+ {%- if tool.function is defined %}
41
+ {%- set tool = tool.function %}
42
+ {%- endif %}
43
+ {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
44
+ {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
45
+ {{- '\n<parameters>' }}
46
+ {%- for param_name, param_fields in tool.parameters.properties|items %}
47
+ {{- '\n<parameter>' }}
48
+ {{- '\n<name>' ~ param_name ~ '</name>' }}
49
+ {%- if param_fields.type is defined %}
50
+ {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
51
+ {%- endif %}
52
+ {%- if param_fields.description is defined %}
53
+ {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
54
+ {%- endif %}
55
+ {{- render_item_list(param_fields.enum, 'enum') }}
56
+ {%- set handled_keys = ['type', 'description', 'enum', 'required'] %}
57
+ {%- for json_key in param_fields.keys() | reject("in", handled_keys) %}
58
+ {%- set normed_json_key = json_key | replace("-", "_") | replace(" ", "_") | replace("$", "") %}
59
+ {%- if param_fields[json_key] is mapping %}
60
+ {{- '\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | tojson | safe) ~ '</' ~ normed_json_key ~ '>' }}
61
+ {%- else %}
62
+ {{-'\n<' ~ normed_json_key ~ '>' ~ (param_fields[json_key] | string) ~ '</' ~ normed_json_key ~ '>' }}
63
+ {%- endif %}
64
+ {%- endfor %}
65
+ {{- render_item_list(param_fields.required, 'required') }}
66
+ {{- '\n</parameter>' }}
67
+ {%- endfor %}
68
+ {{- render_item_list(tool.parameters.required, 'required') }}
69
+ {{- '\n</parameters>' }}
70
+ {%- if tool.return is defined %}
71
+ {%- if tool.return is mapping %}
72
+ {{- '\n<return>' ~ (tool.return | tojson | safe) ~ '</return>' }}
73
+ {%- else %}
74
+ {{- '\n<return>' ~ (tool.return | string) ~ '</return>' }}
75
+ {%- endif %}
76
+ {%- endif %}
77
+ {{- '\n</function>' }}
78
+ {%- endfor %}
79
+ {{- "\n</tools>" }}
80
+ {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
81
+ {%- endif %}
82
+ {%- if system_message is defined %}
83
+ {{- '<|im_end|>\n' }}
84
+ {%- else %}
85
+ {%- if tools is iterable and tools | length > 0 %}
86
+ {{- '<|im_end|>\n' }}
87
+ {%- endif %}
88
+ {%- endif %}
89
+ {%- for message in loop_messages %}
90
+ {%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
91
+ {{- '<|im_start|>' + message.role }}
92
+ {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
93
+ {{- '\n' + message.content | trim + '\n' }}
94
+ {%- endif %}
95
+ {%- for tool_call in message.tool_calls %}
96
+ {%- if tool_call.function is defined %}
97
+ {%- set tool_call = tool_call.function %}
98
+ {%- endif %}
99
+ {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
100
+ {%- if tool_call.arguments is defined %}
101
+ {%- for args_name, args_value in tool_call.arguments|items %}
102
+ {{- '<parameter=' + args_name + '>\n' }}
103
+ {%- set args_value = args_value if args_value is string else args_value | string %}
104
+ {{- args_value }}
105
+ {{- '\n</parameter>\n' }}
106
+ {%- endfor %}
107
+ {%- endif %}
108
+ {{- '</function>\n</tool_call>' }}
109
+ {%- endfor %}
110
+ {{- '<|im_end|>\n' }}
111
+ {%- elif message.role == "user" or message.role == "system" or message.role == "assistant" %}
112
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
113
+ {%- elif message.role == "tool" %}
114
+ {%- if loop.previtem and loop.previtem.role != "tool" %}
115
+ {{- '<|im_start|>user\n' }}
116
+ {%- endif %}
117
+ {{- '<tool_response>\n' }}
118
+ {{- message.content }}
119
+ {{- '\n</tool_response>\n' }}
120
+ {%- if not loop.last and loop.nextitem.role != "tool" %}
121
+ {{- '<|im_end|>\n' }}
122
+ {%- elif loop.last %}
123
+ {{- '<|im_end|>\n' }}
124
+ {%- endif %}
125
+ {%- else %}
126
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
127
+ {%- endif %}
128
+ {%- endfor %}
129
+ {%- if add_generation_prompt %}
130
+ {{- '<|im_start|>assistant\n' }}
131
+ {%- endif %}
config.json ADDED
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3MoeForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "decoder_sparse_step": 1,
8
+ "eos_token_id": 151645,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 6144,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 8192,
14
+ "max_position_embeddings": 262144,
15
+ "max_window_layers": 62,
16
+ "mlp_only_layers": [],
17
+ "model_type": "qwen3_moe",
18
+ "moe_intermediate_size": 2560,
19
+ "norm_topk_prob": true,
20
+ "num_attention_heads": 96,
21
+ "num_experts": 160,
22
+ "num_experts_per_tok": 8,
23
+ "num_hidden_layers": 62,
24
+ "num_key_value_heads": 8,
25
+ "output_router_logits": false,
26
+ "qkv_bias": false,
27
+ "rms_norm_eps": 1e-06,
28
+ "rope_scaling": null,
29
+ "rope_theta": 10000000,
30
+ "router_aux_loss_coef": 0.0,
31
+ "shared_expert_intermediate_size": 0,
32
+ "sliding_window": null,
33
+ "tie_word_embeddings": false,
34
+ "torch_dtype": "bfloat16",
35
+ "transformers_version": "4.53.3",
36
+ "use_cache": true,
37
+ "use_qk_norm": true,
38
+ "use_sliding_window": false,
39
+ "vocab_size": 151936,
40
+ "quantization_config": {
41
+ "config_groups": {
42
+ "group_0": {
43
+ "input_activations": {
44
+ "dynamic": false,
45
+ "num_bits": 4,
46
+ "type": "float",
47
+ "group_size": 16
48
+ },
49
+ "weights": {
50
+ "dynamic": false,
51
+ "num_bits": 4,
52
+ "type": "float",
53
+ "group_size": 16
54
+ }
55
+ }
56
+ },
57
+ "ignore": [
58
+ "model.layers.0.mlp.gate",
59
+ "model.layers.1.mlp.gate",
60
+ "model.layers.10.mlp.gate",
61
+ "model.layers.11.mlp.gate",
62
+ "model.layers.12.mlp.gate",
63
+ "model.layers.13.mlp.gate",
64
+ "model.layers.14.mlp.gate",
65
+ "model.layers.15.mlp.gate",
66
+ "model.layers.16.mlp.gate",
67
+ "model.layers.17.mlp.gate",
68
+ "model.layers.18.mlp.gate",
69
+ "model.layers.19.mlp.gate",
70
+ "model.layers.2.mlp.gate",
71
+ "model.layers.20.mlp.gate",
72
+ "model.layers.21.mlp.gate",
73
+ "model.layers.22.mlp.gate",
74
+ "model.layers.23.mlp.gate",
75
+ "model.layers.24.mlp.gate",
76
+ "model.layers.25.mlp.gate",
77
+ "model.layers.26.mlp.gate",
78
+ "model.layers.27.mlp.gate",
79
+ "model.layers.28.mlp.gate",
80
+ "model.layers.29.mlp.gate",
81
+ "model.layers.3.mlp.gate",
82
+ "model.layers.30.mlp.gate",
83
+ "model.layers.31.mlp.gate",
84
+ "model.layers.32.mlp.gate",
85
+ "model.layers.33.mlp.gate",
86
+ "model.layers.34.mlp.gate",
87
+ "model.layers.35.mlp.gate",
88
+ "model.layers.36.mlp.gate",
89
+ "model.layers.37.mlp.gate",
90
+ "model.layers.38.mlp.gate",
91
+ "model.layers.39.mlp.gate",
92
+ "model.layers.4.mlp.gate",
93
+ "model.layers.40.mlp.gate",
94
+ "model.layers.41.mlp.gate",
95
+ "model.layers.42.mlp.gate",
96
+ "model.layers.43.mlp.gate",
97
+ "model.layers.44.mlp.gate",
98
+ "model.layers.45.mlp.gate",
99
+ "model.layers.46.mlp.gate",
100
+ "model.layers.47.mlp.gate",
101
+ "model.layers.48.mlp.gate",
102
+ "model.layers.49.mlp.gate",
103
+ "model.layers.5.mlp.gate",
104
+ "model.layers.50.mlp.gate",
105
+ "model.layers.51.mlp.gate",
106
+ "model.layers.52.mlp.gate",
107
+ "model.layers.53.mlp.gate",
108
+ "model.layers.54.mlp.gate",
109
+ "model.layers.55.mlp.gate",
110
+ "model.layers.56.mlp.gate",
111
+ "model.layers.57.mlp.gate",
112
+ "model.layers.58.mlp.gate",
113
+ "model.layers.59.mlp.gate",
114
+ "model.layers.6.mlp.gate",
115
+ "model.layers.60.mlp.gate",
116
+ "model.layers.61.mlp.gate",
117
+ "model.layers.7.mlp.gate",
118
+ "model.layers.8.mlp.gate",
119
+ "model.layers.9.mlp.gate",
120
+ "lm_head"
121
+ ],
122
+ "quant_algo": "NVFP4",
123
+ "kv_cache_scheme": "FP8",
124
+ "producer": {
125
+ "name": "modelopt",
126
+ "version": "0.33.0"
127
+ },
128
+ "quant_library": "modelopt"
129
+ }
130
+ }
generation_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_sample": true,
3
+ "eos_token_id": [
4
+ 151645,
5
+ 151643
6
+ ],
7
+ "pad_token_id": 151643,
8
+ "repetition_penalty": 1.05,
9
+ "temperature": 0.7,
10
+ "top_k": 20,
11
+ "top_p": 0.8,
12
+ "transformers_version": "4.53.3"
13
+ }
hf_quant_config.json ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "producer": {
3
+ "name": "modelopt",
4
+ "version": "0.33.0"
5
+ },
6
+ "quantization": {
7
+ "quant_algo": "NVFP4",
8
+ "kv_cache_quant_algo": "FP8",
9
+ "group_size": 16,
10
+ "exclude_modules": [
11
+ "model.layers.0.mlp.gate",
12
+ "model.layers.1.mlp.gate",
13
+ "model.layers.10.mlp.gate",
14
+ "model.layers.11.mlp.gate",
15
+ "model.layers.12.mlp.gate",
16
+ "model.layers.13.mlp.gate",
17
+ "model.layers.14.mlp.gate",
18
+ "model.layers.15.mlp.gate",
19
+ "model.layers.16.mlp.gate",
20
+ "model.layers.17.mlp.gate",
21
+ "model.layers.18.mlp.gate",
22
+ "model.layers.19.mlp.gate",
23
+ "model.layers.2.mlp.gate",
24
+ "model.layers.20.mlp.gate",
25
+ "model.layers.21.mlp.gate",
26
+ "model.layers.22.mlp.gate",
27
+ "model.layers.23.mlp.gate",
28
+ "model.layers.24.mlp.gate",
29
+ "model.layers.25.mlp.gate",
30
+ "model.layers.26.mlp.gate",
31
+ "model.layers.27.mlp.gate",
32
+ "model.layers.28.mlp.gate",
33
+ "model.layers.29.mlp.gate",
34
+ "model.layers.3.mlp.gate",
35
+ "model.layers.30.mlp.gate",
36
+ "model.layers.31.mlp.gate",
37
+ "model.layers.32.mlp.gate",
38
+ "model.layers.33.mlp.gate",
39
+ "model.layers.34.mlp.gate",
40
+ "model.layers.35.mlp.gate",
41
+ "model.layers.36.mlp.gate",
42
+ "model.layers.37.mlp.gate",
43
+ "model.layers.38.mlp.gate",
44
+ "model.layers.39.mlp.gate",
45
+ "model.layers.4.mlp.gate",
46
+ "model.layers.40.mlp.gate",
47
+ "model.layers.41.mlp.gate",
48
+ "model.layers.42.mlp.gate",
49
+ "model.layers.43.mlp.gate",
50
+ "model.layers.44.mlp.gate",
51
+ "model.layers.45.mlp.gate",
52
+ "model.layers.46.mlp.gate",
53
+ "model.layers.47.mlp.gate",
54
+ "model.layers.48.mlp.gate",
55
+ "model.layers.49.mlp.gate",
56
+ "model.layers.5.mlp.gate",
57
+ "model.layers.50.mlp.gate",
58
+ "model.layers.51.mlp.gate",
59
+ "model.layers.52.mlp.gate",
60
+ "model.layers.53.mlp.gate",
61
+ "model.layers.54.mlp.gate",
62
+ "model.layers.55.mlp.gate",
63
+ "model.layers.56.mlp.gate",
64
+ "model.layers.57.mlp.gate",
65
+ "model.layers.58.mlp.gate",
66
+ "model.layers.59.mlp.gate",
67
+ "model.layers.6.mlp.gate",
68
+ "model.layers.60.mlp.gate",
69
+ "model.layers.61.mlp.gate",
70
+ "model.layers.7.mlp.gate",
71
+ "model.layers.8.mlp.gate",
72
+ "model.layers.9.mlp.gate",
73
+ "lm_head"
74
+ ]
75
+ }
76
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model-00001-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1aa09ec5298bdf58ff67e44e06b9b2aedd07573e34fbf1cb0306a2a971cc012
3
+ size 4995779920
model-00002-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6b4460a7aed52c09f3197211e83f7b5a2974b5c17d2dc528a241bdcf9ea68762
3
+ size 4995708720
model-00003-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:948ce53755c09477cb1abdfca006e34c85c9648e7c1cb3f1f0ca9ae2d2f8aa62
3
+ size 5000253260
model-00004-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:420f5e0fa5844bafc2be9a0d42c5b70f3353a2d6db75f1f1ece981882419b64c
3
+ size 4996691588
model-00005-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5cab499ff7f1409bfb51b8118c8a04ecda293bda2d46aa52f4b5b7da2494337
3
+ size 4995708424
model-00006-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97a4ba329fcfac54205fc415c9c7dea78907ea0a45f047c72c0f10fffda9a636
3
+ size 4995708432
model-00007-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d84498eec02829e24ae712c967f12e6e51dc46f45384a42e0fb9a9bdab4e639
3
+ size 4995708424
model-00008-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:183d86febd4df70c57887ff7b8fe2ad6d8e28d0cd500b3dec4578b06b34cbaa0
3
+ size 4995708712
model-00009-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f2bde3e45dec694868b9e6f36b94250b1ceca740975b9d205e3c6775c6883ee
3
+ size 4995708728
model-00010-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92a1b7b192d62ed8507043e3a05df3fbfc56c3b3f99607e99ebf0f926e90570c
3
+ size 5000255116
model-00011-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71d8e69de3fa59e57b7a7dd1de5f01a58ccdda1108594903014317c53041c45b
3
+ size 4996693948
model-00012-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eb9742cd5b7024f0aa020b901f3a3fd33c1c09ee4931bc9e60deec392d78e847
3
+ size 4995710664
model-00013-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a03768a319aabe204abde2510e8b263d67b31eb7f8db354883b622b662f70190
3
+ size 4995710664
model-00014-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c19df2132c83884b2a3f001b1ad12a0be6c614e9668b0111cf0232f1a564595
3
+ size 4995710776
model-00015-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:197954cff07ad196e5f3bafc379fe0de9ee8a797b6216586de4e88ede4efd6cd
3
+ size 4995710960
model-00016-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6ec1331a22e4c534eb1e14bf5ddb43742e90a3e4a77550e9bce43fe9b5cb604
3
+ size 4995710960
model-00017-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c86056d61fdb59705600d0ee01575f05b94d3c71bac5517afcdf3b986299f0c0
3
+ size 5000255140
model-00018-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f0e7978ac99317233447086712f3f1600294af5179ca1baa1832a63d76b5e33
3
+ size 4996693948
model-00019-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ebfd7ca829175b7b389314722dc93fe438ee07d156a48fd45a3781ade0a4d2c9
3
+ size 4995710672
model-00020-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13f3aa7b6c3823d6b64079339407eeac11a90850929a05913cc8dc857643f4e7
3
+ size 4995710664
model-00021-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b15e163709cd4df8daae3b9b7b062705add03903a31bb477fc1edf0759411e41
3
+ size 4995710880
model-00022-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:980fc51a385e005ff566450018855c795924b086088882c7d7aa3b53abc327b1
3
+ size 4995710968
model-00023-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc149242152af6099a153b61e6170f4b135a1b36440ef91262f69df94d57d950
3
+ size 5000255396
model-00024-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b19dfb88adbd884dc97411288b10826db27c0c9003e6e4ba8097559e5cd66b29
3
+ size 4996693876
model-00025-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f8ce0607e19c0ede34e79c1104131503ce443a60b533010ce3a64b690b1e6a8
3
+ size 4995710664
model-00026-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db20b9aa84d53dfb710f106ecc97ac009a29f1b9d9aa7a809a93ed5194a5f65f
3
+ size 4995710664
model-00027-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c60ece2db16fd33569a1452795e33e5ebe261dbf606c825c128f4032913ed53e
3
+ size 4995710704
model-00028-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce61949068c722feaa5c9e76a81cf13a606ae5857cd0fdb5ef85bd87098c83df
3
+ size 4995710960
model-00029-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58815347430deb33aa731f8e5ae6a8de21037b102fc8279ec778bc9d38e765b1
3
+ size 4995710960
model-00030-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:694117f4ebe1a45571c84f9032116e5b6df89add626f4ad41db721def6934d12
3
+ size 5000255212
model-00031-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2cafa58ee5a88ac50070dc06404940e66310edba9f0a0b8e7c81b8c1b2ce885
3
+ size 4996693948
model-00032-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e294c53e772e2de8977e95294af306e185a6b3ed1c09545ebb5d44348ee63594
3
+ size 4995710672
model-00033-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c06d298d97568e6a27e2db40ce60c63920d955eb9592757ab3d8c6e06365962d
3
+ size 4995710664
model-00034-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93d59327ced123676433729f9404a49edffc76b8ebc780e45b2f89e60816358b
3
+ size 4995710808
model-00035-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b91fd2f0508e34b6c901440670d8af3baf26cb91c2b70fb0d275c806c6208704
3
+ size 4995710968
model-00036-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11ac6c234712621d3536d84c7f9e3fdab447120b793824aa11d61501a58a3bb3
3
+ size 4969192216
model-00037-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68e1b441b673580dd459aa2095373d90cd6ea83f350a4dcb003372fae874070e
3
+ size 5000230324
model-00038-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:89c006e797ced65ac2ac34d65d275b2055f94a9626b5431b5a1e8222be50fbce
3
+ size 4996693948
model-00039-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:706b14863f13aede0177e4ae6cd64a79b6a48bfb52f35bbe607f3abe65aa1972
3
+ size 4995710664
model-00040-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:229e1197b10d458f0b4f5cbb5324f133b732d497552a22bd26794bf960234436
3
+ size 4995710672
model-00041-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:434a36f79c9dfbf89e39f3ff918284531be69de30751051d27d4d9e6e80a5853
3
+ size 4995710904
model-00042-of-00055.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7fb61f77ca3ee985a4e6bffcaa9e38b7b4c34354a553986721ff0f34ba8e049d
3
+ size 4995710960