Update README.md

README.md

@@ -1,52 +1,87 @@
 ---
 library_name: transformers
 license: apache-2.0
-license_link: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/blob/main/LICENSE
+license_link: https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507/blob/main/LICENSE
 pipeline_tag: text-generation
-base_model:
-- Qwen/Qwen3-Coder-30B-A3B-Instruct
+base_model: Qwen/Qwen3-30B-A3B-Instruct-2507
 ---
-# Qwen3-Coder-30B-A3B-Instruct-AWQ-8bit
 
-## Method
-[vllm-project/llm-compressor](https://github.com/vllm-project/llm-compressor.git) and [nvidia/Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset) were used to quantize the original model. For further quantization arguments and configurations information, please visit [config.json](https://huggingface.co/cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ-8bit/blob/main/config.json) and [recipe.yaml](https://huggingface.co/cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ-8bit/blob/main/recipe.yaml).
+# Qwen3-30B-A3B-Instruct-2507 AWQ - INT8
+
+## Model Details
+
+### Quantization Details
+
+- **Quantization Method:** cyankiwi AWQ v1.0
+- **Bits:** 8
+- **Group Size:** 32
+- **Calibration Dataset:** [nvidia/Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset)
+- **Quantization Tool:** [llm-compressor](https://github.com/vllm-project/llm-compressor)
+
+### Memory Usage
+
+| **Type** | **Qwen3-30B-A3B-Instruct-2507** | **Qwen3-30B-A3B-Instruct-2507-AWQ-8bit** |
+|:---------------:|:----------------:|:----------------:|
+| **Memory Size** | 56.9 GB | 30.8 GB |
+| **KV Cache per Token** | 48.0 kB | 24.0 kB |
+| **KV Cache per Context** | 12.0 GB | 6.0 GB |
+
+### Evaluations
+
+| **Benchmarks** | **Qwen3-30B-A3B-Instruct-2507** | **Qwen3-30B-A3B-Instruct-2507-AWQ-8bit** |
+|:---------------:|:----------------:|:----------------:|
+| **Perplexity** | 1.61607 | 1.61609 |
+
+- **Evaluation Context Length:** 16384
 
 ## Inference
-Please install the latest vllm releases for better support:
-```
+
+### Prerequisite
+
+```bash
 pip install -U vllm
 ```
 
-Qwen3-Coder-30B-A3B-Instruct-AWQ-8bit example usage:
-```
-vllm serve cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ-8bit \
-    --dtype float16 \
-    --tensor-parallel-size 4  \
-    --enable-auto-tool-choice \
-    --tool-call-parser hermes
+### Basic Usage
+
+```bash
+vllm serve cyankiwi/Qwen3-30B-A3B-Instruct-2507-AWQ-8bit --max-model-len 262144
 ```
 
-# Qwen3-Coder-30B-A3B-Instruct
-<a href="https://chat.qwen.ai/" target="_blank" style="margin: 2px;">
+## Additional Information
+
+### Changelog
+
+- **v1.0.0** - cyankiwi AWQ v1.0 release
+
+### Authors
+
+- **Name:** Ton Cao
+- **Contacts:** [email protected]
+
+# Qwen3-30B-A3B-Instruct-2507
+<a href="https://chat.qwen.ai/?model=Qwen3-30B-A3B-2507" target="_blank" style="margin: 2px;">
     <img alt="Chat" src="https://img.shields.io/badge/%F0%9F%92%9C%EF%B8%8F%20Qwen%20Chat%20-536af5" style="display: inline-block; vertical-align: middle;"/>
 </a>
 
 ## Highlights
 
-**Qwen3-Coder** is available in multiple sizes. Today, we're excited to introduce **Qwen3-Coder-30B-A3B-Instruct**. This streamlined model maintains impressive performance and efficiency, featuring the following key enhancements:  
+We introduce the updated version of the **Qwen3-30B-A3B non-thinking mode**, named **Qwen3-30B-A3B-Instruct-2507**, featuring the following key enhancements:
 
-- **Significant Performance** among open models on **Agentic Coding**, **Agentic Browser-Use**, and other foundational coding tasks.
-- **Long-context Capabilities** with native support for **256K** tokens, extendable up to **1M** tokens using Yarn, optimized for repository-scale understanding.
-- **Agentic Coding** supporting for most platform such as **Qwen Code**, **CLINE**, featuring a specially designed function call format.
+- **Significant improvements** in general capabilities, including **instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage**.
+- **Substantial gains** in long-tail knowledge coverage across **multiple languages**.
+- **Markedly better alignment** with user preferences in **subjective and open-ended tasks**, enabling more helpful responses and higher-quality text generation.
+- **Enhanced capabilities** in **256K long-context understanding**.
 
-![image/jpeg](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-Coder/qwen3-coder-30a3-main.jpg)
+![image/jpeg](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-2507/Qwen3-30B-A3B-Instruct-2507.jpeg)
 
 ## Model Overview
 
-**Qwen3-Coder-30B-A3B-Instruct** has the following features:
+**Qwen3-30B-A3B-Instruct-2507** has the following features:
 - Type: Causal Language Models
 - Training Stage: Pretraining & Post-training
 - Number of Parameters: 30.5B in total and 3.3B activated
+- Number of Paramaters (Non-Embedding): 29.9B
 - Number of Layers: 48
 - Number of Attention Heads (GQA): 32 for Q and 4 for KV
 - Number of Experts: 128
@@ -55,12 +90,53 @@ vllm serve cpatonn/Qwen3-Coder-30B-A3B-Instruct-AWQ-8bit \
 
 **NOTE: This model supports only non-thinking mode and does not generate ``<think></think>`` blocks in its output. Meanwhile, specifying `enable_thinking=False` is no longer required.**
 
-For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3-coder/), [GitHub](https://github.com/QwenLM/Qwen3-Coder), and [Documentation](https://qwen.readthedocs.io/en/latest/).
+For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our [blog](https://qwenlm.github.io/blog/qwen3/), [GitHub](https://github.com/QwenLM/Qwen3), and [Documentation](https://qwen.readthedocs.io/en/latest/).
+
+
+## Performance
+
+|  | Deepseek-V3-0324 | GPT-4o-0327 | Gemini-2.5-Flash Non-Thinking | Qwen3-235B-A22B Non-Thinking | Qwen3-30B-A3B Non-Thinking | Qwen3-30B-A3B-Instruct-2507 |
+|--- | --- | --- | --- | --- | --- | --- |
+| **Knowledge** | | | | | | |
+| MMLU-Pro | **81.2** | 79.8 | 81.1 | 75.2 | 69.1 | 78.4 |
+| MMLU-Redux | 90.4 | **91.3** | 90.6 | 89.2 | 84.1 | 89.3 |
+| GPQA | 68.4 | 66.9 | **78.3** | 62.9 | 54.8 | 70.4 |
+| SuperGPQA | **57.3** | 51.0 | 54.6 | 48.2 | 42.2 | 53.4 |
+| **Reasoning** | | | | | | |
+| AIME25 | 46.6 | 26.7 | **61.6** | 24.7 | 21.6 | 61.3 |
+| HMMT25 | 27.5 | 7.9 | **45.8** | 10.0 | 12.0 | 43.0 |
+| ZebraLogic | 83.4 | 52.6 | 57.9 | 37.7 | 33.2 | **90.0** |
+| LiveBench 20241125 | 66.9 | 63.7 | **69.1** | 62.5 | 59.4 | 69.0 |
+| **Coding** | | | | | | |
+| LiveCodeBench v6 (25.02-25.05) | **45.2** | 35.8 | 40.1 | 32.9 | 29.0 | 43.2 |
+| MultiPL-E | 82.2 | 82.7 | 77.7 | 79.3 | 74.6 | **83.8** |
+| Aider-Polyglot | 55.1 | 45.3 | 44.0 | **59.6** | 24.4 | 35.6 |
+| **Alignment** | | | | | | |
+| IFEval | 82.3 | 83.9 | 84.3 | 83.2 | 83.7 | **84.7** |
+| Arena-Hard v2* | 45.6 | 61.9 | 58.3 | 52.0 | 24.8 | **69.0** |
+| Creative Writing v3 | 81.6 | 84.9 | 84.6 | 80.4 | 68.1 | **86.0** |
+| WritingBench | 74.5 | 75.5 | 80.5 | 77.0 | 72.2 | **85.5** |
+| **Agent** | | | | | | |
+| BFCL-v3 | 64.7 | 66.5 | 66.1 | **68.0** | 58.6 | 65.1 |
+| TAU1-Retail | 49.6 | 60.3# | **65.2** | 65.2 | 38.3 | 59.1 |
+| TAU1-Airline | 32.0 | 42.8# | **48.0** | 32.0 | 18.0 | 40.0 |
+| TAU2-Retail | **71.1** | 66.7# | 64.3 | 64.9 | 31.6 | 57.0 |
+| TAU2-Airline | 36.0 | 42.0# | **42.5** | 36.0 | 18.0 | 38.0 |
+| TAU2-Telecom | **34.0** | 29.8# | 16.9 | 24.6 | 18.4 | 12.3 |
+| **Multilingualism** | | | | | | |
+| MultiIF | 66.5 | 70.4 | 69.4 | 70.2 | **70.8** | 67.9 |
+| MMLU-ProX | 75.8 | 76.2 | **78.3** | 73.2 | 65.1 | 72.0 |
+| INCLUDE | 80.1 | 82.1 | **83.8** | 75.6 | 67.8 | 71.9 |
+| PolyMATH | 32.2 | 25.5 | 41.9 | 27.0 | 23.3 | **43.1** |
+
+*: For reproducibility, we report the win rates evaluated by GPT-4.1.
+
+\#: Results were generated using GPT-4o-20241120, as access to the native function calling API of GPT-4o-0327 was unavailable.
 
 
 ## Quickstart
 
-We advise you to use the latest version of `transformers`.
+The code of Qwen3-MoE has been in the latest Hugging Face `transformers` and we advise you to use the latest version of `transformers`.
 
 With `transformers<4.51.0`, you will encounter the following error:
 ```
@@ -71,7 +147,7 @@ The following contains a code snippet illustrating how to use the model generate
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-model_name = "Qwen/Qwen3-Coder-30B-A3B-Instruct"
+model_name = "Qwen/Qwen3-30B-A3B-Instruct-2507"
 
 # load the tokenizer and the model
 tokenizer = AutoTokenizer.from_pretrained(model_name)
@@ -82,7 +158,7 @@ model = AutoModelForCausalLM.from_pretrained(
 )
 
 # prepare the model input
-prompt = "Write a quick sort algorithm."
+prompt = "Give me a short introduction to large language model."
 messages = [
     {"role": "user", "content": prompt}
 ]
@@ -96,7 +172,7 @@ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
 # conduct text completion
 generated_ids = model.generate(
     **model_inputs,
-    max_new_tokens=65536
+    max_new_tokens=16384
 )
 output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 
 
@@ -105,59 +181,61 @@ content = tokenizer.decode(output_ids, skip_special_tokens=True)
 print("content:", content)
 ```
 
+For deployment, you can use `sglang>=0.4.6.post1` or `vllm>=0.8.5` or to create an OpenAI-compatible API endpoint:
+- SGLang:
+    ```shell
+    python -m sglang.launch_server --model-path Qwen/Qwen3-30B-A3B-Instruct-2507 --context-length 262144
+    ```
+- vLLM:
+    ```shell
+    vllm serve Qwen/Qwen3-30B-A3B-Instruct-2507 --max-model-len 262144
+    ```
+
 **Note: If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as `32,768`.**
 
 For local use, applications such as Ollama, LMStudio, MLX-LM, llama.cpp, and KTransformers have also supported Qwen3.
 
-## Agentic Coding
+## Agentic Use
 
-Qwen3-Coder excels in tool calling capabilities. 
+Qwen3 excels in tool calling capabilities. We recommend using [Qwen-Agent](https://github.com/QwenLM/Qwen-Agent) to make the best use of agentic ability of Qwen3. Qwen-Agent encapsulates tool-calling templates and tool-calling parsers internally, greatly reducing coding complexity.
 
-You can simply define or use any tools as following example.
+To define the available tools, you can use the MCP configuration file, use the integrated tool of Qwen-Agent, or integrate other tools by yourself.
 ```python
-# Your tool implementation
-def square_the_number(num: float) -> dict:
-    return num ** 2
+from qwen_agent.agents import Assistant
+
+# Define LLM
+llm_cfg = {
+    'model': 'Qwen3-30B-A3B-Instruct-2507',
+
+    # Use a custom endpoint compatible with OpenAI API:
+    'model_server': 'http://localhost:8000/v1',  # api_base
+    'api_key': 'EMPTY',
+}
 
 # Define Tools
-tools=[
-    {
-        "type":"function",
-        "function":{
-            "name": "square_the_number",
-            "description": "output the square of the number.",
-            "parameters": {
-                "type": "object",
-                "required": ["input_num"],
-                "properties": {
-                    'input_num': {
-                        'type': 'number', 
-                        'description': 'input_num is a number that will be squared'
-                        }
-                },
+tools = [
+    {'mcpServers': {  # You can specify the MCP configuration file
+            'time': {
+                'command': 'uvx',
+                'args': ['mcp-server-time', '--local-timezone=Asia/Shanghai']
+            },
+            "fetch": {
+                "command": "uvx",
+                "args": ["mcp-server-fetch"]
             }
         }
-    }
+    },
+  'code_interpreter',  # Built-in tools
 ]
 
-import OpenAI
-# Define LLM
-client = OpenAI(
-    # Use a custom endpoint compatible with OpenAI API
-    base_url='http://localhost:8000/v1',  # api_base
-    api_key="EMPTY"
-)
- 
-messages = [{'role': 'user', 'content': 'square the number 1024'}]
-
-completion = client.chat.completions.create(
-    messages=messages,
-    model="Qwen3-Coder-30B-A3B-Instruct",
-    max_tokens=65536,
-    tools=tools,
-)
+# Define Agent
+bot = Assistant(llm=llm_cfg, function_list=tools)
 
-print(completion.choice[0])
+# Streaming generation
+messages = [{'role': 'user', 'content': 'https://qwenlm.github.io/blog/ Introduce the latest developments of Qwen'}]
+for responses in bot.run(messages=messages):
+    pass
+print(responses)
 ```
 
 ## Best Practices
@@ -165,10 +243,14 @@ print(completion.choice[0])
 To achieve optimal performance, we recommend the following settings:
 
 1. **Sampling Parameters**:
-   - We suggest using `temperature=0.7`, `top_p=0.8`, `top_k=20`, `repetition_penalty=1.05`.
+   - We suggest using `Temperature=0.7`, `TopP=0.8`, `TopK=20`, and `MinP=0`.
+   - For supported frameworks, you can adjust the `presence_penalty` parameter between 0 and 2 to reduce endless repetitions. However, using a higher value may occasionally result in language mixing and a slight decrease in model performance.
 
-2. **Adequate Output Length**: We recommend using an output length of 65,536 tokens for most queries, which is adequate for instruct models.
+2. **Adequate Output Length**: We recommend using an output length of 16,384 tokens for most queries, which is adequate for instruct models.
 
+3. **Standardize Output Format**: We recommend using prompts to standardize model outputs when benchmarking.
+   - **Math Problems**: Include "Please reason step by step, and put your final answer within \boxed{}." in the prompt.
+   - **Multiple-Choice Questions**: Add the following JSON structure to the prompt to standardize responses: "Please show your choice in the `answer` field with only the choice letter, e.g., `"answer": "C"`."
 
 ### Citation