Commit
·
5348a89
1
Parent(s):
94b39cb
Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,8 @@ PROMPT FORMAT: [chatml](https://github.com/openai/openai-python/blob/main/chatml
|
|
| 19 |
|
| 20 |
CURRENT MMLU: stem ACC: 43.12 Humanities ACC: 45.15 other ACC: 56.40 social ACC: 57.96 AVERAGE ACC:49.61
|
| 21 |
|
|
|
|
|
|
|
| 22 |
[在制品]
|
| 23 |
|
| 24 |
这是 [通义千问 Qwen/Qwen-7B-Chat](https://huggingface.co/Qwen/Qwen-7B-Chat) 的 LLaMA 化版本,经过重新校准以适应原始的类似 LLaMA/LLaMA-2 的模型结构。
|
|
@@ -29,4 +31,6 @@ CURRENT MMLU: stem ACC: 43.12 Humanities ACC: 45.15 other ACC: 56.40 social ACC:
|
|
| 29 |
|
| 30 |
PROMPT 格式: [chatml](https://github.com/openai/openai-python/blob/main/chatml.md)
|
| 31 |
|
| 32 |
-
当前的 MMLU: stem ACC: 43.12 Humanities ACC: 45.15 other ACC: 56.40 social ACC: 57.96 AVERAGE ACC:49.61
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
CURRENT MMLU: stem ACC: 43.12 Humanities ACC: 45.15 other ACC: 56.40 social ACC: 57.96 AVERAGE ACC:49.61
|
| 21 |
|
| 22 |
+
Issue: Compared to the original Qwen, the MMLU score dropped slightly (4.59) due to insufficient realignment, but the proportions of each item changed, which is unusual.
|
| 23 |
+
|
| 24 |
[在制品]
|
| 25 |
|
| 26 |
这是 [通义千问 Qwen/Qwen-7B-Chat](https://huggingface.co/Qwen/Qwen-7B-Chat) 的 LLaMA 化版本,经过重新校准以适应原始的类似 LLaMA/LLaMA-2 的模型结构。
|
|
|
|
| 31 |
|
| 32 |
PROMPT 格式: [chatml](https://github.com/openai/openai-python/blob/main/chatml.md)
|
| 33 |
|
| 34 |
+
当前的 MMLU: stem ACC: 43.12 Humanities ACC: 45.15 other ACC: 56.40 social ACC: 57.96 AVERAGE ACC:49.61
|
| 35 |
+
|
| 36 |
+
问题:相比原本的Qwen,由于不够充分的重新对齐,MMLU分数略有下降(4.59),但是各项目的比例发生了变化,这是不寻常的。
|