Update README.md
Browse files
README.md
CHANGED
|
@@ -16,6 +16,9 @@ K2 is a fully transparent large language model on par with Llama 2 - 70B.
|
|
| 16 |
<center><img src="eval_table_temp.png" alt="eval table"/></center>
|
| 17 |
|
| 18 |
## Datasets and Mix
|
|
|
|
|
|
|
|
|
|
| 19 |
| Dataset | Starting Tokens | Multiplier | Total Tokens |% of Total |
|
| 20 |
| ----------- | ----------- | ----------- | ----------- | ----------- |
|
| 21 |
| dm-math | 4.33B | 3x | 13B | 1% |
|
|
|
|
| 16 |
<center><img src="eval_table_temp.png" alt="eval table"/></center>
|
| 17 |
|
| 18 |
## Datasets and Mix
|
| 19 |
+
|
| 20 |
+
The following data mix was used to train K2 and achieve results in line with Llama 2 70B. The full data sequence will be available soon.
|
| 21 |
+
|
| 22 |
| Dataset | Starting Tokens | Multiplier | Total Tokens |% of Total |
|
| 23 |
| ----------- | ----------- | ----------- | ----------- | ----------- |
|
| 24 |
| dm-math | 4.33B | 3x | 13B | 1% |
|