Hugging Face's logo

rmdhirr
/
gemma-sft-instruct-1700

PEFT
Safetensors
Model card Files
xet
Community
rmdhirr commited on
Commit
6b01594
·
verified ·
1 Parent(s): 73c58aa

Upload folder using huggingface_hub

Browse files
Files changed (15) hide show
  1. .gitattributes +1 -0
  2. README.md +202 -0
  3. adapter_config.json +40 -0
  4. adapter_model.safetensors +3 -0
  5. added_tokens.json +3 -0
  6. chat_template.jinja +47 -0
  7. optimizer.pt +3 -0
  8. rng_state.pth +3 -0
  9. scheduler.pt +3 -0
  10. special_tokens_map.json +33 -0
  11. tokenizer.json +3 -0
  12. tokenizer.model +3 -0
  13. tokenizer_config.json +0 -0
  14. trainer_state.json +2911 -0
  15. training_args.bin +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: google/gemma-3-12b-it
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.14.0
adapter_config.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "google/gemma-3-12b-it",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": true,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 48,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.05,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": [
21
+ "lm_head",
22
+ "embed_tokens"
23
+ ],
24
+ "peft_type": "LORA",
25
+ "r": 16,
26
+ "rank_pattern": {},
27
+ "revision": null,
28
+ "target_modules": [
29
+ "gate_proj",
30
+ "up_proj",
31
+ "o_proj",
32
+ "v_proj",
33
+ "k_proj",
34
+ "down_proj",
35
+ "q_proj"
36
+ ],
37
+ "task_type": "CAUSAL_LM",
38
+ "use_dora": false,
39
+ "use_rslora": false
40
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ebffd8d41ea375d6a2ac9dd3fd76eb18353e4f6c9e4e86c98d5511f3eb801a2d
3
+ size 4301469136
added_tokens.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "<image_soft_token>": 262144
3
+ }
chat_template.jinja ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {{ bos_token }}
2
+ {%- if messages[0]['role'] == 'system' -%}
3
+ {%- if messages[0]['content'] is string -%}
4
+ {%- set first_user_prefix = messages[0]['content'] + '
5
+
6
+ ' -%}
7
+ {%- else -%}
8
+ {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
9
+
10
+ ' -%}
11
+ {%- endif -%}
12
+ {%- set loop_messages = messages[1:] -%}
13
+ {%- else -%}
14
+ {%- set first_user_prefix = "" -%}
15
+ {%- set loop_messages = messages -%}
16
+ {%- endif -%}
17
+ {%- for message in loop_messages -%}
18
+ {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
19
+ {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
20
+ {%- endif -%}
21
+ {%- if (message['role'] == 'assistant') -%}
22
+ {%- set role = "model" -%}
23
+ {%- else -%}
24
+ {%- set role = message['role'] -%}
25
+ {%- endif -%}
26
+ {{ '<start_of_turn>' + role + '
27
+ ' + (first_user_prefix if loop.first else "") }}
28
+ {%- if message['content'] is string -%}
29
+ {{ message['content'] | trim }}
30
+ {%- elif message['content'] is iterable -%}
31
+ {%- for item in message['content'] -%}
32
+ {%- if item['type'] == 'image' -%}
33
+ {{ '<start_of_image>' }}
34
+ {%- elif item['type'] == 'text' -%}
35
+ {{ item['text'] | trim }}
36
+ {%- endif -%}
37
+ {%- endfor -%}
38
+ {%- else -%}
39
+ {{ raise_exception("Invalid content type") }}
40
+ {%- endif -%}
41
+ {{ '<end_of_turn>
42
+ ' }}
43
+ {%- endfor -%}
44
+ {%- if add_generation_prompt -%}
45
+ {{'<start_of_turn>model
46
+ '}}
47
+ {%- endif -%}
optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b3db9973439a261c46db23afb0aa7a741453647609e05f0a89d43a3bc812508
3
+ size 8341591628
rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d287a4fc93c7965c0d4618c850b9f96d5440911a5cefa2ec166a7553f3d3ff7
3
+ size 14244
scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0903df1186a095b27cc521a20e2338b43ecee0891e2319771a737cd09365d3ce
3
+ size 1064
special_tokens_map.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "boi_token": "<start_of_image>",
3
+ "bos_token": {
4
+ "content": "<bos>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ "eoi_token": "<end_of_image>",
11
+ "eos_token": {
12
+ "content": "<eos>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false
17
+ },
18
+ "image_token": "<image_soft_token>",
19
+ "pad_token": {
20
+ "content": "<pad>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false
25
+ },
26
+ "unk_token": {
27
+ "content": "<unk>",
28
+ "lstrip": false,
29
+ "normalized": false,
30
+ "rstrip": false,
31
+ "single_word": false
32
+ }
33
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
3
+ size 33384568
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
3
+ size 4689074
tokenizer_config.json ADDED
The diff for this file is too large to render. See raw diff
 
trainer_state.json ADDED
@@ -0,0 +1,2911 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 1.0,
6
+ "eval_steps": 300,
7
+ "global_step": 1770,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.0028256569652444193,
14
+ "grad_norm": 125.03303527832031,
15
+ "learning_rate": 6.666666666666666e-07,
16
+ "loss": 19.3957,
17
+ "mean_token_accuracy": 0.443931570649147,
18
+ "step": 5
19
+ },
20
+ {
21
+ "epoch": 0.005651313930488839,
22
+ "grad_norm": 179.0009765625,
23
+ "learning_rate": 1.5e-06,
24
+ "loss": 19.1611,
25
+ "mean_token_accuracy": 0.4493979915976524,
26
+ "step": 10
27
+ },
28
+ {
29
+ "epoch": 0.008476970895733259,
30
+ "grad_norm": 117.55085754394531,
31
+ "learning_rate": 2.3333333333333336e-06,
32
+ "loss": 19.097,
33
+ "mean_token_accuracy": 0.45788785591721537,
34
+ "step": 15
35
+ },
36
+ {
37
+ "epoch": 0.011302627860977677,
38
+ "grad_norm": 109.04960632324219,
39
+ "learning_rate": 3.1666666666666667e-06,
40
+ "loss": 19.207,
41
+ "mean_token_accuracy": 0.4440133422613144,
42
+ "step": 20
43
+ },
44
+ {
45
+ "epoch": 0.014128284826222097,
46
+ "grad_norm": 118.16978454589844,
47
+ "learning_rate": 4e-06,
48
+ "loss": 17.2711,
49
+ "mean_token_accuracy": 0.4634488716721535,
50
+ "step": 25
51
+ },
52
+ {
53
+ "epoch": 0.016953941791466517,
54
+ "grad_norm": 161.21022033691406,
55
+ "learning_rate": 4.833333333333333e-06,
56
+ "loss": 17.7255,
57
+ "mean_token_accuracy": 0.4570723704993725,
58
+ "step": 30
59
+ },
60
+ {
61
+ "epoch": 0.019779598756710936,
62
+ "grad_norm": 111.49407958984375,
63
+ "learning_rate": 5.666666666666667e-06,
64
+ "loss": 18.1505,
65
+ "mean_token_accuracy": 0.46947305351495744,
66
+ "step": 35
67
+ },
68
+ {
69
+ "epoch": 0.022605255721955354,
70
+ "grad_norm": 94.02478790283203,
71
+ "learning_rate": 5.999955686683124e-06,
72
+ "loss": 16.5811,
73
+ "mean_token_accuracy": 0.4923714891076088,
74
+ "step": 40
75
+ },
76
+ {
77
+ "epoch": 0.025430912687199773,
78
+ "grad_norm": 93.49138641357422,
79
+ "learning_rate": 5.999684887820798e-06,
80
+ "loss": 14.6779,
81
+ "mean_token_accuracy": 0.4900531888008118,
82
+ "step": 45
83
+ },
84
+ {
85
+ "epoch": 0.028256569652444195,
86
+ "grad_norm": 89.19295501708984,
87
+ "learning_rate": 5.9991679308007015e-06,
88
+ "loss": 15.0693,
89
+ "mean_token_accuracy": 0.48212875574827196,
90
+ "step": 50
91
+ },
92
+ {
93
+ "epoch": 0.031082226617688613,
94
+ "grad_norm": 117.02027130126953,
95
+ "learning_rate": 5.998404858045021e-06,
96
+ "loss": 14.6772,
97
+ "mean_token_accuracy": 0.4899917796254158,
98
+ "step": 55
99
+ },
100
+ {
101
+ "epoch": 0.033907883582933035,
102
+ "grad_norm": 72.73108673095703,
103
+ "learning_rate": 5.997395732172529e-06,
104
+ "loss": 13.4677,
105
+ "mean_token_accuracy": 0.4900201290845871,
106
+ "step": 60
107
+ },
108
+ {
109
+ "epoch": 0.03673354054817745,
110
+ "grad_norm": 77.72334289550781,
111
+ "learning_rate": 5.996140635993444e-06,
112
+ "loss": 13.2612,
113
+ "mean_token_accuracy": 0.5328952759504318,
114
+ "step": 65
115
+ },
116
+ {
117
+ "epoch": 0.03955919751342187,
118
+ "grad_norm": 80.2801284790039,
119
+ "learning_rate": 5.994639672502639e-06,
120
+ "loss": 11.504,
121
+ "mean_token_accuracy": 0.559516441822052,
122
+ "step": 70
123
+ },
124
+ {
125
+ "epoch": 0.04238485447866629,
126
+ "grad_norm": 61.040287017822266,
127
+ "learning_rate": 5.992892964871187e-06,
128
+ "loss": 11.0794,
129
+ "mean_token_accuracy": 0.5566341817378998,
130
+ "step": 75
131
+ },
132
+ {
133
+ "epoch": 0.04521051144391071,
134
+ "grad_norm": 61.21868896484375,
135
+ "learning_rate": 5.990900656436255e-06,
136
+ "loss": 11.9816,
137
+ "mean_token_accuracy": 0.5282905742526054,
138
+ "step": 80
139
+ },
140
+ {
141
+ "epoch": 0.04803616840915513,
142
+ "grad_norm": 114.57307434082031,
143
+ "learning_rate": 5.988662910689342e-06,
144
+ "loss": 10.3066,
145
+ "mean_token_accuracy": 0.5789915665984153,
146
+ "step": 85
147
+ },
148
+ {
149
+ "epoch": 0.050861825374399545,
150
+ "grad_norm": 76.79177856445312,
151
+ "learning_rate": 5.986179911262859e-06,
152
+ "loss": 10.7346,
153
+ "mean_token_accuracy": 0.5474225521087647,
154
+ "step": 90
155
+ },
156
+ {
157
+ "epoch": 0.05368748233964397,
158
+ "grad_norm": 50.477203369140625,
159
+ "learning_rate": 5.983451861915061e-06,
160
+ "loss": 11.5344,
161
+ "mean_token_accuracy": 0.5186601117253303,
162
+ "step": 95
163
+ },
164
+ {
165
+ "epoch": 0.05651313930488839,
166
+ "grad_norm": 65.024169921875,
167
+ "learning_rate": 5.980478986513332e-06,
168
+ "loss": 10.1865,
169
+ "mean_token_accuracy": 0.5476699441671371,
170
+ "step": 100
171
+ },
172
+ {
173
+ "epoch": 0.059338796270132804,
174
+ "grad_norm": 78.97113037109375,
175
+ "learning_rate": 5.977261529015807e-06,
176
+ "loss": 10.5502,
177
+ "mean_token_accuracy": 0.5594380274415016,
178
+ "step": 105
179
+ },
180
+ {
181
+ "epoch": 0.062164453235377226,
182
+ "grad_norm": 63.45710372924805,
183
+ "learning_rate": 5.9737997534513565e-06,
184
+ "loss": 10.1732,
185
+ "mean_token_accuracy": 0.5498600453138351,
186
+ "step": 110
187
+ },
188
+ {
189
+ "epoch": 0.06499011020062165,
190
+ "grad_norm": 43.65032958984375,
191
+ "learning_rate": 5.970093943897915e-06,
192
+ "loss": 9.483,
193
+ "mean_token_accuracy": 0.5819845095276832,
194
+ "step": 115
195
+ },
196
+ {
197
+ "epoch": 0.06781576716586607,
198
+ "grad_norm": 65.38373565673828,
199
+ "learning_rate": 5.966144404459178e-06,
200
+ "loss": 9.8998,
201
+ "mean_token_accuracy": 0.5803950399160385,
202
+ "step": 120
203
+ },
204
+ {
205
+ "epoch": 0.07064142413111048,
206
+ "grad_norm": 51.20427703857422,
207
+ "learning_rate": 5.96195145923964e-06,
208
+ "loss": 9.5007,
209
+ "mean_token_accuracy": 0.5401258394122124,
210
+ "step": 125
211
+ },
212
+ {
213
+ "epoch": 0.0734670810963549,
214
+ "grad_norm": 43.631324768066406,
215
+ "learning_rate": 5.957515452317996e-06,
216
+ "loss": 9.3663,
217
+ "mean_token_accuracy": 0.5585527911782264,
218
+ "step": 130
219
+ },
220
+ {
221
+ "epoch": 0.07629273806159932,
222
+ "grad_norm": 49.34850311279297,
223
+ "learning_rate": 5.952836747718916e-06,
224
+ "loss": 10.0569,
225
+ "mean_token_accuracy": 0.5572537362575531,
226
+ "step": 135
227
+ },
228
+ {
229
+ "epoch": 0.07911839502684374,
230
+ "grad_norm": 56.23272705078125,
231
+ "learning_rate": 5.947915729383162e-06,
232
+ "loss": 9.1624,
233
+ "mean_token_accuracy": 0.6116252914071083,
234
+ "step": 140
235
+ },
236
+ {
237
+ "epoch": 0.08194405199208817,
238
+ "grad_norm": 54.321495056152344,
239
+ "learning_rate": 5.942752801136086e-06,
240
+ "loss": 9.0426,
241
+ "mean_token_accuracy": 0.5830309092998505,
242
+ "step": 145
243
+ },
244
+ {
245
+ "epoch": 0.08476970895733257,
246
+ "grad_norm": 49.35163116455078,
247
+ "learning_rate": 5.937348386654492e-06,
248
+ "loss": 8.7897,
249
+ "mean_token_accuracy": 0.5833149090409279,
250
+ "step": 150
251
+ },
252
+ {
253
+ "epoch": 0.087595365922577,
254
+ "grad_norm": 41.16096878051758,
255
+ "learning_rate": 5.9317029294318685e-06,
256
+ "loss": 9.2698,
257
+ "mean_token_accuracy": 0.5732270866632462,
258
+ "step": 155
259
+ },
260
+ {
261
+ "epoch": 0.09042102288782142,
262
+ "grad_norm": 56.901546478271484,
263
+ "learning_rate": 5.925816892741992e-06,
264
+ "loss": 8.8971,
265
+ "mean_token_accuracy": 0.5911598846316337,
266
+ "step": 160
267
+ },
268
+ {
269
+ "epoch": 0.09324667985306584,
270
+ "grad_norm": 37.740020751953125,
271
+ "learning_rate": 5.919690759600914e-06,
272
+ "loss": 9.3825,
273
+ "mean_token_accuracy": 0.5646735802292824,
274
+ "step": 165
275
+ },
276
+ {
277
+ "epoch": 0.09607233681831026,
278
+ "grad_norm": 43.82129669189453,
279
+ "learning_rate": 5.913325032727323e-06,
280
+ "loss": 8.921,
281
+ "mean_token_accuracy": 0.5748174145817757,
282
+ "step": 170
283
+ },
284
+ {
285
+ "epoch": 0.09889799378355468,
286
+ "grad_norm": 43.662750244140625,
287
+ "learning_rate": 5.906720234501286e-06,
288
+ "loss": 8.329,
289
+ "mean_token_accuracy": 0.5832746580243111,
290
+ "step": 175
291
+ },
292
+ {
293
+ "epoch": 0.10172365074879909,
294
+ "grad_norm": 63.78599548339844,
295
+ "learning_rate": 5.899876906921388e-06,
296
+ "loss": 9.403,
297
+ "mean_token_accuracy": 0.5746660903096199,
298
+ "step": 180
299
+ },
300
+ {
301
+ "epoch": 0.10454930771404351,
302
+ "grad_norm": 38.50173568725586,
303
+ "learning_rate": 5.892795611560252e-06,
304
+ "loss": 7.9569,
305
+ "mean_token_accuracy": 0.6175401106476783,
306
+ "step": 185
307
+ },
308
+ {
309
+ "epoch": 0.10737496467928793,
310
+ "grad_norm": 46.76047134399414,
311
+ "learning_rate": 5.885476929518457e-06,
312
+ "loss": 8.8664,
313
+ "mean_token_accuracy": 0.5698649421334266,
314
+ "step": 190
315
+ },
316
+ {
317
+ "epoch": 0.11020062164453236,
318
+ "grad_norm": 34.551719665527344,
319
+ "learning_rate": 5.877921461376848e-06,
320
+ "loss": 8.9507,
321
+ "mean_token_accuracy": 0.5774942457675933,
322
+ "step": 195
323
+ },
324
+ {
325
+ "epoch": 0.11302627860977678,
326
+ "grad_norm": 46.14909362792969,
327
+ "learning_rate": 5.8701298271472565e-06,
328
+ "loss": 8.3937,
329
+ "mean_token_accuracy": 0.5869078159332275,
330
+ "step": 200
331
+ },
332
+ {
333
+ "epoch": 0.11585193557502119,
334
+ "grad_norm": 38.04121017456055,
335
+ "learning_rate": 5.862102666221617e-06,
336
+ "loss": 9.1566,
337
+ "mean_token_accuracy": 0.5589849069714546,
338
+ "step": 205
339
+ },
340
+ {
341
+ "epoch": 0.11867759254026561,
342
+ "grad_norm": 47.98865509033203,
343
+ "learning_rate": 5.853840637319504e-06,
344
+ "loss": 8.3272,
345
+ "mean_token_accuracy": 0.613035187125206,
346
+ "step": 210
347
+ },
348
+ {
349
+ "epoch": 0.12150324950551003,
350
+ "grad_norm": 52.02131271362305,
351
+ "learning_rate": 5.845344418434068e-06,
352
+ "loss": 9.2413,
353
+ "mean_token_accuracy": 0.5741102159023285,
354
+ "step": 215
355
+ },
356
+ {
357
+ "epoch": 0.12432890647075445,
358
+ "grad_norm": 46.88343048095703,
359
+ "learning_rate": 5.8366147067764056e-06,
360
+ "loss": 8.468,
361
+ "mean_token_accuracy": 0.5816206842660904,
362
+ "step": 220
363
+ },
364
+ {
365
+ "epoch": 0.12715456343599887,
366
+ "grad_norm": 43.369300842285156,
367
+ "learning_rate": 5.8276522187183435e-06,
368
+ "loss": 7.4452,
369
+ "mean_token_accuracy": 0.6464217156171799,
370
+ "step": 225
371
+ },
372
+ {
373
+ "epoch": 0.1299802204012433,
374
+ "grad_norm": 46.008766174316406,
375
+ "learning_rate": 5.818457689733649e-06,
376
+ "loss": 8.2276,
377
+ "mean_token_accuracy": 0.6063334688544273,
378
+ "step": 230
379
+ },
380
+ {
381
+ "epoch": 0.13280587736648772,
382
+ "grad_norm": 38.97382736206055,
383
+ "learning_rate": 5.809031874337681e-06,
384
+ "loss": 8.029,
385
+ "mean_token_accuracy": 0.6226910144090653,
386
+ "step": 235
387
+ },
388
+ {
389
+ "epoch": 0.13563153433173214,
390
+ "grad_norm": 44.3059196472168,
391
+ "learning_rate": 5.7993755460254685e-06,
392
+ "loss": 8.4051,
393
+ "mean_token_accuracy": 0.5975374907255173,
394
+ "step": 240
395
+ },
396
+ {
397
+ "epoch": 0.13845719129697653,
398
+ "grad_norm": 56.00782012939453,
399
+ "learning_rate": 5.789489497208243e-06,
400
+ "loss": 8.6075,
401
+ "mean_token_accuracy": 0.5798447385430336,
402
+ "step": 245
403
+ },
404
+ {
405
+ "epoch": 0.14128284826222096,
406
+ "grad_norm": 39.610755920410156,
407
+ "learning_rate": 5.779374539148403e-06,
408
+ "loss": 7.9846,
409
+ "mean_token_accuracy": 0.593911099433899,
410
+ "step": 250
411
+ },
412
+ {
413
+ "epoch": 0.14410850522746538,
414
+ "grad_norm": 40.99591827392578,
415
+ "learning_rate": 5.769031501892949e-06,
416
+ "loss": 8.3259,
417
+ "mean_token_accuracy": 0.5966995969414711,
418
+ "step": 255
419
+ },
420
+ {
421
+ "epoch": 0.1469341621927098,
422
+ "grad_norm": 37.0196533203125,
423
+ "learning_rate": 5.7584612342053655e-06,
424
+ "loss": 7.7289,
425
+ "mean_token_accuracy": 0.6313483536243438,
426
+ "step": 260
427
+ },
428
+ {
429
+ "epoch": 0.14975981915795422,
430
+ "grad_norm": 36.436134338378906,
431
+ "learning_rate": 5.7476646034959705e-06,
432
+ "loss": 9.042,
433
+ "mean_token_accuracy": 0.5726025938987732,
434
+ "step": 265
435
+ },
436
+ {
437
+ "epoch": 0.15258547612319864,
438
+ "grad_norm": 49.964351654052734,
439
+ "learning_rate": 5.736642495750733e-06,
440
+ "loss": 7.8111,
441
+ "mean_token_accuracy": 0.6032327204942703,
442
+ "step": 270
443
+ },
444
+ {
445
+ "epoch": 0.15541113308844307,
446
+ "grad_norm": 37.209232330322266,
447
+ "learning_rate": 5.725395815458571e-06,
448
+ "loss": 6.2546,
449
+ "mean_token_accuracy": 0.6550982385873795,
450
+ "step": 275
451
+ },
452
+ {
453
+ "epoch": 0.1582367900536875,
454
+ "grad_norm": 39.0145149230957,
455
+ "learning_rate": 5.713925485537126e-06,
456
+ "loss": 8.2466,
457
+ "mean_token_accuracy": 0.5892508149147033,
458
+ "step": 280
459
+ },
460
+ {
461
+ "epoch": 0.1610624470189319,
462
+ "grad_norm": 46.74445724487305,
463
+ "learning_rate": 5.702232447257029e-06,
464
+ "loss": 8.1135,
465
+ "mean_token_accuracy": 0.6003284469246865,
466
+ "step": 285
467
+ },
468
+ {
469
+ "epoch": 0.16388810398417633,
470
+ "grad_norm": 61.96944046020508,
471
+ "learning_rate": 5.6903176601646535e-06,
472
+ "loss": 8.0814,
473
+ "mean_token_accuracy": 0.6058224648237228,
474
+ "step": 290
475
+ },
476
+ {
477
+ "epoch": 0.16671376094942075,
478
+ "grad_norm": 41.85062026977539,
479
+ "learning_rate": 5.6781821020033794e-06,
480
+ "loss": 7.7756,
481
+ "mean_token_accuracy": 0.6040422543883324,
482
+ "step": 295
483
+ },
484
+ {
485
+ "epoch": 0.16953941791466515,
486
+ "grad_norm": 37.07870864868164,
487
+ "learning_rate": 5.665826768633358e-06,
488
+ "loss": 7.4973,
489
+ "mean_token_accuracy": 0.6139729157090187,
490
+ "step": 300
491
+ },
492
+ {
493
+ "epoch": 0.16953941791466515,
494
+ "eval_loss": 1.9660394191741943,
495
+ "eval_mean_token_accuracy": 0.6081940480295172,
496
+ "eval_runtime": 60.361,
497
+ "eval_samples_per_second": 26.06,
498
+ "eval_steps_per_second": 3.264,
499
+ "step": 300
500
+ },
501
+ {
502
+ "epoch": 0.17236507487990957,
503
+ "grad_norm": 36.64670181274414,
504
+ "learning_rate": 5.6532526739497834e-06,
505
+ "loss": 7.6396,
506
+ "mean_token_accuracy": 0.6107118725776672,
507
+ "step": 305
508
+ },
509
+ {
510
+ "epoch": 0.175190731845154,
511
+ "grad_norm": 54.716243743896484,
512
+ "learning_rate": 5.640460849799702e-06,
513
+ "loss": 7.2263,
514
+ "mean_token_accuracy": 0.6144478976726532,
515
+ "step": 310
516
+ },
517
+ {
518
+ "epoch": 0.1780163888103984,
519
+ "grad_norm": 55.097808837890625,
520
+ "learning_rate": 5.627452345897328e-06,
521
+ "loss": 7.0887,
522
+ "mean_token_accuracy": 0.6283742040395737,
523
+ "step": 315
524
+ },
525
+ {
526
+ "epoch": 0.18084204577564283,
527
+ "grad_norm": 61.68784713745117,
528
+ "learning_rate": 5.614228229737906e-06,
529
+ "loss": 7.8888,
530
+ "mean_token_accuracy": 0.6202467530965805,
531
+ "step": 320
532
+ },
533
+ {
534
+ "epoch": 0.18366770274088726,
535
+ "grad_norm": 50.16268539428711,
536
+ "learning_rate": 5.600789586510113e-06,
537
+ "loss": 8.1697,
538
+ "mean_token_accuracy": 0.6052482485771179,
539
+ "step": 325
540
+ },
541
+ {
542
+ "epoch": 0.18649335970613168,
543
+ "grad_norm": 34.98442840576172,
544
+ "learning_rate": 5.587137519007004e-06,
545
+ "loss": 7.6766,
546
+ "mean_token_accuracy": 0.610385374724865,
547
+ "step": 330
548
+ },
549
+ {
550
+ "epoch": 0.1893190166713761,
551
+ "grad_norm": 54.69295120239258,
552
+ "learning_rate": 5.5732731475355135e-06,
553
+ "loss": 7.4116,
554
+ "mean_token_accuracy": 0.6147442162036896,
555
+ "step": 335
556
+ },
557
+ {
558
+ "epoch": 0.19214467363662052,
559
+ "grad_norm": 36.03273391723633,
560
+ "learning_rate": 5.559197609824526e-06,
561
+ "loss": 7.7704,
562
+ "mean_token_accuracy": 0.6110788837075234,
563
+ "step": 340
564
+ },
565
+ {
566
+ "epoch": 0.19497033060186494,
567
+ "grad_norm": 45.89215087890625,
568
+ "learning_rate": 5.544912060931511e-06,
569
+ "loss": 6.0621,
570
+ "mean_token_accuracy": 0.6817868202924728,
571
+ "step": 345
572
+ },
573
+ {
574
+ "epoch": 0.19779598756710937,
575
+ "grad_norm": 43.32875442504883,
576
+ "learning_rate": 5.530417673147736e-06,
577
+ "loss": 7.5385,
578
+ "mean_token_accuracy": 0.6150149628520012,
579
+ "step": 350
580
+ },
581
+ {
582
+ "epoch": 0.20062164453235376,
583
+ "grad_norm": 35.37779998779297,
584
+ "learning_rate": 5.515715635902064e-06,
585
+ "loss": 6.6067,
586
+ "mean_token_accuracy": 0.6687664300203323,
587
+ "step": 355
588
+ },
589
+ {
590
+ "epoch": 0.20344730149759818,
591
+ "grad_norm": 44.46392822265625,
592
+ "learning_rate": 5.500807155663359e-06,
593
+ "loss": 6.2822,
594
+ "mean_token_accuracy": 0.6637924790382386,
595
+ "step": 360
596
+ },
597
+ {
598
+ "epoch": 0.2062729584628426,
599
+ "grad_norm": 36.08877182006836,
600
+ "learning_rate": 5.485693455841464e-06,
601
+ "loss": 6.894,
602
+ "mean_token_accuracy": 0.6358136102557183,
603
+ "step": 365
604
+ },
605
+ {
606
+ "epoch": 0.20909861542808703,
607
+ "grad_norm": 34.14577865600586,
608
+ "learning_rate": 5.470375776686822e-06,
609
+ "loss": 7.4445,
610
+ "mean_token_accuracy": 0.6168029010295868,
611
+ "step": 370
612
+ },
613
+ {
614
+ "epoch": 0.21192427239333145,
615
+ "grad_norm": 40.84408950805664,
616
+ "learning_rate": 5.454855375188691e-06,
617
+ "loss": 7.0881,
618
+ "mean_token_accuracy": 0.6331988245248794,
619
+ "step": 375
620
+ },
621
+ {
622
+ "epoch": 0.21474992935857587,
623
+ "grad_norm": 47.68820571899414,
624
+ "learning_rate": 5.439133524971994e-06,
625
+ "loss": 7.1198,
626
+ "mean_token_accuracy": 0.6500597685575485,
627
+ "step": 380
628
+ },
629
+ {
630
+ "epoch": 0.2175755863238203,
631
+ "grad_norm": 62.971771240234375,
632
+ "learning_rate": 5.4232115161928125e-06,
633
+ "loss": 7.3537,
634
+ "mean_token_accuracy": 0.6475826740264893,
635
+ "step": 385
636
+ },
637
+ {
638
+ "epoch": 0.2204012432890647,
639
+ "grad_norm": 47.17545700073242,
640
+ "learning_rate": 5.407090655432498e-06,
641
+ "loss": 6.484,
642
+ "mean_token_accuracy": 0.6483543753623963,
643
+ "step": 390
644
+ },
645
+ {
646
+ "epoch": 0.22322690025430914,
647
+ "grad_norm": 47.76310348510742,
648
+ "learning_rate": 5.390772265590469e-06,
649
+ "loss": 7.1721,
650
+ "mean_token_accuracy": 0.6379205271601677,
651
+ "step": 395
652
+ },
653
+ {
654
+ "epoch": 0.22605255721955356,
655
+ "grad_norm": 41.001304626464844,
656
+ "learning_rate": 5.374257685775642e-06,
657
+ "loss": 7.3882,
658
+ "mean_token_accuracy": 0.6161196917295456,
659
+ "step": 400
660
+ },
661
+ {
662
+ "epoch": 0.22887821418479798,
663
+ "grad_norm": 41.1621208190918,
664
+ "learning_rate": 5.357548271196545e-06,
665
+ "loss": 6.6539,
666
+ "mean_token_accuracy": 0.6448143661022187,
667
+ "step": 405
668
+ },
669
+ {
670
+ "epoch": 0.23170387115004237,
671
+ "grad_norm": 56.77714920043945,
672
+ "learning_rate": 5.34064539305011e-06,
673
+ "loss": 6.7501,
674
+ "mean_token_accuracy": 0.6442455291748047,
675
+ "step": 410
676
+ },
677
+ {
678
+ "epoch": 0.2345295281152868,
679
+ "grad_norm": 34.29147720336914,
680
+ "learning_rate": 5.323550438409145e-06,
681
+ "loss": 7.1571,
682
+ "mean_token_accuracy": 0.6502243876457214,
683
+ "step": 415
684
+ },
685
+ {
686
+ "epoch": 0.23735518508053122,
687
+ "grad_norm": 38.84526824951172,
688
+ "learning_rate": 5.306264810108515e-06,
689
+ "loss": 7.4697,
690
+ "mean_token_accuracy": 0.6136257261037826,
691
+ "step": 420
692
+ },
693
+ {
694
+ "epoch": 0.24018084204577564,
695
+ "grad_norm": 39.21842575073242,
696
+ "learning_rate": 5.288789926630018e-06,
697
+ "loss": 5.6676,
698
+ "mean_token_accuracy": 0.6803564548492431,
699
+ "step": 425
700
+ },
701
+ {
702
+ "epoch": 0.24300649901102006,
703
+ "grad_norm": 43.55912780761719,
704
+ "learning_rate": 5.27112722198599e-06,
705
+ "loss": 7.6674,
706
+ "mean_token_accuracy": 0.6143711119890213,
707
+ "step": 430
708
+ },
709
+ {
710
+ "epoch": 0.24583215597626448,
711
+ "grad_norm": 47.082237243652344,
712
+ "learning_rate": 5.253278145601618e-06,
713
+ "loss": 6.9372,
714
+ "mean_token_accuracy": 0.6543397754430771,
715
+ "step": 435
716
+ },
717
+ {
718
+ "epoch": 0.2486578129415089,
719
+ "grad_norm": 71.78733825683594,
720
+ "learning_rate": 5.235244162196007e-06,
721
+ "loss": 6.3831,
722
+ "mean_token_accuracy": 0.6731083989143372,
723
+ "step": 440
724
+ },
725
+ {
726
+ "epoch": 0.2514834699067533,
727
+ "grad_norm": 41.5792121887207,
728
+ "learning_rate": 5.217026751661978e-06,
729
+ "loss": 5.9193,
730
+ "mean_token_accuracy": 0.6984533488750457,
731
+ "step": 445
732
+ },
733
+ {
734
+ "epoch": 0.25430912687199775,
735
+ "grad_norm": 32.45619201660156,
736
+ "learning_rate": 5.198627408944628e-06,
737
+ "loss": 6.1931,
738
+ "mean_token_accuracy": 0.6726161792874337,
739
+ "step": 450
740
+ },
741
+ {
742
+ "epoch": 0.25713478383724214,
743
+ "grad_norm": 54.69667434692383,
744
+ "learning_rate": 5.180047643918653e-06,
745
+ "loss": 5.6802,
746
+ "mean_token_accuracy": 0.700816172361374,
747
+ "step": 455
748
+ },
749
+ {
750
+ "epoch": 0.2599604408024866,
751
+ "grad_norm": 52.139469146728516,
752
+ "learning_rate": 5.161288981264445e-06,
753
+ "loss": 7.4401,
754
+ "mean_token_accuracy": 0.6105516791343689,
755
+ "step": 460
756
+ },
757
+ {
758
+ "epoch": 0.262786097767731,
759
+ "grad_norm": 38.770294189453125,
760
+ "learning_rate": 5.142352960342976e-06,
761
+ "loss": 6.3299,
762
+ "mean_token_accuracy": 0.6778733760118485,
763
+ "step": 465
764
+ },
765
+ {
766
+ "epoch": 0.26561175473297544,
767
+ "grad_norm": 44.44423294067383,
768
+ "learning_rate": 5.123241135069471e-06,
769
+ "loss": 6.3576,
770
+ "mean_token_accuracy": 0.6793156564235687,
771
+ "step": 470
772
+ },
773
+ {
774
+ "epoch": 0.26843741169821983,
775
+ "grad_norm": 52.27141189575195,
776
+ "learning_rate": 5.103955073785902e-06,
777
+ "loss": 5.4571,
778
+ "mean_token_accuracy": 0.7010635808110237,
779
+ "step": 475
780
+ },
781
+ {
782
+ "epoch": 0.2712630686634643,
783
+ "grad_norm": 41.901485443115234,
784
+ "learning_rate": 5.084496359132275e-06,
785
+ "loss": 6.2462,
786
+ "mean_token_accuracy": 0.6626246273517609,
787
+ "step": 480
788
+ },
789
+ {
790
+ "epoch": 0.2740887256287087,
791
+ "grad_norm": 39.49611282348633,
792
+ "learning_rate": 5.064866587916764e-06,
793
+ "loss": 6.3324,
794
+ "mean_token_accuracy": 0.6691112801432609,
795
+ "step": 485
796
+ },
797
+ {
798
+ "epoch": 0.27691438259395307,
799
+ "grad_norm": 41.7744140625,
800
+ "learning_rate": 5.045067370984676e-06,
801
+ "loss": 7.2696,
802
+ "mean_token_accuracy": 0.6365453451871872,
803
+ "step": 490
804
+ },
805
+ {
806
+ "epoch": 0.2797400395591975,
807
+ "grad_norm": 49.72770690917969,
808
+ "learning_rate": 5.02510033308626e-06,
809
+ "loss": 6.5657,
810
+ "mean_token_accuracy": 0.6508561789989471,
811
+ "step": 495
812
+ },
813
+ {
814
+ "epoch": 0.2825656965244419,
815
+ "grad_norm": 32.61491394042969,
816
+ "learning_rate": 5.004967112743376e-06,
817
+ "loss": 6.6128,
818
+ "mean_token_accuracy": 0.6579498335719108,
819
+ "step": 500
820
+ },
821
+ {
822
+ "epoch": 0.28539135348968636,
823
+ "grad_norm": 49.389400482177734,
824
+ "learning_rate": 4.984669362115039e-06,
825
+ "loss": 6.7253,
826
+ "mean_token_accuracy": 0.6457798436284066,
827
+ "step": 505
828
+ },
829
+ {
830
+ "epoch": 0.28821701045493076,
831
+ "grad_norm": 51.6247673034668,
832
+ "learning_rate": 4.964208746861841e-06,
833
+ "loss": 6.2031,
834
+ "mean_token_accuracy": 0.6751710534095764,
835
+ "step": 510
836
+ },
837
+ {
838
+ "epoch": 0.2910426674201752,
839
+ "grad_norm": 28.600732803344727,
840
+ "learning_rate": 4.943586946009259e-06,
841
+ "loss": 6.6904,
842
+ "mean_token_accuracy": 0.6619319871068001,
843
+ "step": 515
844
+ },
845
+ {
846
+ "epoch": 0.2938683243854196,
847
+ "grad_norm": 43.22124481201172,
848
+ "learning_rate": 4.92280565180988e-06,
849
+ "loss": 6.7604,
850
+ "mean_token_accuracy": 0.6478939458727837,
851
+ "step": 520
852
+ },
853
+ {
854
+ "epoch": 0.29669398135066405,
855
+ "grad_norm": 41.86116027832031,
856
+ "learning_rate": 4.901866569604527e-06,
857
+ "loss": 6.0308,
858
+ "mean_token_accuracy": 0.6734458118677139,
859
+ "step": 525
860
+ },
861
+ {
862
+ "epoch": 0.29951963831590844,
863
+ "grad_norm": 41.02956771850586,
864
+ "learning_rate": 4.8807714176823205e-06,
865
+ "loss": 7.0681,
866
+ "mean_token_accuracy": 0.6355025738477706,
867
+ "step": 530
868
+ },
869
+ {
870
+ "epoch": 0.3023452952811529,
871
+ "grad_norm": 44.11521530151367,
872
+ "learning_rate": 4.859521927139664e-06,
873
+ "loss": 6.1855,
874
+ "mean_token_accuracy": 0.6703523576259613,
875
+ "step": 535
876
+ },
877
+ {
878
+ "epoch": 0.3051709522463973,
879
+ "grad_norm": 32.89723587036133,
880
+ "learning_rate": 4.838119841738205e-06,
881
+ "loss": 6.0888,
882
+ "mean_token_accuracy": 0.6834497556090355,
883
+ "step": 540
884
+ },
885
+ {
886
+ "epoch": 0.3079966092116417,
887
+ "grad_norm": 32.18568420410156,
888
+ "learning_rate": 4.816566917761719e-06,
889
+ "loss": 6.9014,
890
+ "mean_token_accuracy": 0.6485872358083725,
891
+ "step": 545
892
+ },
893
+ {
894
+ "epoch": 0.31082226617688613,
895
+ "grad_norm": 46.209686279296875,
896
+ "learning_rate": 4.794864923872006e-06,
897
+ "loss": 6.1183,
898
+ "mean_token_accuracy": 0.6952649801969528,
899
+ "step": 550
900
+ },
901
+ {
902
+ "epoch": 0.3136479231421305,
903
+ "grad_norm": 42.59083938598633,
904
+ "learning_rate": 4.773015640963735e-06,
905
+ "loss": 6.1898,
906
+ "mean_token_accuracy": 0.6966498523950577,
907
+ "step": 555
908
+ },
909
+ {
910
+ "epoch": 0.316473580107375,
911
+ "grad_norm": 34.99408721923828,
912
+ "learning_rate": 4.751020862018315e-06,
913
+ "loss": 6.4017,
914
+ "mean_token_accuracy": 0.6635166749358177,
915
+ "step": 560
916
+ },
917
+ {
918
+ "epoch": 0.31929923707261937,
919
+ "grad_norm": 44.781253814697266,
920
+ "learning_rate": 4.728882391956751e-06,
921
+ "loss": 6.294,
922
+ "mean_token_accuracy": 0.6908501267433167,
923
+ "step": 565
924
+ },
925
+ {
926
+ "epoch": 0.3221248940378638,
927
+ "grad_norm": 38.54107666015625,
928
+ "learning_rate": 4.706602047491535e-06,
929
+ "loss": 6.3961,
930
+ "mean_token_accuracy": 0.6642886430025101,
931
+ "step": 570
932
+ },
933
+ {
934
+ "epoch": 0.3249505510031082,
935
+ "grad_norm": 39.30413055419922,
936
+ "learning_rate": 4.68418165697756e-06,
937
+ "loss": 7.0987,
938
+ "mean_token_accuracy": 0.6421632379293442,
939
+ "step": 575
940
+ },
941
+ {
942
+ "epoch": 0.32777620796835266,
943
+ "grad_norm": 32.5236930847168,
944
+ "learning_rate": 4.66162306026209e-06,
945
+ "loss": 6.3601,
946
+ "mean_token_accuracy": 0.6777586549520492,
947
+ "step": 580
948
+ },
949
+ {
950
+ "epoch": 0.33060186493359706,
951
+ "grad_norm": 44.02008056640625,
952
+ "learning_rate": 4.638928108533771e-06,
953
+ "loss": 6.7745,
954
+ "mean_token_accuracy": 0.6339758485555649,
955
+ "step": 585
956
+ },
957
+ {
958
+ "epoch": 0.3334275218988415,
959
+ "grad_norm": 42.38660430908203,
960
+ "learning_rate": 4.616098664170726e-06,
961
+ "loss": 6.7977,
962
+ "mean_token_accuracy": 0.6368318185210228,
963
+ "step": 590
964
+ },
965
+ {
966
+ "epoch": 0.3362531788640859,
967
+ "grad_norm": 56.86142349243164,
968
+ "learning_rate": 4.5931366005877205e-06,
969
+ "loss": 6.5369,
970
+ "mean_token_accuracy": 0.6513374149799347,
971
+ "step": 595
972
+ },
973
+ {
974
+ "epoch": 0.3390788358293303,
975
+ "grad_norm": 46.34036636352539,
976
+ "learning_rate": 4.570043802082435e-06,
977
+ "loss": 6.975,
978
+ "mean_token_accuracy": 0.6324821501970291,
979
+ "step": 600
980
+ },
981
+ {
982
+ "epoch": 0.3390788358293303,
983
+ "eval_loss": 1.6515789031982422,
984
+ "eval_mean_token_accuracy": 0.6573993170321896,
985
+ "eval_runtime": 60.1378,
986
+ "eval_samples_per_second": 26.157,
987
+ "eval_steps_per_second": 3.276,
988
+ "step": 600
989
+ },
990
+ {
991
+ "epoch": 0.34190449279457474,
992
+ "grad_norm": 34.469337463378906,
993
+ "learning_rate": 4.546822163680829e-06,
994
+ "loss": 5.6408,
995
+ "mean_token_accuracy": 0.6926419764757157,
996
+ "step": 605
997
+ },
998
+ {
999
+ "epoch": 0.34473014975981914,
1000
+ "grad_norm": 54.577877044677734,
1001
+ "learning_rate": 4.523473590981639e-06,
1002
+ "loss": 5.1717,
1003
+ "mean_token_accuracy": 0.708769902586937,
1004
+ "step": 610
1005
+ },
1006
+ {
1007
+ "epoch": 0.3475558067250636,
1008
+ "grad_norm": 42.55693817138672,
1009
+ "learning_rate": 4.5e-06,
1010
+ "loss": 6.131,
1011
+ "mean_token_accuracy": 0.6689537853002548,
1012
+ "step": 615
1013
+ },
1014
+ {
1015
+ "epoch": 0.350381463690308,
1016
+ "grad_norm": 32.82284927368164,
1017
+ "learning_rate": 4.476403317010212e-06,
1018
+ "loss": 6.4724,
1019
+ "mean_token_accuracy": 0.6572571873664856,
1020
+ "step": 620
1021
+ },
1022
+ {
1023
+ "epoch": 0.35320712065555243,
1024
+ "grad_norm": 40.628170013427734,
1025
+ "learning_rate": 4.452685478387672e-06,
1026
+ "loss": 7.2712,
1027
+ "mean_token_accuracy": 0.6308314383029938,
1028
+ "step": 625
1029
+ },
1030
+ {
1031
+ "epoch": 0.3560327776207968,
1032
+ "grad_norm": 32.64014434814453,
1033
+ "learning_rate": 4.4288484304499706e-06,
1034
+ "loss": 5.8603,
1035
+ "mean_token_accuracy": 0.6960221052169799,
1036
+ "step": 630
1037
+ },
1038
+ {
1039
+ "epoch": 0.3588584345860413,
1040
+ "grad_norm": 45.46329116821289,
1041
+ "learning_rate": 4.404894129297172e-06,
1042
+ "loss": 6.1867,
1043
+ "mean_token_accuracy": 0.6657388493418693,
1044
+ "step": 635
1045
+ },
1046
+ {
1047
+ "epoch": 0.36168409155128567,
1048
+ "grad_norm": 105.43537139892578,
1049
+ "learning_rate": 4.380824540651301e-06,
1050
+ "loss": 6.4694,
1051
+ "mean_token_accuracy": 0.6702811747789383,
1052
+ "step": 640
1053
+ },
1054
+ {
1055
+ "epoch": 0.3645097485165301,
1056
+ "grad_norm": 41.170921325683594,
1057
+ "learning_rate": 4.356641639695022e-06,
1058
+ "loss": 6.4786,
1059
+ "mean_token_accuracy": 0.6519061028957367,
1060
+ "step": 645
1061
+ },
1062
+ {
1063
+ "epoch": 0.3673354054817745,
1064
+ "grad_norm": 33.53129959106445,
1065
+ "learning_rate": 4.332347410909566e-06,
1066
+ "loss": 6.4479,
1067
+ "mean_token_accuracy": 0.6696308821439743,
1068
+ "step": 650
1069
+ },
1070
+ {
1071
+ "epoch": 0.3701610624470189,
1072
+ "grad_norm": 44.768699645996094,
1073
+ "learning_rate": 4.307943847911868e-06,
1074
+ "loss": 6.6492,
1075
+ "mean_token_accuracy": 0.6501506567001343,
1076
+ "step": 655
1077
+ },
1078
+ {
1079
+ "epoch": 0.37298671941226336,
1080
+ "grad_norm": 38.888816833496094,
1081
+ "learning_rate": 4.283432953290981e-06,
1082
+ "loss": 6.7759,
1083
+ "mean_token_accuracy": 0.6582424193620682,
1084
+ "step": 660
1085
+ },
1086
+ {
1087
+ "epoch": 0.37581237637750775,
1088
+ "grad_norm": 42.18849563598633,
1089
+ "learning_rate": 4.258816738443731e-06,
1090
+ "loss": 6.0352,
1091
+ "mean_token_accuracy": 0.672022745013237,
1092
+ "step": 665
1093
+ },
1094
+ {
1095
+ "epoch": 0.3786380333427522,
1096
+ "grad_norm": 32.39656066894531,
1097
+ "learning_rate": 4.234097223409664e-06,
1098
+ "loss": 6.2633,
1099
+ "mean_token_accuracy": 0.6928743287920952,
1100
+ "step": 670
1101
+ },
1102
+ {
1103
+ "epoch": 0.3814636903079966,
1104
+ "grad_norm": 38.409385681152344,
1105
+ "learning_rate": 4.209276436705276e-06,
1106
+ "loss": 6.2866,
1107
+ "mean_token_accuracy": 0.6808415204286575,
1108
+ "step": 675
1109
+ },
1110
+ {
1111
+ "epoch": 0.38428934727324104,
1112
+ "grad_norm": 41.57419204711914,
1113
+ "learning_rate": 4.184356415157556e-06,
1114
+ "loss": 5.9778,
1115
+ "mean_token_accuracy": 0.7001727074384689,
1116
+ "step": 680
1117
+ },
1118
+ {
1119
+ "epoch": 0.38711500423848544,
1120
+ "grad_norm": 34.73065185546875,
1121
+ "learning_rate": 4.159339203736831e-06,
1122
+ "loss": 6.0301,
1123
+ "mean_token_accuracy": 0.6711145430803299,
1124
+ "step": 685
1125
+ },
1126
+ {
1127
+ "epoch": 0.3899406612037299,
1128
+ "grad_norm": 40.23607635498047,
1129
+ "learning_rate": 4.134226855388963e-06,
1130
+ "loss": 5.7109,
1131
+ "mean_token_accuracy": 0.7057820171117782,
1132
+ "step": 690
1133
+ },
1134
+ {
1135
+ "epoch": 0.3927663181689743,
1136
+ "grad_norm": 38.31148910522461,
1137
+ "learning_rate": 4.10902143086688e-06,
1138
+ "loss": 6.6394,
1139
+ "mean_token_accuracy": 0.6580009430646896,
1140
+ "step": 695
1141
+ },
1142
+ {
1143
+ "epoch": 0.39559197513421873,
1144
+ "grad_norm": 50.16755676269531,
1145
+ "learning_rate": 4.08372499856146e-06,
1146
+ "loss": 5.611,
1147
+ "mean_token_accuracy": 0.7013431131839752,
1148
+ "step": 700
1149
+ },
1150
+ {
1151
+ "epoch": 0.3984176320994631,
1152
+ "grad_norm": 30.385122299194336,
1153
+ "learning_rate": 4.0583396343318025e-06,
1154
+ "loss": 6.5846,
1155
+ "mean_token_accuracy": 0.6525479450821876,
1156
+ "step": 705
1157
+ },
1158
+ {
1159
+ "epoch": 0.4012432890647075,
1160
+ "grad_norm": 40.52418518066406,
1161
+ "learning_rate": 4.032867421334884e-06,
1162
+ "loss": 5.4611,
1163
+ "mean_token_accuracy": 0.698312160372734,
1164
+ "step": 710
1165
+ },
1166
+ {
1167
+ "epoch": 0.40406894602995197,
1168
+ "grad_norm": 39.72053146362305,
1169
+ "learning_rate": 4.0073104498546036e-06,
1170
+ "loss": 5.9016,
1171
+ "mean_token_accuracy": 0.6850436985492706,
1172
+ "step": 715
1173
+ },
1174
+ {
1175
+ "epoch": 0.40689460299519636,
1176
+ "grad_norm": 38.679527282714844,
1177
+ "learning_rate": 3.981670817130255e-06,
1178
+ "loss": 6.0392,
1179
+ "mean_token_accuracy": 0.6699395298957824,
1180
+ "step": 720
1181
+ },
1182
+ {
1183
+ "epoch": 0.4097202599604408,
1184
+ "grad_norm": 42.41149139404297,
1185
+ "learning_rate": 3.955950627184426e-06,
1186
+ "loss": 6.2423,
1187
+ "mean_token_accuracy": 0.6726677268743515,
1188
+ "step": 725
1189
+ },
1190
+ {
1191
+ "epoch": 0.4125459169256852,
1192
+ "grad_norm": 36.73404312133789,
1193
+ "learning_rate": 3.930151990650336e-06,
1194
+ "loss": 5.0402,
1195
+ "mean_token_accuracy": 0.7204348385334015,
1196
+ "step": 730
1197
+ },
1198
+ {
1199
+ "epoch": 0.41537157389092966,
1200
+ "grad_norm": 33.893760681152344,
1201
+ "learning_rate": 3.904277024598638e-06,
1202
+ "loss": 5.5147,
1203
+ "mean_token_accuracy": 0.6979331076145172,
1204
+ "step": 735
1205
+ },
1206
+ {
1207
+ "epoch": 0.41819723085617405,
1208
+ "grad_norm": 47.56764602661133,
1209
+ "learning_rate": 3.878327852363686e-06,
1210
+ "loss": 5.7995,
1211
+ "mean_token_accuracy": 0.7078070282936096,
1212
+ "step": 740
1213
+ },
1214
+ {
1215
+ "epoch": 0.4210228878214185,
1216
+ "grad_norm": 31.648059844970703,
1217
+ "learning_rate": 3.852306603369294e-06,
1218
+ "loss": 6.8761,
1219
+ "mean_token_accuracy": 0.665014611184597,
1220
+ "step": 745
1221
+ },
1222
+ {
1223
+ "epoch": 0.4238485447866629,
1224
+ "grad_norm": 38.39750289916992,
1225
+ "learning_rate": 3.826215412953991e-06,
1226
+ "loss": 6.0653,
1227
+ "mean_token_accuracy": 0.6770342886447906,
1228
+ "step": 750
1229
+ },
1230
+ {
1231
+ "epoch": 0.42667420175190734,
1232
+ "grad_norm": 30.88237190246582,
1233
+ "learning_rate": 3.800056422195792e-06,
1234
+ "loss": 6.535,
1235
+ "mean_token_accuracy": 0.6590037375688553,
1236
+ "step": 755
1237
+ },
1238
+ {
1239
+ "epoch": 0.42949985871715174,
1240
+ "grad_norm": 63.2603874206543,
1241
+ "learning_rate": 3.773831777736499e-06,
1242
+ "loss": 6.5161,
1243
+ "mean_token_accuracy": 0.6474016666412353,
1244
+ "step": 760
1245
+ },
1246
+ {
1247
+ "epoch": 0.43232551568239613,
1248
+ "grad_norm": 50.654457092285156,
1249
+ "learning_rate": 3.747543631605547e-06,
1250
+ "loss": 6.7369,
1251
+ "mean_token_accuracy": 0.6452984467148781,
1252
+ "step": 765
1253
+ },
1254
+ {
1255
+ "epoch": 0.4351511726476406,
1256
+ "grad_norm": 37.10031509399414,
1257
+ "learning_rate": 3.721194141043398e-06,
1258
+ "loss": 6.2939,
1259
+ "mean_token_accuracy": 0.6634088665246963,
1260
+ "step": 770
1261
+ },
1262
+ {
1263
+ "epoch": 0.437976829612885,
1264
+ "grad_norm": 32.4448356628418,
1265
+ "learning_rate": 3.694785468324526e-06,
1266
+ "loss": 5.3857,
1267
+ "mean_token_accuracy": 0.7086734473705292,
1268
+ "step": 775
1269
+ },
1270
+ {
1271
+ "epoch": 0.4408024865781294,
1272
+ "grad_norm": 37.90336990356445,
1273
+ "learning_rate": 3.6683197805799684e-06,
1274
+ "loss": 5.5692,
1275
+ "mean_token_accuracy": 0.691333469748497,
1276
+ "step": 780
1277
+ },
1278
+ {
1279
+ "epoch": 0.4436281435433738,
1280
+ "grad_norm": 33.28772735595703,
1281
+ "learning_rate": 3.641799249619492e-06,
1282
+ "loss": 5.7555,
1283
+ "mean_token_accuracy": 0.6938249558210373,
1284
+ "step": 785
1285
+ },
1286
+ {
1287
+ "epoch": 0.44645380050861827,
1288
+ "grad_norm": 40.561336517333984,
1289
+ "learning_rate": 3.6152260517533743e-06,
1290
+ "loss": 6.5292,
1291
+ "mean_token_accuracy": 0.6575401365756989,
1292
+ "step": 790
1293
+ },
1294
+ {
1295
+ "epoch": 0.44927945747386266,
1296
+ "grad_norm": 34.72788619995117,
1297
+ "learning_rate": 3.588602367613805e-06,
1298
+ "loss": 5.6275,
1299
+ "mean_token_accuracy": 0.6937674105167388,
1300
+ "step": 795
1301
+ },
1302
+ {
1303
+ "epoch": 0.4521051144391071,
1304
+ "grad_norm": 51.351802825927734,
1305
+ "learning_rate": 3.56193038197595e-06,
1306
+ "loss": 5.8965,
1307
+ "mean_token_accuracy": 0.7013622283935547,
1308
+ "step": 800
1309
+ },
1310
+ {
1311
+ "epoch": 0.4549307714043515,
1312
+ "grad_norm": 34.20261001586914,
1313
+ "learning_rate": 3.5352122835786555e-06,
1314
+ "loss": 6.2313,
1315
+ "mean_token_accuracy": 0.6691349744796753,
1316
+ "step": 805
1317
+ },
1318
+ {
1319
+ "epoch": 0.45775642836959596,
1320
+ "grad_norm": 32.29352951049805,
1321
+ "learning_rate": 3.508450264944848e-06,
1322
+ "loss": 6.0912,
1323
+ "mean_token_accuracy": 0.6840269297361374,
1324
+ "step": 810
1325
+ },
1326
+ {
1327
+ "epoch": 0.46058208533484035,
1328
+ "grad_norm": 39.749229431152344,
1329
+ "learning_rate": 3.481646522201602e-06,
1330
+ "loss": 5.9559,
1331
+ "mean_token_accuracy": 0.6829979822039605,
1332
+ "step": 815
1333
+ },
1334
+ {
1335
+ "epoch": 0.46340774230008475,
1336
+ "grad_norm": 39.798587799072266,
1337
+ "learning_rate": 3.4548032548999336e-06,
1338
+ "loss": 6.161,
1339
+ "mean_token_accuracy": 0.6791020795702934,
1340
+ "step": 820
1341
+ },
1342
+ {
1343
+ "epoch": 0.4662333992653292,
1344
+ "grad_norm": 51.893211364746094,
1345
+ "learning_rate": 3.4279226658342925e-06,
1346
+ "loss": 6.3016,
1347
+ "mean_token_accuracy": 0.6657601609826088,
1348
+ "step": 825
1349
+ },
1350
+ {
1351
+ "epoch": 0.4690590562305736,
1352
+ "grad_norm": 50.02108383178711,
1353
+ "learning_rate": 3.4010069608618056e-06,
1354
+ "loss": 6.0286,
1355
+ "mean_token_accuracy": 0.6940437912940979,
1356
+ "step": 830
1357
+ },
1358
+ {
1359
+ "epoch": 0.47188471319581804,
1360
+ "grad_norm": 55.37822723388672,
1361
+ "learning_rate": 3.374058348721255e-06,
1362
+ "loss": 6.2305,
1363
+ "mean_token_accuracy": 0.6752733439207077,
1364
+ "step": 835
1365
+ },
1366
+ {
1367
+ "epoch": 0.47471037016106243,
1368
+ "grad_norm": 39.274688720703125,
1369
+ "learning_rate": 3.347079040851833e-06,
1370
+ "loss": 6.4344,
1371
+ "mean_token_accuracy": 0.6463314086198807,
1372
+ "step": 840
1373
+ },
1374
+ {
1375
+ "epoch": 0.4775360271263069,
1376
+ "grad_norm": 33.772884368896484,
1377
+ "learning_rate": 3.3200712512116598e-06,
1378
+ "loss": 4.5399,
1379
+ "mean_token_accuracy": 0.7319628089666367,
1380
+ "step": 845
1381
+ },
1382
+ {
1383
+ "epoch": 0.4803616840915513,
1384
+ "grad_norm": 47.047767639160156,
1385
+ "learning_rate": 3.293037196096113e-06,
1386
+ "loss": 6.0456,
1387
+ "mean_token_accuracy": 0.664868313074112,
1388
+ "step": 850
1389
+ },
1390
+ {
1391
+ "epoch": 0.4831873410567957,
1392
+ "grad_norm": 41.05727767944336,
1393
+ "learning_rate": 3.2659790939559453e-06,
1394
+ "loss": 5.7682,
1395
+ "mean_token_accuracy": 0.7045676440000535,
1396
+ "step": 855
1397
+ },
1398
+ {
1399
+ "epoch": 0.4860129980220401,
1400
+ "grad_norm": 37.37389373779297,
1401
+ "learning_rate": 3.238899165215245e-06,
1402
+ "loss": 5.2326,
1403
+ "mean_token_accuracy": 0.7269378632307053,
1404
+ "step": 860
1405
+ },
1406
+ {
1407
+ "epoch": 0.48883865498728457,
1408
+ "grad_norm": 39.5576286315918,
1409
+ "learning_rate": 3.211799632089216e-06,
1410
+ "loss": 5.1324,
1411
+ "mean_token_accuracy": 0.7152336061000824,
1412
+ "step": 865
1413
+ },
1414
+ {
1415
+ "epoch": 0.49166431195252897,
1416
+ "grad_norm": 41.0853157043457,
1417
+ "learning_rate": 3.1846827184018294e-06,
1418
+ "loss": 5.7599,
1419
+ "mean_token_accuracy": 0.6923278480768204,
1420
+ "step": 870
1421
+ },
1422
+ {
1423
+ "epoch": 0.49448996891777336,
1424
+ "grad_norm": 51.03715896606445,
1425
+ "learning_rate": 3.157550649403322e-06,
1426
+ "loss": 4.9395,
1427
+ "mean_token_accuracy": 0.7514464080333709,
1428
+ "step": 875
1429
+ },
1430
+ {
1431
+ "epoch": 0.4973156258830178,
1432
+ "grad_norm": 37.161231994628906,
1433
+ "learning_rate": 3.1304056515876024e-06,
1434
+ "loss": 7.0247,
1435
+ "mean_token_accuracy": 0.6437601447105408,
1436
+ "step": 880
1437
+ },
1438
+ {
1439
+ "epoch": 0.5001412828482622,
1440
+ "grad_norm": 47.0338134765625,
1441
+ "learning_rate": 3.1032499525095303e-06,
1442
+ "loss": 5.647,
1443
+ "mean_token_accuracy": 0.6982032418251037,
1444
+ "step": 885
1445
+ },
1446
+ {
1447
+ "epoch": 0.5029669398135066,
1448
+ "grad_norm": 32.64208221435547,
1449
+ "learning_rate": 3.076085780602128e-06,
1450
+ "loss": 5.6704,
1451
+ "mean_token_accuracy": 0.6939798533916474,
1452
+ "step": 890
1453
+ },
1454
+ {
1455
+ "epoch": 0.5057925967787511,
1456
+ "grad_norm": 32.635005950927734,
1457
+ "learning_rate": 3.048915364993708e-06,
1458
+ "loss": 5.2166,
1459
+ "mean_token_accuracy": 0.7231873899698258,
1460
+ "step": 895
1461
+ },
1462
+ {
1463
+ "epoch": 0.5086182537439955,
1464
+ "grad_norm": 29.856887817382812,
1465
+ "learning_rate": 3.0217409353249512e-06,
1466
+ "loss": 5.9994,
1467
+ "mean_token_accuracy": 0.6912487387657166,
1468
+ "step": 900
1469
+ },
1470
+ {
1471
+ "epoch": 0.5086182537439955,
1472
+ "eval_loss": 1.5281765460968018,
1473
+ "eval_mean_token_accuracy": 0.6797472610691477,
1474
+ "eval_runtime": 60.4506,
1475
+ "eval_samples_per_second": 26.021,
1476
+ "eval_steps_per_second": 3.259,
1477
+ "step": 900
1478
+ },
1479
+ {
1480
+ "epoch": 0.5114439107092399,
1481
+ "grad_norm": 46.50059509277344,
1482
+ "learning_rate": 2.994564721565935e-06,
1483
+ "loss": 5.3769,
1484
+ "mean_token_accuracy": 0.7131396651268005,
1485
+ "step": 905
1486
+ },
1487
+ {
1488
+ "epoch": 0.5142695676744843,
1489
+ "grad_norm": 36.98368453979492,
1490
+ "learning_rate": 2.9673889538331435e-06,
1491
+ "loss": 5.5554,
1492
+ "mean_token_accuracy": 0.7053171962499618,
1493
+ "step": 910
1494
+ },
1495
+ {
1496
+ "epoch": 0.5170952246397288,
1497
+ "grad_norm": 40.522071838378906,
1498
+ "learning_rate": 2.94021586220646e-06,
1499
+ "loss": 5.9756,
1500
+ "mean_token_accuracy": 0.6839839160442353,
1501
+ "step": 915
1502
+ },
1503
+ {
1504
+ "epoch": 0.5199208816049732,
1505
+ "grad_norm": 39.4980583190918,
1506
+ "learning_rate": 2.9130476765461605e-06,
1507
+ "loss": 5.4328,
1508
+ "mean_token_accuracy": 0.7064492374658584,
1509
+ "step": 920
1510
+ },
1511
+ {
1512
+ "epoch": 0.5227465385702176,
1513
+ "grad_norm": 57.04869842529297,
1514
+ "learning_rate": 2.8858866263099325e-06,
1515
+ "loss": 6.114,
1516
+ "mean_token_accuracy": 0.6875434190034866,
1517
+ "step": 925
1518
+ },
1519
+ {
1520
+ "epoch": 0.525572195535462,
1521
+ "grad_norm": 31.3112850189209,
1522
+ "learning_rate": 2.858734940369919e-06,
1523
+ "loss": 5.63,
1524
+ "mean_token_accuracy": 0.7200439631938934,
1525
+ "step": 930
1526
+ },
1527
+ {
1528
+ "epoch": 0.5283978525007064,
1529
+ "grad_norm": 47.432926177978516,
1530
+ "learning_rate": 2.831594846829821e-06,
1531
+ "loss": 5.7099,
1532
+ "mean_token_accuracy": 0.695030590891838,
1533
+ "step": 935
1534
+ },
1535
+ {
1536
+ "epoch": 0.5312235094659509,
1537
+ "grad_norm": 35.99347686767578,
1538
+ "learning_rate": 2.8044685728420472e-06,
1539
+ "loss": 4.4271,
1540
+ "mean_token_accuracy": 0.7551506340503693,
1541
+ "step": 940
1542
+ },
1543
+ {
1544
+ "epoch": 0.5340491664311953,
1545
+ "grad_norm": 38.81227111816406,
1546
+ "learning_rate": 2.777358344424957e-06,
1547
+ "loss": 4.8282,
1548
+ "mean_token_accuracy": 0.7347202837467194,
1549
+ "step": 945
1550
+ },
1551
+ {
1552
+ "epoch": 0.5368748233964397,
1553
+ "grad_norm": 39.581241607666016,
1554
+ "learning_rate": 2.7502663862801866e-06,
1555
+ "loss": 5.3346,
1556
+ "mean_token_accuracy": 0.7030239164829254,
1557
+ "step": 950
1558
+ },
1559
+ {
1560
+ "epoch": 0.539700480361684,
1561
+ "grad_norm": 45.003013610839844,
1562
+ "learning_rate": 2.7231949216100943e-06,
1563
+ "loss": 6.2008,
1564
+ "mean_token_accuracy": 0.6608472660183906,
1565
+ "step": 955
1566
+ },
1567
+ {
1568
+ "epoch": 0.5425261373269286,
1569
+ "grad_norm": 45.456016540527344,
1570
+ "learning_rate": 2.696146171935312e-06,
1571
+ "loss": 4.8497,
1572
+ "mean_token_accuracy": 0.7226766556501388,
1573
+ "step": 960
1574
+ },
1575
+ {
1576
+ "epoch": 0.545351794292173,
1577
+ "grad_norm": 44.264671325683594,
1578
+ "learning_rate": 2.6691223569124495e-06,
1579
+ "loss": 5.5343,
1580
+ "mean_token_accuracy": 0.7043332427740097,
1581
+ "step": 965
1582
+ },
1583
+ {
1584
+ "epoch": 0.5481774512574173,
1585
+ "grad_norm": 43.33969497680664,
1586
+ "learning_rate": 2.6421256941519453e-06,
1587
+ "loss": 5.0521,
1588
+ "mean_token_accuracy": 0.7183876752853393,
1589
+ "step": 970
1590
+ },
1591
+ {
1592
+ "epoch": 0.5510031082226617,
1593
+ "grad_norm": 69.75106048583984,
1594
+ "learning_rate": 2.61515839903609e-06,
1595
+ "loss": 5.2742,
1596
+ "mean_token_accuracy": 0.7265293389558792,
1597
+ "step": 975
1598
+ },
1599
+ {
1600
+ "epoch": 0.5538287651879061,
1601
+ "grad_norm": 45.96234130859375,
1602
+ "learning_rate": 2.588222684537222e-06,
1603
+ "loss": 5.7874,
1604
+ "mean_token_accuracy": 0.6878435671329498,
1605
+ "step": 980
1606
+ },
1607
+ {
1608
+ "epoch": 0.5566544221531506,
1609
+ "grad_norm": 32.10597229003906,
1610
+ "learning_rate": 2.5613207610361338e-06,
1611
+ "loss": 5.682,
1612
+ "mean_token_accuracy": 0.6860069572925568,
1613
+ "step": 985
1614
+ },
1615
+ {
1616
+ "epoch": 0.559480079118395,
1617
+ "grad_norm": 49.05516815185547,
1618
+ "learning_rate": 2.5344548361406842e-06,
1619
+ "loss": 5.7007,
1620
+ "mean_token_accuracy": 0.6855567038059235,
1621
+ "step": 990
1622
+ },
1623
+ {
1624
+ "epoch": 0.5623057360836394,
1625
+ "grad_norm": 74.12281799316406,
1626
+ "learning_rate": 2.507627114504637e-06,
1627
+ "loss": 6.9595,
1628
+ "mean_token_accuracy": 0.6414749681949615,
1629
+ "step": 995
1630
+ },
1631
+ {
1632
+ "epoch": 0.5651313930488838,
1633
+ "grad_norm": 47.54744338989258,
1634
+ "learning_rate": 2.480839797646746e-06,
1635
+ "loss": 5.8039,
1636
+ "mean_token_accuracy": 0.7000325888395309,
1637
+ "step": 1000
1638
+ },
1639
+ {
1640
+ "epoch": 0.5679570500141283,
1641
+ "grad_norm": 38.07027053833008,
1642
+ "learning_rate": 2.4540950837700923e-06,
1643
+ "loss": 6.192,
1644
+ "mean_token_accuracy": 0.702258163690567,
1645
+ "step": 1005
1646
+ },
1647
+ {
1648
+ "epoch": 0.5707827069793727,
1649
+ "grad_norm": 47.82304382324219,
1650
+ "learning_rate": 2.4273951675817043e-06,
1651
+ "loss": 5.2627,
1652
+ "mean_token_accuracy": 0.704213073849678,
1653
+ "step": 1010
1654
+ },
1655
+ {
1656
+ "epoch": 0.5736083639446171,
1657
+ "grad_norm": 33.01102066040039,
1658
+ "learning_rate": 2.4007422401124488e-06,
1659
+ "loss": 5.9233,
1660
+ "mean_token_accuracy": 0.6820877581834793,
1661
+ "step": 1015
1662
+ },
1663
+ {
1664
+ "epoch": 0.5764340209098615,
1665
+ "grad_norm": 28.866247177124023,
1666
+ "learning_rate": 2.3741384885372346e-06,
1667
+ "loss": 5.5267,
1668
+ "mean_token_accuracy": 0.705855768918991,
1669
+ "step": 1020
1670
+ },
1671
+ {
1672
+ "epoch": 0.579259677875106,
1673
+ "grad_norm": 46.4747428894043,
1674
+ "learning_rate": 2.347586095995532e-06,
1675
+ "loss": 6.0163,
1676
+ "mean_token_accuracy": 0.6707334235310555,
1677
+ "step": 1025
1678
+ },
1679
+ {
1680
+ "epoch": 0.5820853348403504,
1681
+ "grad_norm": 41.4154052734375,
1682
+ "learning_rate": 2.3210872414122224e-06,
1683
+ "loss": 5.1814,
1684
+ "mean_token_accuracy": 0.729784882068634,
1685
+ "step": 1030
1686
+ },
1687
+ {
1688
+ "epoch": 0.5849109918055948,
1689
+ "grad_norm": 36.9775505065918,
1690
+ "learning_rate": 2.2946440993187876e-06,
1691
+ "loss": 5.3629,
1692
+ "mean_token_accuracy": 0.6963009983301163,
1693
+ "step": 1035
1694
+ },
1695
+ {
1696
+ "epoch": 0.5877366487708392,
1697
+ "grad_norm": 33.48902893066406,
1698
+ "learning_rate": 2.2682588396748687e-06,
1699
+ "loss": 6.5289,
1700
+ "mean_token_accuracy": 0.6563118815422058,
1701
+ "step": 1040
1702
+ },
1703
+ {
1704
+ "epoch": 0.5905623057360836,
1705
+ "grad_norm": 41.8765983581543,
1706
+ "learning_rate": 2.241933627690196e-06,
1707
+ "loss": 6.1125,
1708
+ "mean_token_accuracy": 0.6872710019350052,
1709
+ "step": 1045
1710
+ },
1711
+ {
1712
+ "epoch": 0.5933879627013281,
1713
+ "grad_norm": 33.268428802490234,
1714
+ "learning_rate": 2.2156706236469088e-06,
1715
+ "loss": 6.1175,
1716
+ "mean_token_accuracy": 0.6717882409691811,
1717
+ "step": 1050
1718
+ },
1719
+ {
1720
+ "epoch": 0.5962136196665725,
1721
+ "grad_norm": 39.825626373291016,
1722
+ "learning_rate": 2.1894719827222783e-06,
1723
+ "loss": 5.5307,
1724
+ "mean_token_accuracy": 0.7009034663438797,
1725
+ "step": 1055
1726
+ },
1727
+ {
1728
+ "epoch": 0.5990392766318169,
1729
+ "grad_norm": 44.565643310546875,
1730
+ "learning_rate": 2.1633398548118515e-06,
1731
+ "loss": 4.8155,
1732
+ "mean_token_accuracy": 0.7297946393489838,
1733
+ "step": 1060
1734
+ },
1735
+ {
1736
+ "epoch": 0.6018649335970613,
1737
+ "grad_norm": 39.603511810302734,
1738
+ "learning_rate": 2.137276384353032e-06,
1739
+ "loss": 4.6931,
1740
+ "mean_token_accuracy": 0.7337764650583267,
1741
+ "step": 1065
1742
+ },
1743
+ {
1744
+ "epoch": 0.6046905905623058,
1745
+ "grad_norm": 42.91708755493164,
1746
+ "learning_rate": 2.111283710149097e-06,
1747
+ "loss": 5.3974,
1748
+ "mean_token_accuracy": 0.7125359356403351,
1749
+ "step": 1070
1750
+ },
1751
+ {
1752
+ "epoch": 0.6075162475275502,
1753
+ "grad_norm": 57.45164108276367,
1754
+ "learning_rate": 2.08536396519369e-06,
1755
+ "loss": 5.4134,
1756
+ "mean_token_accuracy": 0.6924123004078865,
1757
+ "step": 1075
1758
+ },
1759
+ {
1760
+ "epoch": 0.6103419044927946,
1761
+ "grad_norm": 32.598880767822266,
1762
+ "learning_rate": 2.0595192764957815e-06,
1763
+ "loss": 5.6697,
1764
+ "mean_token_accuracy": 0.6915164411067962,
1765
+ "step": 1080
1766
+ },
1767
+ {
1768
+ "epoch": 0.613167561458039,
1769
+ "grad_norm": 44.280662536621094,
1770
+ "learning_rate": 2.0337517649051282e-06,
1771
+ "loss": 5.7898,
1772
+ "mean_token_accuracy": 0.7014181435108184,
1773
+ "step": 1085
1774
+ },
1775
+ {
1776
+ "epoch": 0.6159932184232834,
1777
+ "grad_norm": 50.706207275390625,
1778
+ "learning_rate": 2.008063544938227e-06,
1779
+ "loss": 6.3557,
1780
+ "mean_token_accuracy": 0.6665431886911393,
1781
+ "step": 1090
1782
+ },
1783
+ {
1784
+ "epoch": 0.6188188753885279,
1785
+ "grad_norm": 34.798702239990234,
1786
+ "learning_rate": 1.982456724604798e-06,
1787
+ "loss": 5.8921,
1788
+ "mean_token_accuracy": 0.6692013502120971,
1789
+ "step": 1095
1790
+ },
1791
+ {
1792
+ "epoch": 0.6216445323537723,
1793
+ "grad_norm": 43.494693756103516,
1794
+ "learning_rate": 1.956933405234799e-06,
1795
+ "loss": 5.0179,
1796
+ "mean_token_accuracy": 0.7225543946027756,
1797
+ "step": 1100
1798
+ },
1799
+ {
1800
+ "epoch": 0.6244701893190167,
1801
+ "grad_norm": 30.238845825195312,
1802
+ "learning_rate": 1.9314956813059893e-06,
1803
+ "loss": 5.3437,
1804
+ "mean_token_accuracy": 0.7133805066347122,
1805
+ "step": 1105
1806
+ },
1807
+ {
1808
+ "epoch": 0.627295846284261,
1809
+ "grad_norm": 58.761962890625,
1810
+ "learning_rate": 1.906145640272049e-06,
1811
+ "loss": 5.6678,
1812
+ "mean_token_accuracy": 0.6920988261699677,
1813
+ "step": 1110
1814
+ },
1815
+ {
1816
+ "epoch": 0.6301215032495056,
1817
+ "grad_norm": 36.33903884887695,
1818
+ "learning_rate": 1.8808853623912808e-06,
1819
+ "loss": 4.9883,
1820
+ "mean_token_accuracy": 0.7231852769851684,
1821
+ "step": 1115
1822
+ },
1823
+ {
1824
+ "epoch": 0.63294716021475,
1825
+ "grad_norm": 29.945993423461914,
1826
+ "learning_rate": 1.8557169205559086e-06,
1827
+ "loss": 5.3157,
1828
+ "mean_token_accuracy": 0.706680515408516,
1829
+ "step": 1120
1830
+ },
1831
+ {
1832
+ "epoch": 0.6357728171799943,
1833
+ "grad_norm": 28.975589752197266,
1834
+ "learning_rate": 1.830642380121962e-06,
1835
+ "loss": 5.2056,
1836
+ "mean_token_accuracy": 0.7050720751285553,
1837
+ "step": 1125
1838
+ },
1839
+ {
1840
+ "epoch": 0.6385984741452387,
1841
+ "grad_norm": 44.97232437133789,
1842
+ "learning_rate": 1.8056637987397989e-06,
1843
+ "loss": 6.0927,
1844
+ "mean_token_accuracy": 0.6807661324739456,
1845
+ "step": 1130
1846
+ },
1847
+ {
1848
+ "epoch": 0.6414241311104832,
1849
+ "grad_norm": 34.31592559814453,
1850
+ "learning_rate": 1.7807832261852462e-06,
1851
+ "loss": 5.5395,
1852
+ "mean_token_accuracy": 0.6893488377332687,
1853
+ "step": 1135
1854
+ },
1855
+ {
1856
+ "epoch": 0.6442497880757276,
1857
+ "grad_norm": 39.61166763305664,
1858
+ "learning_rate": 1.7560027041913992e-06,
1859
+ "loss": 5.0804,
1860
+ "mean_token_accuracy": 0.7302613139152527,
1861
+ "step": 1140
1862
+ },
1863
+ {
1864
+ "epoch": 0.647075445040972,
1865
+ "grad_norm": 42.036598205566406,
1866
+ "learning_rate": 1.7313242662810682e-06,
1867
+ "loss": 5.247,
1868
+ "mean_token_accuracy": 0.6992710053920745,
1869
+ "step": 1145
1870
+ },
1871
+ {
1872
+ "epoch": 0.6499011020062164,
1873
+ "grad_norm": 45.033103942871094,
1874
+ "learning_rate": 1.7067499375999042e-06,
1875
+ "loss": 5.6366,
1876
+ "mean_token_accuracy": 0.6878565683960914,
1877
+ "step": 1150
1878
+ },
1879
+ {
1880
+ "epoch": 0.6527267589714608,
1881
+ "grad_norm": 45.477500915527344,
1882
+ "learning_rate": 1.6822817347502192e-06,
1883
+ "loss": 5.1352,
1884
+ "mean_token_accuracy": 0.7178653836250305,
1885
+ "step": 1155
1886
+ },
1887
+ {
1888
+ "epoch": 0.6555524159367053,
1889
+ "grad_norm": 30.850934982299805,
1890
+ "learning_rate": 1.657921665625497e-06,
1891
+ "loss": 5.3625,
1892
+ "mean_token_accuracy": 0.6953006356954574,
1893
+ "step": 1160
1894
+ },
1895
+ {
1896
+ "epoch": 0.6583780729019497,
1897
+ "grad_norm": 44.302703857421875,
1898
+ "learning_rate": 1.6336717292456232e-06,
1899
+ "loss": 4.9628,
1900
+ "mean_token_accuracy": 0.7283646464347839,
1901
+ "step": 1165
1902
+ },
1903
+ {
1904
+ "epoch": 0.6612037298671941,
1905
+ "grad_norm": 36.562110900878906,
1906
+ "learning_rate": 1.6095339155928395e-06,
1907
+ "loss": 6.379,
1908
+ "mean_token_accuracy": 0.6473036587238312,
1909
+ "step": 1170
1910
+ },
1911
+ {
1912
+ "epoch": 0.6640293868324385,
1913
+ "grad_norm": 45.95467758178711,
1914
+ "learning_rate": 1.5855102054484505e-06,
1915
+ "loss": 5.2969,
1916
+ "mean_token_accuracy": 0.7192613005638122,
1917
+ "step": 1175
1918
+ },
1919
+ {
1920
+ "epoch": 0.666855043797683,
1921
+ "grad_norm": 39.13850021362305,
1922
+ "learning_rate": 1.5616025702302725e-06,
1923
+ "loss": 5.9703,
1924
+ "mean_token_accuracy": 0.6769262015819549,
1925
+ "step": 1180
1926
+ },
1927
+ {
1928
+ "epoch": 0.6696807007629274,
1929
+ "grad_norm": 51.65994644165039,
1930
+ "learning_rate": 1.53781297183086e-06,
1931
+ "loss": 5.5809,
1932
+ "mean_token_accuracy": 0.7111178368330002,
1933
+ "step": 1185
1934
+ },
1935
+ {
1936
+ "epoch": 0.6725063577281718,
1937
+ "grad_norm": 43.34739303588867,
1938
+ "learning_rate": 1.5141433624565027e-06,
1939
+ "loss": 5.735,
1940
+ "mean_token_accuracy": 0.7082278728485107,
1941
+ "step": 1190
1942
+ },
1943
+ {
1944
+ "epoch": 0.6753320146934162,
1945
+ "grad_norm": 40.05953598022461,
1946
+ "learning_rate": 1.490595684467038e-06,
1947
+ "loss": 5.2052,
1948
+ "mean_token_accuracy": 0.7205279141664505,
1949
+ "step": 1195
1950
+ },
1951
+ {
1952
+ "epoch": 0.6781576716586606,
1953
+ "grad_norm": 33.837886810302734,
1954
+ "learning_rate": 1.4671718702164435e-06,
1955
+ "loss": 5.9794,
1956
+ "mean_token_accuracy": 0.6879066616296768,
1957
+ "step": 1200
1958
+ },
1959
+ {
1960
+ "epoch": 0.6781576716586606,
1961
+ "eval_loss": 1.4738432168960571,
1962
+ "eval_mean_token_accuracy": 0.6890995811084806,
1963
+ "eval_runtime": 60.3641,
1964
+ "eval_samples_per_second": 26.059,
1965
+ "eval_steps_per_second": 3.264,
1966
+ "step": 1200
1967
+ },
1968
+ {
1969
+ "epoch": 0.6809833286239051,
1970
+ "grad_norm": 34.58961486816406,
1971
+ "learning_rate": 1.4438738418942778e-06,
1972
+ "loss": 5.1884,
1973
+ "mean_token_accuracy": 0.7247902929782868,
1974
+ "step": 1205
1975
+ },
1976
+ {
1977
+ "epoch": 0.6838089855891495,
1978
+ "grad_norm": 33.61336135864258,
1979
+ "learning_rate": 1.4207035113679322e-06,
1980
+ "loss": 6.2893,
1981
+ "mean_token_accuracy": 0.6808875143527985,
1982
+ "step": 1210
1983
+ },
1984
+ {
1985
+ "epoch": 0.6866346425543939,
1986
+ "grad_norm": 33.33090591430664,
1987
+ "learning_rate": 1.3976627800257539e-06,
1988
+ "loss": 4.3805,
1989
+ "mean_token_accuracy": 0.7549025654792786,
1990
+ "step": 1215
1991
+ },
1992
+ {
1993
+ "epoch": 0.6894602995196383,
1994
+ "grad_norm": 59.58811950683594,
1995
+ "learning_rate": 1.374753538621e-06,
1996
+ "loss": 5.5812,
1997
+ "mean_token_accuracy": 0.7062610507011413,
1998
+ "step": 1220
1999
+ },
2000
+ {
2001
+ "epoch": 0.6922859564848828,
2002
+ "grad_norm": 30.559871673583984,
2003
+ "learning_rate": 1.3519776671166916e-06,
2004
+ "loss": 5.7867,
2005
+ "mean_token_accuracy": 0.6923086196184158,
2006
+ "step": 1225
2007
+ },
2008
+ {
2009
+ "epoch": 0.6951116134501272,
2010
+ "grad_norm": 43.92509841918945,
2011
+ "learning_rate": 1.3293370345313334e-06,
2012
+ "loss": 4.5499,
2013
+ "mean_token_accuracy": 0.7372392445802689,
2014
+ "step": 1230
2015
+ },
2016
+ {
2017
+ "epoch": 0.6979372704153716,
2018
+ "grad_norm": 59.36580276489258,
2019
+ "learning_rate": 1.3068334987855499e-06,
2020
+ "loss": 5.9539,
2021
+ "mean_token_accuracy": 0.6813401341438293,
2022
+ "step": 1235
2023
+ },
2024
+ {
2025
+ "epoch": 0.700762927380616,
2026
+ "grad_norm": 54.31859588623047,
2027
+ "learning_rate": 1.284468906549608e-06,
2028
+ "loss": 4.7439,
2029
+ "mean_token_accuracy": 0.7365440279245377,
2030
+ "step": 1240
2031
+ },
2032
+ {
2033
+ "epoch": 0.7035885843458604,
2034
+ "grad_norm": 32.31938171386719,
2035
+ "learning_rate": 1.2622450930918888e-06,
2036
+ "loss": 4.8476,
2037
+ "mean_token_accuracy": 0.7421909987926483,
2038
+ "step": 1245
2039
+ },
2040
+ {
2041
+ "epoch": 0.7064142413111049,
2042
+ "grad_norm": 44.08262634277344,
2043
+ "learning_rate": 1.2401638821282773e-06,
2044
+ "loss": 5.5941,
2045
+ "mean_token_accuracy": 0.6980589658021927,
2046
+ "step": 1250
2047
+ },
2048
+ {
2049
+ "epoch": 0.7092398982763493,
2050
+ "grad_norm": 38.87665939331055,
2051
+ "learning_rate": 1.2182270856725072e-06,
2052
+ "loss": 5.3259,
2053
+ "mean_token_accuracy": 0.7116535753011703,
2054
+ "step": 1255
2055
+ },
2056
+ {
2057
+ "epoch": 0.7120655552415937,
2058
+ "grad_norm": 40.80810546875,
2059
+ "learning_rate": 1.1964365038874623e-06,
2060
+ "loss": 4.7656,
2061
+ "mean_token_accuracy": 0.7319334447383881,
2062
+ "step": 1260
2063
+ },
2064
+ {
2065
+ "epoch": 0.714891212206838,
2066
+ "grad_norm": 41.92693328857422,
2067
+ "learning_rate": 1.1747939249374573e-06,
2068
+ "loss": 5.6642,
2069
+ "mean_token_accuracy": 0.6878768473863601,
2070
+ "step": 1265
2071
+ },
2072
+ {
2073
+ "epoch": 0.7177168691720825,
2074
+ "grad_norm": 48.853965759277344,
2075
+ "learning_rate": 1.1533011248414951e-06,
2076
+ "loss": 6.1051,
2077
+ "mean_token_accuracy": 0.6856672108173371,
2078
+ "step": 1270
2079
+ },
2080
+ {
2081
+ "epoch": 0.7205425261373269,
2082
+ "grad_norm": 39.21954345703125,
2083
+ "learning_rate": 1.1319598673275266e-06,
2084
+ "loss": 4.7105,
2085
+ "mean_token_accuracy": 0.713962522149086,
2086
+ "step": 1275
2087
+ },
2088
+ {
2089
+ "epoch": 0.7233681831025713,
2090
+ "grad_norm": 37.65949630737305,
2091
+ "learning_rate": 1.1107719036877118e-06,
2092
+ "loss": 6.5123,
2093
+ "mean_token_accuracy": 0.6622111082077027,
2094
+ "step": 1280
2095
+ },
2096
+ {
2097
+ "epoch": 0.7261938400678157,
2098
+ "grad_norm": 35.390350341796875,
2099
+ "learning_rate": 1.0897389726347142e-06,
2100
+ "loss": 5.0129,
2101
+ "mean_token_accuracy": 0.7237377732992172,
2102
+ "step": 1285
2103
+ },
2104
+ {
2105
+ "epoch": 0.7290194970330602,
2106
+ "grad_norm": 50.039676666259766,
2107
+ "learning_rate": 1.0688628001590143e-06,
2108
+ "loss": 5.7967,
2109
+ "mean_token_accuracy": 0.6911179929971695,
2110
+ "step": 1290
2111
+ },
2112
+ {
2113
+ "epoch": 0.7318451539983046,
2114
+ "grad_norm": 53.71048355102539,
2115
+ "learning_rate": 1.0481450993872745e-06,
2116
+ "loss": 5.3526,
2117
+ "mean_token_accuracy": 0.7101652979850769,
2118
+ "step": 1295
2119
+ },
2120
+ {
2121
+ "epoch": 0.734670810963549,
2122
+ "grad_norm": 39.231807708740234,
2123
+ "learning_rate": 1.027587570441754e-06,
2124
+ "loss": 5.4725,
2125
+ "mean_token_accuracy": 0.7123759001493454,
2126
+ "step": 1300
2127
+ },
2128
+ {
2129
+ "epoch": 0.7374964679287934,
2130
+ "grad_norm": 55.94086837768555,
2131
+ "learning_rate": 1.0071919003008014e-06,
2132
+ "loss": 5.2319,
2133
+ "mean_token_accuracy": 0.710223987698555,
2134
+ "step": 1305
2135
+ },
2136
+ {
2137
+ "epoch": 0.7403221248940378,
2138
+ "grad_norm": 40.64888381958008,
2139
+ "learning_rate": 9.869597626604127e-07,
2140
+ "loss": 4.9426,
2141
+ "mean_token_accuracy": 0.7288057208061218,
2142
+ "step": 1310
2143
+ },
2144
+ {
2145
+ "epoch": 0.7431477818592823,
2146
+ "grad_norm": 38.66706085205078,
2147
+ "learning_rate": 9.668928177968906e-07,
2148
+ "loss": 4.9169,
2149
+ "mean_token_accuracy": 0.7256118148565293,
2150
+ "step": 1315
2151
+ },
2152
+ {
2153
+ "epoch": 0.7459734388245267,
2154
+ "grad_norm": 40.90673065185547,
2155
+ "learning_rate": 9.469927124305921e-07,
2156
+ "loss": 4.8499,
2157
+ "mean_token_accuracy": 0.7230159670114518,
2158
+ "step": 1320
2159
+ },
2160
+ {
2161
+ "epoch": 0.7487990957897711,
2162
+ "grad_norm": 36.83356857299805,
2163
+ "learning_rate": 9.272610795908089e-07,
2164
+ "loss": 4.8283,
2165
+ "mean_token_accuracy": 0.7131752580404281,
2166
+ "step": 1325
2167
+ },
2168
+ {
2169
+ "epoch": 0.7516247527550155,
2170
+ "grad_norm": 36.77878189086914,
2171
+ "learning_rate": 9.076995384817457e-07,
2172
+ "loss": 5.5567,
2173
+ "mean_token_accuracy": 0.7089558184146881,
2174
+ "step": 1330
2175
+ },
2176
+ {
2177
+ "epoch": 0.75445040972026,
2178
+ "grad_norm": 28.54437828063965,
2179
+ "learning_rate": 8.883096943496551e-07,
2180
+ "loss": 5.7075,
2181
+ "mean_token_accuracy": 0.6992707073688507,
2182
+ "step": 1335
2183
+ },
2184
+ {
2185
+ "epoch": 0.7572760666855044,
2186
+ "grad_norm": 39.31511688232422,
2187
+ "learning_rate": 8.690931383511032e-07,
2188
+ "loss": 5.3457,
2189
+ "mean_token_accuracy": 0.705685855448246,
2190
+ "step": 1340
2191
+ },
2192
+ {
2193
+ "epoch": 0.7601017236507488,
2194
+ "grad_norm": 40.097965240478516,
2195
+ "learning_rate": 8.500514474224046e-07,
2196
+ "loss": 6.0836,
2197
+ "mean_token_accuracy": 0.6828179478645324,
2198
+ "step": 1345
2199
+ },
2200
+ {
2201
+ "epoch": 0.7629273806159932,
2202
+ "grad_norm": 67.73812866210938,
2203
+ "learning_rate": 8.311861841502068e-07,
2204
+ "loss": 5.6864,
2205
+ "mean_token_accuracy": 0.7019049197435379,
2206
+ "step": 1350
2207
+ },
2208
+ {
2209
+ "epoch": 0.7657530375812376,
2210
+ "grad_norm": 51.00642776489258,
2211
+ "learning_rate": 8.124988966432725e-07,
2212
+ "loss": 6.4324,
2213
+ "mean_token_accuracy": 0.6755870655179024,
2214
+ "step": 1355
2215
+ },
2216
+ {
2217
+ "epoch": 0.7685786945464821,
2218
+ "grad_norm": 35.96385955810547,
2219
+ "learning_rate": 7.939911184054287e-07,
2220
+ "loss": 5.8962,
2221
+ "mean_token_accuracy": 0.6799912601709366,
2222
+ "step": 1360
2223
+ },
2224
+ {
2225
+ "epoch": 0.7714043515117265,
2226
+ "grad_norm": 32.220497131347656,
2227
+ "learning_rate": 7.756643682097395e-07,
2228
+ "loss": 5.7617,
2229
+ "mean_token_accuracy": 0.6959689676761627,
2230
+ "step": 1365
2231
+ },
2232
+ {
2233
+ "epoch": 0.7742300084769709,
2234
+ "grad_norm": 32.340248107910156,
2235
+ "learning_rate": 7.575201499738583e-07,
2236
+ "loss": 5.9751,
2237
+ "mean_token_accuracy": 0.6865959882736206,
2238
+ "step": 1370
2239
+ },
2240
+ {
2241
+ "epoch": 0.7770556654422153,
2242
+ "grad_norm": 31.520416259765625,
2243
+ "learning_rate": 7.395599526366254e-07,
2244
+ "loss": 5.1095,
2245
+ "mean_token_accuracy": 0.7318976312875748,
2246
+ "step": 1375
2247
+ },
2248
+ {
2249
+ "epoch": 0.7798813224074598,
2250
+ "grad_norm": 35.09196090698242,
2251
+ "learning_rate": 7.217852500358784e-07,
2252
+ "loss": 5.2077,
2253
+ "mean_token_accuracy": 0.7353829309344292,
2254
+ "step": 1380
2255
+ },
2256
+ {
2257
+ "epoch": 0.7827069793727042,
2258
+ "grad_norm": 35.470611572265625,
2259
+ "learning_rate": 7.041975007875101e-07,
2260
+ "loss": 5.9019,
2261
+ "mean_token_accuracy": 0.6903541892766952,
2262
+ "step": 1385
2263
+ },
2264
+ {
2265
+ "epoch": 0.7855326363379486,
2266
+ "grad_norm": 33.57234191894531,
2267
+ "learning_rate": 6.867981481657677e-07,
2268
+ "loss": 5.1315,
2269
+ "mean_token_accuracy": 0.7129396319389343,
2270
+ "step": 1390
2271
+ },
2272
+ {
2273
+ "epoch": 0.788358293303193,
2274
+ "grad_norm": 48.42726135253906,
2275
+ "learning_rate": 6.69588619984822e-07,
2276
+ "loss": 5.756,
2277
+ "mean_token_accuracy": 0.696381214261055,
2278
+ "step": 1395
2279
+ },
2280
+ {
2281
+ "epoch": 0.7911839502684375,
2282
+ "grad_norm": 37.467071533203125,
2283
+ "learning_rate": 6.525703284815951e-07,
2284
+ "loss": 4.3335,
2285
+ "mean_token_accuracy": 0.7404863983392715,
2286
+ "step": 1400
2287
+ },
2288
+ {
2289
+ "epoch": 0.7940096072336819,
2290
+ "grad_norm": 32.14668655395508,
2291
+ "learning_rate": 6.357446701998724e-07,
2292
+ "loss": 5.9586,
2293
+ "mean_token_accuracy": 0.7138706892728806,
2294
+ "step": 1405
2295
+ },
2296
+ {
2297
+ "epoch": 0.7968352641989263,
2298
+ "grad_norm": 38.9743766784668,
2299
+ "learning_rate": 6.191130258756955e-07,
2300
+ "loss": 5.779,
2301
+ "mean_token_accuracy": 0.6866494208574295,
2302
+ "step": 1410
2303
+ },
2304
+ {
2305
+ "epoch": 0.7996609211641706,
2306
+ "grad_norm": 32.78689193725586,
2307
+ "learning_rate": 6.026767603240652e-07,
2308
+ "loss": 4.4426,
2309
+ "mean_token_accuracy": 0.756053912639618,
2310
+ "step": 1415
2311
+ },
2312
+ {
2313
+ "epoch": 0.802486578129415,
2314
+ "grad_norm": 30.37912368774414,
2315
+ "learning_rate": 5.864372223269371e-07,
2316
+ "loss": 5.5608,
2317
+ "mean_token_accuracy": 0.7019012093544006,
2318
+ "step": 1420
2319
+ },
2320
+ {
2321
+ "epoch": 0.8053122350946595,
2322
+ "grad_norm": 30.248153686523438,
2323
+ "learning_rate": 5.7039574452254e-07,
2324
+ "loss": 5.377,
2325
+ "mean_token_accuracy": 0.70918008685112,
2326
+ "step": 1425
2327
+ },
2328
+ {
2329
+ "epoch": 0.8081378920599039,
2330
+ "grad_norm": 32.646583557128906,
2331
+ "learning_rate": 5.545536432960177e-07,
2332
+ "loss": 5.4907,
2333
+ "mean_token_accuracy": 0.7188392832875252,
2334
+ "step": 1430
2335
+ },
2336
+ {
2337
+ "epoch": 0.8109635490251483,
2338
+ "grad_norm": 36.80207443237305,
2339
+ "learning_rate": 5.38912218671406e-07,
2340
+ "loss": 5.5116,
2341
+ "mean_token_accuracy": 0.7240961819887162,
2342
+ "step": 1435
2343
+ },
2344
+ {
2345
+ "epoch": 0.8137892059903927,
2346
+ "grad_norm": 41.93488693237305,
2347
+ "learning_rate": 5.234727542049499e-07,
2348
+ "loss": 5.4272,
2349
+ "mean_token_accuracy": 0.6992593050003052,
2350
+ "step": 1440
2351
+ },
2352
+ {
2353
+ "epoch": 0.8166148629556372,
2354
+ "grad_norm": 52.579830169677734,
2355
+ "learning_rate": 5.082365168797743e-07,
2356
+ "loss": 6.1273,
2357
+ "mean_token_accuracy": 0.6790527045726776,
2358
+ "step": 1445
2359
+ },
2360
+ {
2361
+ "epoch": 0.8194405199208816,
2362
+ "grad_norm": 40.94260025024414,
2363
+ "learning_rate": 4.932047570019094e-07,
2364
+ "loss": 5.834,
2365
+ "mean_token_accuracy": 0.6879254311323166,
2366
+ "step": 1450
2367
+ },
2368
+ {
2369
+ "epoch": 0.822266176886126,
2370
+ "grad_norm": 33.04994201660156,
2371
+ "learning_rate": 4.783787080976977e-07,
2372
+ "loss": 5.563,
2373
+ "mean_token_accuracy": 0.6916647553443909,
2374
+ "step": 1455
2375
+ },
2376
+ {
2377
+ "epoch": 0.8250918338513704,
2378
+ "grad_norm": 41.77531814575195,
2379
+ "learning_rate": 4.637595868125587e-07,
2380
+ "loss": 5.9548,
2381
+ "mean_token_accuracy": 0.7029547303915024,
2382
+ "step": 1460
2383
+ },
2384
+ {
2385
+ "epoch": 0.8279174908166148,
2386
+ "grad_norm": 40.28774642944336,
2387
+ "learning_rate": 4.4934859281115804e-07,
2388
+ "loss": 6.2817,
2389
+ "mean_token_accuracy": 0.6684960305690766,
2390
+ "step": 1465
2391
+ },
2392
+ {
2393
+ "epoch": 0.8307431477818593,
2394
+ "grad_norm": 40.56486129760742,
2395
+ "learning_rate": 4.351469086789546e-07,
2396
+ "loss": 5.1229,
2397
+ "mean_token_accuracy": 0.7215188801288605,
2398
+ "step": 1470
2399
+ },
2400
+ {
2401
+ "epoch": 0.8335688047471037,
2402
+ "grad_norm": 37.47682189941406,
2403
+ "learning_rate": 4.2115569982516347e-07,
2404
+ "loss": 5.2746,
2405
+ "mean_token_accuracy": 0.737200066447258,
2406
+ "step": 1475
2407
+ },
2408
+ {
2409
+ "epoch": 0.8363944617123481,
2410
+ "grad_norm": 35.24783706665039,
2411
+ "learning_rate": 4.073761143871122e-07,
2412
+ "loss": 6.237,
2413
+ "mean_token_accuracy": 0.6709472090005875,
2414
+ "step": 1480
2415
+ },
2416
+ {
2417
+ "epoch": 0.8392201186775925,
2418
+ "grad_norm": 37.6098518371582,
2419
+ "learning_rate": 3.9380928313603084e-07,
2420
+ "loss": 4.9182,
2421
+ "mean_token_accuracy": 0.7347900182008743,
2422
+ "step": 1485
2423
+ },
2424
+ {
2425
+ "epoch": 0.842045775642837,
2426
+ "grad_norm": 32.11830139160156,
2427
+ "learning_rate": 3.8045631938425263e-07,
2428
+ "loss": 4.9045,
2429
+ "mean_token_accuracy": 0.7163156539201736,
2430
+ "step": 1490
2431
+ },
2432
+ {
2433
+ "epoch": 0.8448714326080814,
2434
+ "grad_norm": 52.64603805541992,
2435
+ "learning_rate": 3.6731831889386203e-07,
2436
+ "loss": 6.053,
2437
+ "mean_token_accuracy": 0.6679681688547134,
2438
+ "step": 1495
2439
+ },
2440
+ {
2441
+ "epoch": 0.8476970895733258,
2442
+ "grad_norm": 40.70310974121094,
2443
+ "learning_rate": 3.543963597867661e-07,
2444
+ "loss": 5.8567,
2445
+ "mean_token_accuracy": 0.6777797520160675,
2446
+ "step": 1500
2447
+ },
2448
+ {
2449
+ "epoch": 0.8476970895733258,
2450
+ "eval_loss": 1.456650972366333,
2451
+ "eval_mean_token_accuracy": 0.6919084944700832,
2452
+ "eval_runtime": 60.3571,
2453
+ "eval_samples_per_second": 26.062,
2454
+ "eval_steps_per_second": 3.264,
2455
+ "step": 1500
2456
+ },
2457
+ {
2458
+ "epoch": 0.8505227465385702,
2459
+ "grad_norm": 32.694461822509766,
2460
+ "learning_rate": 3.416915024562297e-07,
2461
+ "loss": 5.4508,
2462
+ "mean_token_accuracy": 0.7071023032069206,
2463
+ "step": 1505
2464
+ },
2465
+ {
2466
+ "epoch": 0.8533484035038147,
2467
+ "grad_norm": 51.687705993652344,
2468
+ "learning_rate": 3.2920478947985376e-07,
2469
+ "loss": 6.0779,
2470
+ "mean_token_accuracy": 0.6801360875368119,
2471
+ "step": 1510
2472
+ },
2473
+ {
2474
+ "epoch": 0.8561740604690591,
2475
+ "grad_norm": 37.62479019165039,
2476
+ "learning_rate": 3.169372455340224e-07,
2477
+ "loss": 5.7466,
2478
+ "mean_token_accuracy": 0.6816089674830437,
2479
+ "step": 1515
2480
+ },
2481
+ {
2482
+ "epoch": 0.8589997174343035,
2483
+ "grad_norm": 41.220970153808594,
2484
+ "learning_rate": 3.048898773098143e-07,
2485
+ "loss": 5.2133,
2486
+ "mean_token_accuracy": 0.7165019750595093,
2487
+ "step": 1520
2488
+ },
2489
+ {
2490
+ "epoch": 0.8618253743995479,
2491
+ "grad_norm": 46.828941345214844,
2492
+ "learning_rate": 2.9306367343039495e-07,
2493
+ "loss": 5.9703,
2494
+ "mean_token_accuracy": 0.6783023476600647,
2495
+ "step": 1525
2496
+ },
2497
+ {
2498
+ "epoch": 0.8646510313647923,
2499
+ "grad_norm": 36.49775314331055,
2500
+ "learning_rate": 2.814596043698874e-07,
2501
+ "loss": 6.7952,
2502
+ "mean_token_accuracy": 0.635238167643547,
2503
+ "step": 1530
2504
+ },
2505
+ {
2506
+ "epoch": 0.8674766883300368,
2507
+ "grad_norm": 34.13252258300781,
2508
+ "learning_rate": 2.7007862237373436e-07,
2509
+ "loss": 4.8581,
2510
+ "mean_token_accuracy": 0.7302194058895111,
2511
+ "step": 1535
2512
+ },
2513
+ {
2514
+ "epoch": 0.8703023452952812,
2515
+ "grad_norm": 49.1467170715332,
2516
+ "learning_rate": 2.5892166138055373e-07,
2517
+ "loss": 5.1559,
2518
+ "mean_token_accuracy": 0.722932118177414,
2519
+ "step": 1540
2520
+ },
2521
+ {
2522
+ "epoch": 0.8731280022605256,
2523
+ "grad_norm": 39.007266998291016,
2524
+ "learning_rate": 2.479896369455018e-07,
2525
+ "loss": 5.1747,
2526
+ "mean_token_accuracy": 0.7344535678625107,
2527
+ "step": 1545
2528
+ },
2529
+ {
2530
+ "epoch": 0.87595365922577,
2531
+ "grad_norm": 56.238033294677734,
2532
+ "learning_rate": 2.3728344616513897e-07,
2533
+ "loss": 5.6727,
2534
+ "mean_token_accuracy": 0.682933346927166,
2535
+ "step": 1550
2536
+ },
2537
+ {
2538
+ "epoch": 0.8787793161910145,
2539
+ "grad_norm": 43.23501968383789,
2540
+ "learning_rate": 2.268039676038145e-07,
2541
+ "loss": 5.1879,
2542
+ "mean_token_accuracy": 0.7229666203260422,
2543
+ "step": 1555
2544
+ },
2545
+ {
2546
+ "epoch": 0.8816049731562589,
2547
+ "grad_norm": 39.608184814453125,
2548
+ "learning_rate": 2.1655206122156722e-07,
2549
+ "loss": 5.8491,
2550
+ "mean_token_accuracy": 0.6864037364721298,
2551
+ "step": 1560
2552
+ },
2553
+ {
2554
+ "epoch": 0.8844306301215032,
2555
+ "grad_norm": 54.192378997802734,
2556
+ "learning_rate": 2.0652856830356105e-07,
2557
+ "loss": 5.9526,
2558
+ "mean_token_accuracy": 0.6815391644835472,
2559
+ "step": 1565
2560
+ },
2561
+ {
2562
+ "epoch": 0.8872562870867476,
2563
+ "grad_norm": 32.23185729980469,
2564
+ "learning_rate": 1.9673431139104336e-07,
2565
+ "loss": 4.7181,
2566
+ "mean_token_accuracy": 0.7502319306135178,
2567
+ "step": 1570
2568
+ },
2569
+ {
2570
+ "epoch": 0.890081944051992,
2571
+ "grad_norm": 50.909183502197266,
2572
+ "learning_rate": 1.8717009421384924e-07,
2573
+ "loss": 5.5584,
2574
+ "mean_token_accuracy": 0.7034726709127426,
2575
+ "step": 1575
2576
+ },
2577
+ {
2578
+ "epoch": 0.8929076010172365,
2579
+ "grad_norm": 43.879451751708984,
2580
+ "learning_rate": 1.778367016244433e-07,
2581
+ "loss": 5.9297,
2582
+ "mean_token_accuracy": 0.6852664202451706,
2583
+ "step": 1580
2584
+ },
2585
+ {
2586
+ "epoch": 0.8957332579824809,
2587
+ "grad_norm": 42.80601501464844,
2588
+ "learning_rate": 1.687348995335175e-07,
2589
+ "loss": 5.7088,
2590
+ "mean_token_accuracy": 0.7013660132884979,
2591
+ "step": 1585
2592
+ },
2593
+ {
2594
+ "epoch": 0.8985589149477253,
2595
+ "grad_norm": 44.30033493041992,
2596
+ "learning_rate": 1.5986543484713644e-07,
2597
+ "loss": 4.74,
2598
+ "mean_token_accuracy": 0.7463016867637634,
2599
+ "step": 1590
2600
+ },
2601
+ {
2602
+ "epoch": 0.9013845719129697,
2603
+ "grad_norm": 26.85874366760254,
2604
+ "learning_rate": 1.5122903540544696e-07,
2605
+ "loss": 5.8042,
2606
+ "mean_token_accuracy": 0.7009043395519257,
2607
+ "step": 1595
2608
+ },
2609
+ {
2610
+ "epoch": 0.9042102288782142,
2611
+ "grad_norm": 48.70513153076172,
2612
+ "learning_rate": 1.4282640992294816e-07,
2613
+ "loss": 5.4964,
2614
+ "mean_token_accuracy": 0.6988536447286606,
2615
+ "step": 1600
2616
+ },
2617
+ {
2618
+ "epoch": 0.9070358858434586,
2619
+ "grad_norm": 38.958900451660156,
2620
+ "learning_rate": 1.3465824793033722e-07,
2621
+ "loss": 4.6997,
2622
+ "mean_token_accuracy": 0.7541861921548844,
2623
+ "step": 1605
2624
+ },
2625
+ {
2626
+ "epoch": 0.909861542808703,
2627
+ "grad_norm": 33.20226287841797,
2628
+ "learning_rate": 1.2672521971792272e-07,
2629
+ "loss": 5.1598,
2630
+ "mean_token_accuracy": 0.7369467288255691,
2631
+ "step": 1610
2632
+ },
2633
+ {
2634
+ "epoch": 0.9126871997739474,
2635
+ "grad_norm": 41.06365203857422,
2636
+ "learning_rate": 1.1902797628061902e-07,
2637
+ "loss": 4.7037,
2638
+ "mean_token_accuracy": 0.7468703001737594,
2639
+ "step": 1615
2640
+ },
2641
+ {
2642
+ "epoch": 0.9155128567391919,
2643
+ "grad_norm": 44.9619026184082,
2644
+ "learning_rate": 1.1156714926452849e-07,
2645
+ "loss": 6.052,
2646
+ "mean_token_accuracy": 0.7028705894947052,
2647
+ "step": 1620
2648
+ },
2649
+ {
2650
+ "epoch": 0.9183385137044363,
2651
+ "grad_norm": 35.89250564575195,
2652
+ "learning_rate": 1.0434335091510383e-07,
2653
+ "loss": 5.2497,
2654
+ "mean_token_accuracy": 0.7146025896072388,
2655
+ "step": 1625
2656
+ },
2657
+ {
2658
+ "epoch": 0.9211641706696807,
2659
+ "grad_norm": 31.00511360168457,
2660
+ "learning_rate": 9.735717402690914e-08,
2661
+ "loss": 6.0851,
2662
+ "mean_token_accuracy": 0.6714588910341263,
2663
+ "step": 1630
2664
+ },
2665
+ {
2666
+ "epoch": 0.9239898276349251,
2667
+ "grad_norm": 44.473785400390625,
2668
+ "learning_rate": 9.060919189497185e-08,
2669
+ "loss": 4.9915,
2670
+ "mean_token_accuracy": 0.7288325905799866,
2671
+ "step": 1635
2672
+ },
2673
+ {
2674
+ "epoch": 0.9268154846001695,
2675
+ "grad_norm": 31.428377151489258,
2676
+ "learning_rate": 8.409995826774009e-08,
2677
+ "loss": 5.694,
2678
+ "mean_token_accuracy": 0.6871195703744888,
2679
+ "step": 1640
2680
+ },
2681
+ {
2682
+ "epoch": 0.929641141565414,
2683
+ "grad_norm": 37.793704986572266,
2684
+ "learning_rate": 7.783000730163992e-08,
2685
+ "loss": 5.9048,
2686
+ "mean_token_accuracy": 0.6889952480792999,
2687
+ "step": 1645
2688
+ },
2689
+ {
2690
+ "epoch": 0.9324667985306584,
2691
+ "grad_norm": 38.92344284057617,
2692
+ "learning_rate": 7.179985351724184e-08,
2693
+ "loss": 5.224,
2694
+ "mean_token_accuracy": 0.7157989650964737,
2695
+ "step": 1650
2696
+ },
2697
+ {
2698
+ "epoch": 0.9352924554959028,
2699
+ "grad_norm": 35.70161819458008,
2700
+ "learning_rate": 6.600999175703715e-08,
2701
+ "loss": 5.7048,
2702
+ "mean_token_accuracy": 0.6825587332248688,
2703
+ "step": 1655
2704
+ },
2705
+ {
2706
+ "epoch": 0.9381181124611472,
2707
+ "grad_norm": 28.339340209960938,
2708
+ "learning_rate": 6.046089714483449e-08,
2709
+ "loss": 5.5082,
2710
+ "mean_token_accuracy": 0.7190721690654754,
2711
+ "step": 1660
2712
+ },
2713
+ {
2714
+ "epoch": 0.9409437694263917,
2715
+ "grad_norm": 38.164451599121094,
2716
+ "learning_rate": 5.515302504676567e-08,
2717
+ "loss": 5.684,
2718
+ "mean_token_accuracy": 0.7025900691747665,
2719
+ "step": 1665
2720
+ },
2721
+ {
2722
+ "epoch": 0.9437694263916361,
2723
+ "grad_norm": 52.140602111816406,
2724
+ "learning_rate": 5.0086811033921876e-08,
2725
+ "loss": 4.3807,
2726
+ "mean_token_accuracy": 0.7526253074407577,
2727
+ "step": 1670
2728
+ },
2729
+ {
2730
+ "epoch": 0.9465950833568805,
2731
+ "grad_norm": 40.35009765625,
2732
+ "learning_rate": 4.5262670846606625e-08,
2733
+ "loss": 5.2456,
2734
+ "mean_token_accuracy": 0.7174136430025101,
2735
+ "step": 1675
2736
+ },
2737
+ {
2738
+ "epoch": 0.9494207403221249,
2739
+ "grad_norm": 28.01421356201172,
2740
+ "learning_rate": 4.068100036022304e-08,
2741
+ "loss": 5.2989,
2742
+ "mean_token_accuracy": 0.7095394700765609,
2743
+ "step": 1680
2744
+ },
2745
+ {
2746
+ "epoch": 0.9522463972873693,
2747
+ "grad_norm": 37.986968994140625,
2748
+ "learning_rate": 3.6342175552784516e-08,
2749
+ "loss": 5.4715,
2750
+ "mean_token_accuracy": 0.712908712029457,
2751
+ "step": 1685
2752
+ },
2753
+ {
2754
+ "epoch": 0.9550720542526138,
2755
+ "grad_norm": 37.408931732177734,
2756
+ "learning_rate": 3.2246552474063383e-08,
2757
+ "loss": 6.6352,
2758
+ "mean_token_accuracy": 0.6513298183679581,
2759
+ "step": 1690
2760
+ },
2761
+ {
2762
+ "epoch": 0.9578977112178582,
2763
+ "grad_norm": 43.11896514892578,
2764
+ "learning_rate": 2.839446721637229e-08,
2765
+ "loss": 6.3529,
2766
+ "mean_token_accuracy": 0.6613780409097672,
2767
+ "step": 1695
2768
+ },
2769
+ {
2770
+ "epoch": 0.9607233681831026,
2771
+ "grad_norm": 40.20124053955078,
2772
+ "learning_rate": 2.4786235886985255e-08,
2773
+ "loss": 5.8995,
2774
+ "mean_token_accuracy": 0.6976839542388916,
2775
+ "step": 1700
2776
+ },
2777
+ {
2778
+ "epoch": 0.963549025148347,
2779
+ "grad_norm": 40.93437957763672,
2780
+ "learning_rate": 2.1422154582195097e-08,
2781
+ "loss": 5.3056,
2782
+ "mean_token_accuracy": 0.7231557071208954,
2783
+ "step": 1705
2784
+ },
2785
+ {
2786
+ "epoch": 0.9663746821135915,
2787
+ "grad_norm": 35.955535888671875,
2788
+ "learning_rate": 1.8302499363017868e-08,
2789
+ "loss": 5.0696,
2790
+ "mean_token_accuracy": 0.7379933744668961,
2791
+ "step": 1710
2792
+ },
2793
+ {
2794
+ "epoch": 0.9692003390788358,
2795
+ "grad_norm": 42.10887908935547,
2796
+ "learning_rate": 1.542752623253696e-08,
2797
+ "loss": 5.867,
2798
+ "mean_token_accuracy": 0.7048305153846741,
2799
+ "step": 1715
2800
+ },
2801
+ {
2802
+ "epoch": 0.9720259960440802,
2803
+ "grad_norm": 39.272769927978516,
2804
+ "learning_rate": 1.2797471114896598e-08,
2805
+ "loss": 5.4597,
2806
+ "mean_token_accuracy": 0.7091452926397324,
2807
+ "step": 1720
2808
+ },
2809
+ {
2810
+ "epoch": 0.9748516530093246,
2811
+ "grad_norm": 34.45295715332031,
2812
+ "learning_rate": 1.0412549835940643e-08,
2813
+ "loss": 5.5044,
2814
+ "mean_token_accuracy": 0.7051479339599609,
2815
+ "step": 1725
2816
+ },
2817
+ {
2818
+ "epoch": 0.9776773099745691,
2819
+ "grad_norm": 41.140411376953125,
2820
+ "learning_rate": 8.272958105502105e-09,
2821
+ "loss": 5.5974,
2822
+ "mean_token_accuracy": 0.6941758543252945,
2823
+ "step": 1730
2824
+ },
2825
+ {
2826
+ "epoch": 0.9805029669398135,
2827
+ "grad_norm": 37.316165924072266,
2828
+ "learning_rate": 6.378871501342642e-09,
2829
+ "loss": 6.0725,
2830
+ "mean_token_accuracy": 0.6813987091183662,
2831
+ "step": 1735
2832
+ },
2833
+ {
2834
+ "epoch": 0.9833286239050579,
2835
+ "grad_norm": 35.23774337768555,
2836
+ "learning_rate": 4.730445454745436e-09,
2837
+ "loss": 6.3342,
2838
+ "mean_token_accuracy": 0.6705699622631073,
2839
+ "step": 1740
2840
+ },
2841
+ {
2842
+ "epoch": 0.9861542808703023,
2843
+ "grad_norm": 55.81275939941406,
2844
+ "learning_rate": 3.327815237757714e-09,
2845
+ "loss": 5.9213,
2846
+ "mean_token_accuracy": 0.6835766971111298,
2847
+ "step": 1745
2848
+ },
2849
+ {
2850
+ "epoch": 0.9889799378355467,
2851
+ "grad_norm": 39.690311431884766,
2852
+ "learning_rate": 2.171095952093638e-09,
2853
+ "loss": 4.39,
2854
+ "mean_token_accuracy": 0.7710116803646088,
2855
+ "step": 1750
2856
+ },
2857
+ {
2858
+ "epoch": 0.9918055948007912,
2859
+ "grad_norm": 38.35824203491211,
2860
+ "learning_rate": 1.2603825196861873e-09,
2861
+ "loss": 5.5187,
2862
+ "mean_token_accuracy": 0.6992709279060364,
2863
+ "step": 1755
2864
+ },
2865
+ {
2866
+ "epoch": 0.9946312517660356,
2867
+ "grad_norm": 46.17632293701172,
2868
+ "learning_rate": 5.957496748987268e-10,
2869
+ "loss": 6.3355,
2870
+ "mean_token_accuracy": 0.6866354376077652,
2871
+ "step": 1760
2872
+ },
2873
+ {
2874
+ "epoch": 0.99745690873128,
2875
+ "grad_norm": 39.88469314575195,
2876
+ "learning_rate": 1.7725195839224384e-10,
2877
+ "loss": 5.7136,
2878
+ "mean_token_accuracy": 0.6923755198717118,
2879
+ "step": 1765
2880
+ },
2881
+ {
2882
+ "epoch": 1.0,
2883
+ "grad_norm": 72.74512481689453,
2884
+ "learning_rate": 4.92371264992908e-12,
2885
+ "loss": 5.6788,
2886
+ "mean_token_accuracy": 0.6680825932158364,
2887
+ "step": 1770
2888
+ }
2889
+ ],
2890
+ "logging_steps": 5,
2891
+ "max_steps": 1770,
2892
+ "num_input_tokens_seen": 0,
2893
+ "num_train_epochs": 1,
2894
+ "save_steps": 300,
2895
+ "stateful_callbacks": {
2896
+ "TrainerControl": {
2897
+ "args": {
2898
+ "should_epoch_stop": false,
2899
+ "should_evaluate": false,
2900
+ "should_log": false,
2901
+ "should_save": true,
2902
+ "should_training_stop": true
2903
+ },
2904
+ "attributes": {}
2905
+ }
2906
+ },
2907
+ "total_flos": 1.107854306877479e+17,
2908
+ "train_batch_size": 2,
2909
+ "trial_name": null,
2910
+ "trial_params": null
2911
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3bd56f5e149cf0b150f90ab3dcb1fdb48f7680a084d366beb4c63c4d56318225
3
+ size 5624