Hugging Face's logo

sci-m-wang
/
deepseek-llm-7b-chat-sa-v0.1

PEFT
Safetensors
llama-factory
lora
Generated from Trainer
Model card Files
xet
Community
sci-m-wang commited on
Commit
740432a
·
verified ·
1 Parent(s): d071b25

Upload 13 files

Browse files
Files changed (13) hide show
  1. README.md +58 -0
  2. adapter_config.json +29 -0
  3. adapter_model.safetensors +3 -0
  4. all_results.json +8 -0
  5. running_log.txt +1346 -0
  6. special_tokens_map.json +23 -0
  7. tokenizer.json +0 -0
  8. tokenizer_config.json +139 -0
  9. train_results.json +8 -0
  10. trainer_config.yaml +28 -0
  11. trainer_log.jsonl +0 -0
  12. trainer_state.json +3761 -0
  13. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ library_name: peft
4
+ tags:
5
+ - llama-factory
6
+ - lora
7
+ - generated_from_trainer
8
+ base_model: deepseek-ai/deepseek-llm-7b-chat
9
+ model-index:
10
+ - name: deepseek-llm-7b-chat
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # deepseek-llm-7b-chat
18
+
19
+ This model is a fine-tuned version of [/datas/huggingface/deepseek-llm-7b-chat](https://huggingface.co//datas/huggingface/deepseek-llm-7b-chat) on the LangGPT_community, the LangGPT_alpaca and the LangGPT_seed datasets.
20
+
21
+ ## Model description
22
+
23
+ More information needed
24
+
25
+ ## Intended uses & limitations
26
+
27
+ More information needed
28
+
29
+ ## Training and evaluation data
30
+
31
+ More information needed
32
+
33
+ ## Training procedure
34
+
35
+ ### Training hyperparameters
36
+
37
+ The following hyperparameters were used during training:
38
+ - learning_rate: 5e-05
39
+ - train_batch_size: 2
40
+ - eval_batch_size: 8
41
+ - seed: 42
42
+ - gradient_accumulation_steps: 8
43
+ - total_train_batch_size: 16
44
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
+ - lr_scheduler_type: cosine
46
+ - num_epochs: 5.0
47
+
48
+ ### Training results
49
+
50
+
51
+
52
+ ### Framework versions
53
+
54
+ - PEFT 0.10.0
55
+ - Transformers 4.40.2
56
+ - Pytorch 2.3.0+cu121
57
+ - Datasets 2.19.1
58
+ - Tokenizers 0.19.1
adapter_config.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "deepseek-ai/deepseek-llm-7b-chat",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layer_replication": null,
10
+ "layers_pattern": null,
11
+ "layers_to_transform": null,
12
+ "loftq_config": {},
13
+ "lora_alpha": 16,
14
+ "lora_dropout": 0,
15
+ "megatron_config": null,
16
+ "megatron_core": "megatron.core",
17
+ "modules_to_save": null,
18
+ "peft_type": "LORA",
19
+ "r": 8,
20
+ "rank_pattern": {},
21
+ "revision": null,
22
+ "target_modules": [
23
+ "q_proj",
24
+ "v_proj"
25
+ ],
26
+ "task_type": "CAUSAL_LM",
27
+ "use_dora": false,
28
+ "use_rslora": false
29
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:56df04a3731620c031c216d4cd12b6c0c5f892fe0e059732cad8f9da7d12c3e1
3
+ size 15744552
all_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 4.9976558837318334,
3
+ "total_flos": 1.9473516408636703e+18,
4
+ "train_loss": 0.5572177461119575,
5
+ "train_runtime": 35175.961,
6
+ "train_samples_per_second": 1.213,
7
+ "train_steps_per_second": 0.076
8
+ }
running_log.txt ADDED
@@ -0,0 +1,1346 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 05/30/2024 22:53:40 - INFO - transformers.tokenization_utils_base - loading file tokenizer.model
2
+
3
+ 05/30/2024 22:53:40 - INFO - transformers.tokenization_utils_base - loading file tokenizer.json
4
+
5
+ 05/30/2024 22:53:40 - INFO - transformers.tokenization_utils_base - loading file added_tokens.json
6
+
7
+ 05/30/2024 22:53:40 - INFO - transformers.tokenization_utils_base - loading file special_tokens_map.json
8
+
9
+ 05/30/2024 22:53:40 - INFO - transformers.tokenization_utils_base - loading file tokenizer_config.json
10
+
11
+ 05/30/2024 22:53:41 - WARNING - transformers.tokenization_utils_base - Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
12
+
13
+ 05/30/2024 22:53:41 - INFO - llmtuner.data.loader - Loading dataset /datas/wangm/LLM4LangGPT/constructed_datasets/LangGPT_community.jsonl...
14
+
15
+ 05/30/2024 22:53:41 - WARNING - llmtuner.data.utils - Checksum failed: missing SHA-1 hash value in dataset_info.json.
16
+
17
+ 05/30/2024 22:53:42 - INFO - llmtuner.data.loader - Loading dataset /datas/wangm/LLM4LangGPT/constructed_datasets/langgpt_alpaca.jsonl...
18
+
19
+ 05/30/2024 22:53:42 - WARNING - llmtuner.data.utils - Checksum failed: missing SHA-1 hash value in dataset_info.json.
20
+
21
+ 05/30/2024 22:53:44 - INFO - llmtuner.data.loader - Loading dataset /datas/wangm/LLM4LangGPT/constructed_datasets/langgpt_seed.jsonl...
22
+
23
+ 05/30/2024 22:53:44 - WARNING - llmtuner.data.utils - Checksum failed: missing SHA-1 hash value in dataset_info.json.
24
+
25
+ 05/30/2024 22:54:12 - INFO - transformers.configuration_utils - loading configuration file /datas/huggingface/deepseek-llm-7b-chat/config.json
26
+
27
+ 05/30/2024 22:54:12 - INFO - transformers.configuration_utils - Model config LlamaConfig {
28
+ "_name_or_path": "/datas/huggingface/deepseek-llm-7b-chat",
29
+ "architectures": [
30
+ "LlamaForCausalLM"
31
+ ],
32
+ "attention_bias": false,
33
+ "attention_dropout": 0.0,
34
+ "bos_token_id": 100000,
35
+ "eos_token_id": 100001,
36
+ "hidden_act": "silu",
37
+ "hidden_size": 4096,
38
+ "initializer_range": 0.02,
39
+ "intermediate_size": 11008,
40
+ "max_position_embeddings": 4096,
41
+ "model_type": "llama",
42
+ "num_attention_heads": 32,
43
+ "num_hidden_layers": 30,
44
+ "num_key_value_heads": 32,
45
+ "pretraining_tp": 1,
46
+ "rms_norm_eps": 1e-06,
47
+ "rope_scaling": null,
48
+ "rope_theta": 10000.0,
49
+ "tie_word_embeddings": false,
50
+ "torch_dtype": "bfloat16",
51
+ "transformers_version": "4.40.2",
52
+ "use_cache": true,
53
+ "vocab_size": 102400
54
+ }
55
+
56
+
57
+ 05/30/2024 22:54:12 - INFO - transformers.modeling_utils - loading weights file /datas/huggingface/deepseek-llm-7b-chat/pytorch_model.bin.index.json
58
+
59
+ 05/30/2024 22:54:12 - INFO - transformers.modeling_utils - Instantiating LlamaForCausalLM model under default dtype torch.bfloat16.
60
+
61
+ 05/30/2024 22:54:12 - INFO - transformers.generation.configuration_utils - Generate config GenerationConfig {
62
+ "bos_token_id": 100000,
63
+ "eos_token_id": 100001
64
+ }
65
+
66
+
67
+ 05/30/2024 22:55:19 - INFO - transformers.modeling_utils - All model checkpoint weights were used when initializing LlamaForCausalLM.
68
+
69
+
70
+ 05/30/2024 22:55:19 - INFO - transformers.modeling_utils - All the weights of LlamaForCausalLM were initialized from the model checkpoint at /datas/huggingface/deepseek-llm-7b-chat.
71
+ If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.
72
+
73
+ 05/30/2024 22:55:19 - INFO - transformers.generation.configuration_utils - loading configuration file /datas/huggingface/deepseek-llm-7b-chat/generation_config.json
74
+
75
+ 05/30/2024 22:55:19 - INFO - transformers.generation.configuration_utils - Generate config GenerationConfig {
76
+ "bos_token_id": 100000,
77
+ "do_sample": true,
78
+ "eos_token_id": 100001,
79
+ "temperature": 0.7,
80
+ "top_p": 0.95
81
+ }
82
+
83
+
84
+ 05/30/2024 22:55:19 - INFO - llmtuner.model.utils.checkpointing - Gradient checkpointing enabled.
85
+
86
+ 05/30/2024 22:55:19 - INFO - llmtuner.model.utils.attention - Using torch SDPA for faster training and inference.
87
+
88
+ 05/30/2024 22:55:19 - INFO - llmtuner.model.adapter - Fine-tuning method: LoRA
89
+
90
+ 05/30/2024 22:55:19 - INFO - llmtuner.model.loader - trainable params: 3932160 || all params: 6914297856 || trainable%: 0.0569
91
+
92
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - Using auto half precision backend
93
+
94
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - ***** Running training *****
95
+
96
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - Num examples = 8,531
97
+
98
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - Num Epochs = 5
99
+
100
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - Instantaneous batch size per device = 2
101
+
102
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - Total train batch size (w. parallel, distributed & accumulation) = 16
103
+
104
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - Gradient Accumulation steps = 8
105
+
106
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - Total optimization steps = 2,665
107
+
108
+ 05/30/2024 22:55:19 - INFO - transformers.trainer - Number of trainable parameters = 3,932,160
109
+
110
+ 05/30/2024 22:56:32 - INFO - llmtuner.extras.callbacks - {'loss': 0.8631, 'learning_rate': 5.0000e-05, 'epoch': 0.01}
111
+
112
+ 05/30/2024 22:57:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.8372, 'learning_rate': 4.9998e-05, 'epoch': 0.02}
113
+
114
+ 05/30/2024 22:58:42 - INFO - llmtuner.extras.callbacks - {'loss': 0.7690, 'learning_rate': 4.9996e-05, 'epoch': 0.03}
115
+
116
+ 05/30/2024 22:59:45 - INFO - llmtuner.extras.callbacks - {'loss': 0.7971, 'learning_rate': 4.9993e-05, 'epoch': 0.04}
117
+
118
+ 05/30/2024 23:00:50 - INFO - llmtuner.extras.callbacks - {'loss': 0.7329, 'learning_rate': 4.9989e-05, 'epoch': 0.05}
119
+
120
+ 05/30/2024 23:01:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.7026, 'learning_rate': 4.9984e-05, 'epoch': 0.06}
121
+
122
+ 05/30/2024 23:03:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.7233, 'learning_rate': 4.9979e-05, 'epoch': 0.07}
123
+
124
+ 05/30/2024 23:04:07 - INFO - llmtuner.extras.callbacks - {'loss': 0.7185, 'learning_rate': 4.9972e-05, 'epoch': 0.08}
125
+
126
+ 05/30/2024 23:05:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.7380, 'learning_rate': 4.9965e-05, 'epoch': 0.08}
127
+
128
+ 05/30/2024 23:06:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.6857, 'learning_rate': 4.9957e-05, 'epoch': 0.09}
129
+
130
+ 05/30/2024 23:07:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.6763, 'learning_rate': 4.9947e-05, 'epoch': 0.10}
131
+
132
+ 05/30/2024 23:08:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.6673, 'learning_rate': 4.9937e-05, 'epoch': 0.11}
133
+
134
+ 05/30/2024 23:09:32 - INFO - llmtuner.extras.callbacks - {'loss': 0.6489, 'learning_rate': 4.9927e-05, 'epoch': 0.12}
135
+
136
+ 05/30/2024 23:10:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.6335, 'learning_rate': 4.9915e-05, 'epoch': 0.13}
137
+
138
+ 05/30/2024 23:11:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.6739, 'learning_rate': 4.9902e-05, 'epoch': 0.14}
139
+
140
+ 05/30/2024 23:12:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.6087, 'learning_rate': 4.9889e-05, 'epoch': 0.15}
141
+
142
+ 05/30/2024 23:14:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.6207, 'learning_rate': 4.9875e-05, 'epoch': 0.16}
143
+
144
+ 05/30/2024 23:15:05 - INFO - llmtuner.extras.callbacks - {'loss': 0.6429, 'learning_rate': 4.9859e-05, 'epoch': 0.17}
145
+
146
+ 05/30/2024 23:16:09 - INFO - llmtuner.extras.callbacks - {'loss': 0.6033, 'learning_rate': 4.9843e-05, 'epoch': 0.18}
147
+
148
+ 05/30/2024 23:17:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.5861, 'learning_rate': 4.9826e-05, 'epoch': 0.19}
149
+
150
+ 05/30/2024 23:17:13 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-100
151
+
152
+ 05/30/2024 23:17:13 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-100/tokenizer_config.json
153
+
154
+ 05/30/2024 23:17:13 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-100/special_tokens_map.json
155
+
156
+ 05/30/2024 23:18:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.6604, 'learning_rate': 4.9809e-05, 'epoch': 0.20}
157
+
158
+ 05/30/2024 23:19:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.6140, 'learning_rate': 4.9790e-05, 'epoch': 0.21}
159
+
160
+ 05/30/2024 23:20:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.6376, 'learning_rate': 4.9771e-05, 'epoch': 0.22}
161
+
162
+ 05/30/2024 23:21:42 - INFO - llmtuner.extras.callbacks - {'loss': 0.6502, 'learning_rate': 4.9750e-05, 'epoch': 0.23}
163
+
164
+ 05/30/2024 23:22:47 - INFO - llmtuner.extras.callbacks - {'loss': 0.6162, 'learning_rate': 4.9729e-05, 'epoch': 0.23}
165
+
166
+ 05/30/2024 23:23:55 - INFO - llmtuner.extras.callbacks - {'loss': 0.6333, 'learning_rate': 4.9707e-05, 'epoch': 0.24}
167
+
168
+ 05/30/2024 23:25:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.6405, 'learning_rate': 4.9684e-05, 'epoch': 0.25}
169
+
170
+ 05/30/2024 23:26:05 - INFO - llmtuner.extras.callbacks - {'loss': 0.5961, 'learning_rate': 4.9660e-05, 'epoch': 0.26}
171
+
172
+ 05/30/2024 23:27:09 - INFO - llmtuner.extras.callbacks - {'loss': 0.5913, 'learning_rate': 4.9636e-05, 'epoch': 0.27}
173
+
174
+ 05/30/2024 23:28:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.6354, 'learning_rate': 4.9610e-05, 'epoch': 0.28}
175
+
176
+ 05/30/2024 23:29:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.5861, 'learning_rate': 4.9584e-05, 'epoch': 0.29}
177
+
178
+ 05/30/2024 23:30:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.6287, 'learning_rate': 4.9557e-05, 'epoch': 0.30}
179
+
180
+ 05/30/2024 23:31:29 - INFO - llmtuner.extras.callbacks - {'loss': 0.6480, 'learning_rate': 4.9529e-05, 'epoch': 0.31}
181
+
182
+ 05/30/2024 23:32:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.6162, 'learning_rate': 4.9500e-05, 'epoch': 0.32}
183
+
184
+ 05/30/2024 23:33:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.6344, 'learning_rate': 4.9470e-05, 'epoch': 0.33}
185
+
186
+ 05/30/2024 23:34:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.6029, 'learning_rate': 4.9439e-05, 'epoch': 0.34}
187
+
188
+ 05/30/2024 23:36:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5946, 'learning_rate': 4.9408e-05, 'epoch': 0.35}
189
+
190
+ 05/30/2024 23:37:07 - INFO - llmtuner.extras.callbacks - {'loss': 0.5829, 'learning_rate': 4.9376e-05, 'epoch': 0.36}
191
+
192
+ 05/30/2024 23:38:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.6590, 'learning_rate': 4.9342e-05, 'epoch': 0.37}
193
+
194
+ 05/30/2024 23:39:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.6483, 'learning_rate': 4.9308e-05, 'epoch': 0.38}
195
+
196
+ 05/30/2024 23:39:23 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-200
197
+
198
+ 05/30/2024 23:39:23 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-200/tokenizer_config.json
199
+
200
+ 05/30/2024 23:39:23 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-200/special_tokens_map.json
201
+
202
+ 05/30/2024 23:40:37 - INFO - llmtuner.extras.callbacks - {'loss': 0.6110, 'learning_rate': 4.9274e-05, 'epoch': 0.38}
203
+
204
+ 05/30/2024 23:41:43 - INFO - llmtuner.extras.callbacks - {'loss': 0.6201, 'learning_rate': 4.9238e-05, 'epoch': 0.39}
205
+
206
+ 05/30/2024 23:42:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5775, 'learning_rate': 4.9201e-05, 'epoch': 0.40}
207
+
208
+ 05/30/2024 23:43:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.6115, 'learning_rate': 4.9164e-05, 'epoch': 0.41}
209
+
210
+ 05/30/2024 23:45:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5879, 'learning_rate': 4.9126e-05, 'epoch': 0.42}
211
+
212
+ 05/30/2024 23:46:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5617, 'learning_rate': 4.9087e-05, 'epoch': 0.43}
213
+
214
+ 05/30/2024 23:47:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.6132, 'learning_rate': 4.9047e-05, 'epoch': 0.44}
215
+
216
+ 05/30/2024 23:48:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.5923, 'learning_rate': 4.9006e-05, 'epoch': 0.45}
217
+
218
+ 05/30/2024 23:49:32 - INFO - llmtuner.extras.callbacks - {'loss': 0.6131, 'learning_rate': 4.8965e-05, 'epoch': 0.46}
219
+
220
+ 05/30/2024 23:50:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.6137, 'learning_rate': 4.8922e-05, 'epoch': 0.47}
221
+
222
+ 05/30/2024 23:51:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.6239, 'learning_rate': 4.8879e-05, 'epoch': 0.48}
223
+
224
+ 05/30/2024 23:52:52 - INFO - llmtuner.extras.callbacks - {'loss': 0.5665, 'learning_rate': 4.8835e-05, 'epoch': 0.49}
225
+
226
+ 05/30/2024 23:53:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5801, 'learning_rate': 4.8790e-05, 'epoch': 0.50}
227
+
228
+ 05/30/2024 23:55:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5682, 'learning_rate': 4.8744e-05, 'epoch': 0.51}
229
+
230
+ 05/30/2024 23:56:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5698, 'learning_rate': 4.8698e-05, 'epoch': 0.52}
231
+
232
+ 05/30/2024 23:57:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.5948, 'learning_rate': 4.8650e-05, 'epoch': 0.53}
233
+
234
+ 05/30/2024 23:58:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5660, 'learning_rate': 4.8602e-05, 'epoch': 0.53}
235
+
236
+ 05/30/2024 23:59:22 - INFO - llmtuner.extras.callbacks - {'loss': 0.5867, 'learning_rate': 4.8553e-05, 'epoch': 0.54}
237
+
238
+ 05/31/2024 00:00:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.6045, 'learning_rate': 4.8503e-05, 'epoch': 0.55}
239
+
240
+ 05/31/2024 00:01:32 - INFO - llmtuner.extras.callbacks - {'loss': 0.5963, 'learning_rate': 4.8453e-05, 'epoch': 0.56}
241
+
242
+ 05/31/2024 00:01:32 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-300
243
+
244
+ 05/31/2024 00:01:32 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-300/tokenizer_config.json
245
+
246
+ 05/31/2024 00:01:32 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-300/special_tokens_map.json
247
+
248
+ 05/31/2024 00:02:38 - INFO - llmtuner.extras.callbacks - {'loss': 0.6019, 'learning_rate': 4.8401e-05, 'epoch': 0.57}
249
+
250
+ 05/31/2024 00:03:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5811, 'learning_rate': 4.8349e-05, 'epoch': 0.58}
251
+
252
+ 05/31/2024 00:04:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5702, 'learning_rate': 4.8296e-05, 'epoch': 0.59}
253
+
254
+ 05/31/2024 00:05:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.6386, 'learning_rate': 4.8242e-05, 'epoch': 0.60}
255
+
256
+ 05/31/2024 00:06:59 - INFO - llmtuner.extras.callbacks - {'loss': 0.5937, 'learning_rate': 4.8188e-05, 'epoch': 0.61}
257
+
258
+ 05/31/2024 00:08:01 - INFO - llmtuner.extras.callbacks - {'loss': 0.5537, 'learning_rate': 4.8132e-05, 'epoch': 0.62}
259
+
260
+ 05/31/2024 00:09:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.6272, 'learning_rate': 4.8076e-05, 'epoch': 0.63}
261
+
262
+ 05/31/2024 00:10:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5732, 'learning_rate': 4.8019e-05, 'epoch': 0.64}
263
+
264
+ 05/31/2024 00:11:22 - INFO - llmtuner.extras.callbacks - {'loss': 0.6062, 'learning_rate': 4.7961e-05, 'epoch': 0.65}
265
+
266
+ 05/31/2024 00:12:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.6212, 'learning_rate': 4.7902e-05, 'epoch': 0.66}
267
+
268
+ 05/31/2024 00:13:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5662, 'learning_rate': 4.7843e-05, 'epoch': 0.67}
269
+
270
+ 05/31/2024 00:14:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.6173, 'learning_rate': 4.7782e-05, 'epoch': 0.68}
271
+
272
+ 05/31/2024 00:15:46 - INFO - llmtuner.extras.callbacks - {'loss': 0.5543, 'learning_rate': 4.7721e-05, 'epoch': 0.68}
273
+
274
+ 05/31/2024 00:16:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.5608, 'learning_rate': 4.7659e-05, 'epoch': 0.69}
275
+
276
+ 05/31/2024 00:17:58 - INFO - llmtuner.extras.callbacks - {'loss': 0.6113, 'learning_rate': 4.7597e-05, 'epoch': 0.70}
277
+
278
+ 05/31/2024 00:19:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.6040, 'learning_rate': 4.7533e-05, 'epoch': 0.71}
279
+
280
+ 05/31/2024 00:20:09 - INFO - llmtuner.extras.callbacks - {'loss': 0.5725, 'learning_rate': 4.7469e-05, 'epoch': 0.72}
281
+
282
+ 05/31/2024 00:21:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.5792, 'learning_rate': 4.7404e-05, 'epoch': 0.73}
283
+
284
+ 05/31/2024 00:22:17 - INFO - llmtuner.extras.callbacks - {'loss': 0.6010, 'learning_rate': 4.7338e-05, 'epoch': 0.74}
285
+
286
+ 05/31/2024 00:23:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.6168, 'learning_rate': 4.7272e-05, 'epoch': 0.75}
287
+
288
+ 05/31/2024 00:23:23 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-400
289
+
290
+ 05/31/2024 00:23:23 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-400/tokenizer_config.json
291
+
292
+ 05/31/2024 00:23:23 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-400/special_tokens_map.json
293
+
294
+ 05/31/2024 00:24:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5719, 'learning_rate': 4.7204e-05, 'epoch': 0.76}
295
+
296
+ 05/31/2024 00:25:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5798, 'learning_rate': 4.7136e-05, 'epoch': 0.77}
297
+
298
+ 05/31/2024 00:26:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5852, 'learning_rate': 4.7068e-05, 'epoch': 0.78}
299
+
300
+ 05/31/2024 00:27:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.6183, 'learning_rate': 4.6998e-05, 'epoch': 0.79}
301
+
302
+ 05/31/2024 00:29:01 - INFO - llmtuner.extras.callbacks - {'loss': 0.5900, 'learning_rate': 4.6928e-05, 'epoch': 0.80}
303
+
304
+ 05/31/2024 00:30:05 - INFO - llmtuner.extras.callbacks - {'loss': 0.5661, 'learning_rate': 4.6856e-05, 'epoch': 0.81}
305
+
306
+ 05/31/2024 00:31:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5222, 'learning_rate': 4.6784e-05, 'epoch': 0.82}
307
+
308
+ 05/31/2024 00:32:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.5407, 'learning_rate': 4.6712e-05, 'epoch': 0.83}
309
+
310
+ 05/31/2024 00:33:24 - INFO - llmtuner.extras.callbacks - {'loss': 0.5922, 'learning_rate': 4.6638e-05, 'epoch': 0.83}
311
+
312
+ 05/31/2024 00:34:28 - INFO - llmtuner.extras.callbacks - {'loss': 0.5714, 'learning_rate': 4.6564e-05, 'epoch': 0.84}
313
+
314
+ 05/31/2024 00:35:34 - INFO - llmtuner.extras.callbacks - {'loss': 0.5588, 'learning_rate': 4.6489e-05, 'epoch': 0.85}
315
+
316
+ 05/31/2024 00:36:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.5935, 'learning_rate': 4.6414e-05, 'epoch': 0.86}
317
+
318
+ 05/31/2024 00:37:43 - INFO - llmtuner.extras.callbacks - {'loss': 0.5626, 'learning_rate': 4.6337e-05, 'epoch': 0.87}
319
+
320
+ 05/31/2024 00:38:47 - INFO - llmtuner.extras.callbacks - {'loss': 0.5625, 'learning_rate': 4.6260e-05, 'epoch': 0.88}
321
+
322
+ 05/31/2024 00:39:52 - INFO - llmtuner.extras.callbacks - {'loss': 0.5860, 'learning_rate': 4.6182e-05, 'epoch': 0.89}
323
+
324
+ 05/31/2024 00:40:58 - INFO - llmtuner.extras.callbacks - {'loss': 0.5806, 'learning_rate': 4.6103e-05, 'epoch': 0.90}
325
+
326
+ 05/31/2024 00:42:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5682, 'learning_rate': 4.6024e-05, 'epoch': 0.91}
327
+
328
+ 05/31/2024 00:43:06 - INFO - llmtuner.extras.callbacks - {'loss': 0.5594, 'learning_rate': 4.5944e-05, 'epoch': 0.92}
329
+
330
+ 05/31/2024 00:44:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.6643, 'learning_rate': 4.5863e-05, 'epoch': 0.93}
331
+
332
+ 05/31/2024 00:45:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5672, 'learning_rate': 4.5782e-05, 'epoch': 0.94}
333
+
334
+ 05/31/2024 00:45:21 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-500
335
+
336
+ 05/31/2024 00:45:21 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-500/tokenizer_config.json
337
+
338
+ 05/31/2024 00:45:21 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-500/special_tokens_map.json
339
+
340
+ 05/31/2024 00:46:28 - INFO - llmtuner.extras.callbacks - {'loss': 0.5906, 'learning_rate': 4.5699e-05, 'epoch': 0.95}
341
+
342
+ 05/31/2024 00:47:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.6222, 'learning_rate': 4.5616e-05, 'epoch': 0.96}
343
+
344
+ 05/31/2024 00:48:45 - INFO - llmtuner.extras.callbacks - {'loss': 0.5787, 'learning_rate': 4.5533e-05, 'epoch': 0.97}
345
+
346
+ 05/31/2024 00:49:50 - INFO - llmtuner.extras.callbacks - {'loss': 0.5627, 'learning_rate': 4.5448e-05, 'epoch': 0.98}
347
+
348
+ 05/31/2024 00:51:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.5704, 'learning_rate': 4.5363e-05, 'epoch': 0.98}
349
+
350
+ 05/31/2024 00:52:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5879, 'learning_rate': 4.5277e-05, 'epoch': 0.99}
351
+
352
+ 05/31/2024 00:53:11 - INFO - llmtuner.extras.callbacks - {'loss': 0.5509, 'learning_rate': 4.5191e-05, 'epoch': 1.00}
353
+
354
+ 05/31/2024 00:54:16 - INFO - llmtuner.extras.callbacks - {'loss': 0.5293, 'learning_rate': 4.5103e-05, 'epoch': 1.01}
355
+
356
+ 05/31/2024 00:55:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5393, 'learning_rate': 4.5016e-05, 'epoch': 1.02}
357
+
358
+ 05/31/2024 00:56:41 - INFO - llmtuner.extras.callbacks - {'loss': 0.5949, 'learning_rate': 4.4927e-05, 'epoch': 1.03}
359
+
360
+ 05/31/2024 00:57:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.5715, 'learning_rate': 4.4838e-05, 'epoch': 1.04}
361
+
362
+ 05/31/2024 00:59:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5321, 'learning_rate': 4.4748e-05, 'epoch': 1.05}
363
+
364
+ 05/31/2024 01:00:06 - INFO - llmtuner.extras.callbacks - {'loss': 0.5910, 'learning_rate': 4.4657e-05, 'epoch': 1.06}
365
+
366
+ 05/31/2024 01:01:11 - INFO - llmtuner.extras.callbacks - {'loss': 0.5834, 'learning_rate': 4.4565e-05, 'epoch': 1.07}
367
+
368
+ 05/31/2024 01:02:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.5609, 'learning_rate': 4.4473e-05, 'epoch': 1.08}
369
+
370
+ 05/31/2024 01:03:17 - INFO - llmtuner.extras.callbacks - {'loss': 0.5792, 'learning_rate': 4.4381e-05, 'epoch': 1.09}
371
+
372
+ 05/31/2024 01:04:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5680, 'learning_rate': 4.4287e-05, 'epoch': 1.10}
373
+
374
+ 05/31/2024 01:05:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.5428, 'learning_rate': 4.4193e-05, 'epoch': 1.11}
375
+
376
+ 05/31/2024 01:06:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.5640, 'learning_rate': 4.4098e-05, 'epoch': 1.12}
377
+
378
+ 05/31/2024 01:07:45 - INFO - llmtuner.extras.callbacks - {'loss': 0.5625, 'learning_rate': 4.4003e-05, 'epoch': 1.13}
379
+
380
+ 05/31/2024 01:07:45 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-600
381
+
382
+ 05/31/2024 01:07:45 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-600/tokenizer_config.json
383
+
384
+ 05/31/2024 01:07:45 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-600/special_tokens_map.json
385
+
386
+ 05/31/2024 01:08:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5716, 'learning_rate': 4.3907e-05, 'epoch': 1.13}
387
+
388
+ 05/31/2024 01:09:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.5394, 'learning_rate': 4.3810e-05, 'epoch': 1.14}
389
+
390
+ 05/31/2024 01:11:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5344, 'learning_rate': 4.3713e-05, 'epoch': 1.15}
391
+
392
+ 05/31/2024 01:12:07 - INFO - llmtuner.extras.callbacks - {'loss': 0.5440, 'learning_rate': 4.3615e-05, 'epoch': 1.16}
393
+
394
+ 05/31/2024 01:13:11 - INFO - llmtuner.extras.callbacks - {'loss': 0.5481, 'learning_rate': 4.3516e-05, 'epoch': 1.17}
395
+
396
+ 05/31/2024 01:14:22 - INFO - llmtuner.extras.callbacks - {'loss': 0.5715, 'learning_rate': 4.3417e-05, 'epoch': 1.18}
397
+
398
+ 05/31/2024 01:15:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5602, 'learning_rate': 4.3317e-05, 'epoch': 1.19}
399
+
400
+ 05/31/2024 01:16:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5441, 'learning_rate': 4.3216e-05, 'epoch': 1.20}
401
+
402
+ 05/31/2024 01:17:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5369, 'learning_rate': 4.3115e-05, 'epoch': 1.21}
403
+
404
+ 05/31/2024 01:18:37 - INFO - llmtuner.extras.callbacks - {'loss': 0.5770, 'learning_rate': 4.3013e-05, 'epoch': 1.22}
405
+
406
+ 05/31/2024 01:19:50 - INFO - llmtuner.extras.callbacks - {'loss': 0.5762, 'learning_rate': 4.2911e-05, 'epoch': 1.23}
407
+
408
+ 05/31/2024 01:20:55 - INFO - llmtuner.extras.callbacks - {'loss': 0.5726, 'learning_rate': 4.2807e-05, 'epoch': 1.24}
409
+
410
+ 05/31/2024 01:22:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5686, 'learning_rate': 4.2704e-05, 'epoch': 1.25}
411
+
412
+ 05/31/2024 01:23:06 - INFO - llmtuner.extras.callbacks - {'loss': 0.6473, 'learning_rate': 4.2599e-05, 'epoch': 1.26}
413
+
414
+ 05/31/2024 01:24:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.6009, 'learning_rate': 4.2494e-05, 'epoch': 1.27}
415
+
416
+ 05/31/2024 01:25:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.5301, 'learning_rate': 4.2389e-05, 'epoch': 1.28}
417
+
418
+ 05/31/2024 01:26:24 - INFO - llmtuner.extras.callbacks - {'loss': 0.5715, 'learning_rate': 4.2283e-05, 'epoch': 1.28}
419
+
420
+ 05/31/2024 01:27:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.6022, 'learning_rate': 4.2176e-05, 'epoch': 1.29}
421
+
422
+ 05/31/2024 01:28:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5765, 'learning_rate': 4.2069e-05, 'epoch': 1.30}
423
+
424
+ 05/31/2024 01:29:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.5521, 'learning_rate': 4.1961e-05, 'epoch': 1.31}
425
+
426
+ 05/31/2024 01:29:40 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-700
427
+
428
+ 05/31/2024 01:29:40 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-700/tokenizer_config.json
429
+
430
+ 05/31/2024 01:29:40 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-700/special_tokens_map.json
431
+
432
+ 05/31/2024 01:30:47 - INFO - llmtuner.extras.callbacks - {'loss': 0.5709, 'learning_rate': 4.1852e-05, 'epoch': 1.32}
433
+
434
+ 05/31/2024 01:31:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5751, 'learning_rate': 4.1743e-05, 'epoch': 1.33}
435
+
436
+ 05/31/2024 01:32:58 - INFO - llmtuner.extras.callbacks - {'loss': 0.5276, 'learning_rate': 4.1633e-05, 'epoch': 1.34}
437
+
438
+ 05/31/2024 01:34:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5654, 'learning_rate': 4.1523e-05, 'epoch': 1.35}
439
+
440
+ 05/31/2024 01:35:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5771, 'learning_rate': 4.1412e-05, 'epoch': 1.36}
441
+
442
+ 05/31/2024 01:36:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5849, 'learning_rate': 4.1301e-05, 'epoch': 1.37}
443
+
444
+ 05/31/2024 01:37:20 - INFO - llmtuner.extras.callbacks - {'loss': 0.5539, 'learning_rate': 4.1189e-05, 'epoch': 1.38}
445
+
446
+ 05/31/2024 01:38:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5408, 'learning_rate': 4.1076e-05, 'epoch': 1.39}
447
+
448
+ 05/31/2024 01:39:24 - INFO - llmtuner.extras.callbacks - {'loss': 0.5717, 'learning_rate': 4.0963e-05, 'epoch': 1.40}
449
+
450
+ 05/31/2024 01:40:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5612, 'learning_rate': 4.0849e-05, 'epoch': 1.41}
451
+
452
+ 05/31/2024 01:41:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5587, 'learning_rate': 4.0735e-05, 'epoch': 1.42}
453
+
454
+ 05/31/2024 01:42:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5965, 'learning_rate': 4.0620e-05, 'epoch': 1.43}
455
+
456
+ 05/31/2024 01:43:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5576, 'learning_rate': 4.0505e-05, 'epoch': 1.43}
457
+
458
+ 05/31/2024 01:44:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5481, 'learning_rate': 4.0389e-05, 'epoch': 1.44}
459
+
460
+ 05/31/2024 01:46:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5343, 'learning_rate': 4.0273e-05, 'epoch': 1.45}
461
+
462
+ 05/31/2024 01:47:06 - INFO - llmtuner.extras.callbacks - {'loss': 0.5677, 'learning_rate': 4.0156e-05, 'epoch': 1.46}
463
+
464
+ 05/31/2024 01:48:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5650, 'learning_rate': 4.0038e-05, 'epoch': 1.47}
465
+
466
+ 05/31/2024 01:49:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.6299, 'learning_rate': 3.9920e-05, 'epoch': 1.48}
467
+
468
+ 05/31/2024 01:50:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.5588, 'learning_rate': 3.9802e-05, 'epoch': 1.49}
469
+
470
+ 05/31/2024 01:51:28 - INFO - llmtuner.extras.callbacks - {'loss': 0.6042, 'learning_rate': 3.9683e-05, 'epoch': 1.50}
471
+
472
+ 05/31/2024 01:51:28 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-800
473
+
474
+ 05/31/2024 01:51:28 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-800/tokenizer_config.json
475
+
476
+ 05/31/2024 01:51:28 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-800/special_tokens_map.json
477
+
478
+ 05/31/2024 01:52:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5613, 'learning_rate': 3.9563e-05, 'epoch': 1.51}
479
+
480
+ 05/31/2024 01:53:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5466, 'learning_rate': 3.9443e-05, 'epoch': 1.52}
481
+
482
+ 05/31/2024 01:54:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.5305, 'learning_rate': 3.9323e-05, 'epoch': 1.53}
483
+
484
+ 05/31/2024 01:55:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5822, 'learning_rate': 3.9202e-05, 'epoch': 1.54}
485
+
486
+ 05/31/2024 01:56:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5585, 'learning_rate': 3.9080e-05, 'epoch': 1.55}
487
+
488
+ 05/31/2024 01:57:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.5693, 'learning_rate': 3.8958e-05, 'epoch': 1.56}
489
+
490
+ 05/31/2024 01:59:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.5137, 'learning_rate': 3.8836e-05, 'epoch': 1.57}
491
+
492
+ 05/31/2024 02:00:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5622, 'learning_rate': 3.8713e-05, 'epoch': 1.58}
493
+
494
+ 05/31/2024 02:01:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.5321, 'learning_rate': 3.8589e-05, 'epoch': 1.58}
495
+
496
+ 05/31/2024 02:02:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5623, 'learning_rate': 3.8465e-05, 'epoch': 1.59}
497
+
498
+ 05/31/2024 02:03:24 - INFO - llmtuner.extras.callbacks - {'loss': 0.5781, 'learning_rate': 3.8341e-05, 'epoch': 1.60}
499
+
500
+ 05/31/2024 02:04:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5724, 'learning_rate': 3.8216e-05, 'epoch': 1.61}
501
+
502
+ 05/31/2024 02:05:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.5351, 'learning_rate': 3.8091e-05, 'epoch': 1.62}
503
+
504
+ 05/31/2024 02:06:42 - INFO - llmtuner.extras.callbacks - {'loss': 0.5726, 'learning_rate': 3.7965e-05, 'epoch': 1.63}
505
+
506
+ 05/31/2024 02:07:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5326, 'learning_rate': 3.7839e-05, 'epoch': 1.64}
507
+
508
+ 05/31/2024 02:08:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.5384, 'learning_rate': 3.7712e-05, 'epoch': 1.65}
509
+
510
+ 05/31/2024 02:10:01 - INFO - llmtuner.extras.callbacks - {'loss': 0.5808, 'learning_rate': 3.7585e-05, 'epoch': 1.66}
511
+
512
+ 05/31/2024 02:11:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5123, 'learning_rate': 3.7457e-05, 'epoch': 1.67}
513
+
514
+ 05/31/2024 02:12:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5437, 'learning_rate': 3.7329e-05, 'epoch': 1.68}
515
+
516
+ 05/31/2024 02:13:20 - INFO - llmtuner.extras.callbacks - {'loss': 0.6003, 'learning_rate': 3.7201e-05, 'epoch': 1.69}
517
+
518
+ 05/31/2024 02:13:20 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-900
519
+
520
+ 05/31/2024 02:13:20 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-900/tokenizer_config.json
521
+
522
+ 05/31/2024 02:13:20 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-900/special_tokens_map.json
523
+
524
+ 05/31/2024 02:14:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5941, 'learning_rate': 3.7072e-05, 'epoch': 1.70}
525
+
526
+ 05/31/2024 02:15:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.6019, 'learning_rate': 3.6943e-05, 'epoch': 1.71}
527
+
528
+ 05/31/2024 02:16:45 - INFO - llmtuner.extras.callbacks - {'loss': 0.5774, 'learning_rate': 3.6813e-05, 'epoch': 1.72}
529
+
530
+ 05/31/2024 02:17:58 - INFO - llmtuner.extras.callbacks - {'loss': 0.6598, 'learning_rate': 3.6683e-05, 'epoch': 1.73}
531
+
532
+ 05/31/2024 02:19:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5536, 'learning_rate': 3.6553e-05, 'epoch': 1.73}
533
+
534
+ 05/31/2024 02:20:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5606, 'learning_rate': 3.6422e-05, 'epoch': 1.74}
535
+
536
+ 05/31/2024 02:21:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5553, 'learning_rate': 3.6291e-05, 'epoch': 1.75}
537
+
538
+ 05/31/2024 02:22:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5974, 'learning_rate': 3.6159e-05, 'epoch': 1.76}
539
+
540
+ 05/31/2024 02:23:25 - INFO - llmtuner.extras.callbacks - {'loss': 0.5381, 'learning_rate': 3.6027e-05, 'epoch': 1.77}
541
+
542
+ 05/31/2024 02:24:37 - INFO - llmtuner.extras.callbacks - {'loss': 0.5858, 'learning_rate': 3.5894e-05, 'epoch': 1.78}
543
+
544
+ 05/31/2024 02:25:41 - INFO - llmtuner.extras.callbacks - {'loss': 0.5520, 'learning_rate': 3.5762e-05, 'epoch': 1.79}
545
+
546
+ 05/31/2024 02:26:52 - INFO - llmtuner.extras.callbacks - {'loss': 0.6042, 'learning_rate': 3.5628e-05, 'epoch': 1.80}
547
+
548
+ 05/31/2024 02:27:55 - INFO - llmtuner.extras.callbacks - {'loss': 0.5418, 'learning_rate': 3.5495e-05, 'epoch': 1.81}
549
+
550
+ 05/31/2024 02:29:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5743, 'learning_rate': 3.5361e-05, 'epoch': 1.82}
551
+
552
+ 05/31/2024 02:30:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5245, 'learning_rate': 3.5227e-05, 'epoch': 1.83}
553
+
554
+ 05/31/2024 02:31:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.5232, 'learning_rate': 3.5092e-05, 'epoch': 1.84}
555
+
556
+ 05/31/2024 02:32:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.5354, 'learning_rate': 3.4957e-05, 'epoch': 1.85}
557
+
558
+ 05/31/2024 02:33:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5296, 'learning_rate': 3.4822e-05, 'epoch': 1.86}
559
+
560
+ 05/31/2024 02:34:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5486, 'learning_rate': 3.4686e-05, 'epoch': 1.87}
561
+
562
+ 05/31/2024 02:35:42 - INFO - llmtuner.extras.callbacks - {'loss': 0.5339, 'learning_rate': 3.4550e-05, 'epoch': 1.88}
563
+
564
+ 05/31/2024 02:35:42 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1000
565
+
566
+ 05/31/2024 02:35:42 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1000/tokenizer_config.json
567
+
568
+ 05/31/2024 02:35:42 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1000/special_tokens_map.json
569
+
570
+ 05/31/2024 02:36:46 - INFO - llmtuner.extras.callbacks - {'loss': 0.5105, 'learning_rate': 3.4414e-05, 'epoch': 1.88}
571
+
572
+ 05/31/2024 02:37:51 - INFO - llmtuner.extras.callbacks - {'loss': 0.5353, 'learning_rate': 3.4277e-05, 'epoch': 1.89}
573
+
574
+ 05/31/2024 02:38:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5740, 'learning_rate': 3.4140e-05, 'epoch': 1.90}
575
+
576
+ 05/31/2024 02:40:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5580, 'learning_rate': 3.4003e-05, 'epoch': 1.91}
577
+
578
+ 05/31/2024 02:41:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5312, 'learning_rate': 3.3865e-05, 'epoch': 1.92}
579
+
580
+ 05/31/2024 02:42:09 - INFO - llmtuner.extras.callbacks - {'loss': 0.5401, 'learning_rate': 3.3727e-05, 'epoch': 1.93}
581
+
582
+ 05/31/2024 02:43:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5468, 'learning_rate': 3.3589e-05, 'epoch': 1.94}
583
+
584
+ 05/31/2024 02:44:20 - INFO - llmtuner.extras.callbacks - {'loss': 0.5533, 'learning_rate': 3.3450e-05, 'epoch': 1.95}
585
+
586
+ 05/31/2024 02:45:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5615, 'learning_rate': 3.3312e-05, 'epoch': 1.96}
587
+
588
+ 05/31/2024 02:46:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5201, 'learning_rate': 3.3172e-05, 'epoch': 1.97}
589
+
590
+ 05/31/2024 02:47:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5404, 'learning_rate': 3.3033e-05, 'epoch': 1.98}
591
+
592
+ 05/31/2024 02:48:41 - INFO - llmtuner.extras.callbacks - {'loss': 0.5162, 'learning_rate': 3.2893e-05, 'epoch': 1.99}
593
+
594
+ 05/31/2024 02:49:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5430, 'learning_rate': 3.2753e-05, 'epoch': 2.00}
595
+
596
+ 05/31/2024 02:51:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.4996, 'learning_rate': 3.2613e-05, 'epoch': 2.01}
597
+
598
+ 05/31/2024 02:52:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.5008, 'learning_rate': 3.2473e-05, 'epoch': 2.02}
599
+
600
+ 05/31/2024 02:53:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5758, 'learning_rate': 3.2332e-05, 'epoch': 2.03}
601
+
602
+ 05/31/2024 02:54:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.5323, 'learning_rate': 3.2191e-05, 'epoch': 2.03}
603
+
604
+ 05/31/2024 02:55:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5572, 'learning_rate': 3.2050e-05, 'epoch': 2.04}
605
+
606
+ 05/31/2024 02:56:41 - INFO - llmtuner.extras.callbacks - {'loss': 0.5761, 'learning_rate': 3.1908e-05, 'epoch': 2.05}
607
+
608
+ 05/31/2024 02:57:46 - INFO - llmtuner.extras.callbacks - {'loss': 0.5328, 'learning_rate': 3.1767e-05, 'epoch': 2.06}
609
+
610
+ 05/31/2024 02:57:46 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1100
611
+
612
+ 05/31/2024 02:57:46 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1100/tokenizer_config.json
613
+
614
+ 05/31/2024 02:57:46 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1100/special_tokens_map.json
615
+
616
+ 05/31/2024 02:58:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.4987, 'learning_rate': 3.1625e-05, 'epoch': 2.07}
617
+
618
+ 05/31/2024 02:59:58 - INFO - llmtuner.extras.callbacks - {'loss': 0.5158, 'learning_rate': 3.1482e-05, 'epoch': 2.08}
619
+
620
+ 05/31/2024 03:01:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5462, 'learning_rate': 3.1340e-05, 'epoch': 2.09}
621
+
622
+ 05/31/2024 03:02:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5337, 'learning_rate': 3.1197e-05, 'epoch': 2.10}
623
+
624
+ 05/31/2024 03:03:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.5591, 'learning_rate': 3.1054e-05, 'epoch': 2.11}
625
+
626
+ 05/31/2024 03:04:16 - INFO - llmtuner.extras.callbacks - {'loss': 0.5473, 'learning_rate': 3.0911e-05, 'epoch': 2.12}
627
+
628
+ 05/31/2024 03:05:22 - INFO - llmtuner.extras.callbacks - {'loss': 0.5381, 'learning_rate': 3.0768e-05, 'epoch': 2.13}
629
+
630
+ 05/31/2024 03:06:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.6433, 'learning_rate': 3.0625e-05, 'epoch': 2.14}
631
+
632
+ 05/31/2024 03:07:38 - INFO - llmtuner.extras.callbacks - {'loss': 0.5256, 'learning_rate': 3.0481e-05, 'epoch': 2.15}
633
+
634
+ 05/31/2024 03:08:41 - INFO - llmtuner.extras.callbacks - {'loss': 0.5120, 'learning_rate': 3.0337e-05, 'epoch': 2.16}
635
+
636
+ 05/31/2024 03:09:46 - INFO - llmtuner.extras.callbacks - {'loss': 0.5319, 'learning_rate': 3.0193e-05, 'epoch': 2.17}
637
+
638
+ 05/31/2024 03:10:59 - INFO - llmtuner.extras.callbacks - {'loss': 0.5360, 'learning_rate': 3.0049e-05, 'epoch': 2.18}
639
+
640
+ 05/31/2024 03:12:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.5440, 'learning_rate': 2.9904e-05, 'epoch': 2.18}
641
+
642
+ 05/31/2024 03:13:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5989, 'learning_rate': 2.9760e-05, 'epoch': 2.19}
643
+
644
+ 05/31/2024 03:14:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.5598, 'learning_rate': 2.9615e-05, 'epoch': 2.20}
645
+
646
+ 05/31/2024 03:15:17 - INFO - llmtuner.extras.callbacks - {'loss': 0.5667, 'learning_rate': 2.9470e-05, 'epoch': 2.21}
647
+
648
+ 05/31/2024 03:16:28 - INFO - llmtuner.extras.callbacks - {'loss': 0.5661, 'learning_rate': 2.9325e-05, 'epoch': 2.22}
649
+
650
+ 05/31/2024 03:17:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.5312, 'learning_rate': 2.9180e-05, 'epoch': 2.23}
651
+
652
+ 05/31/2024 03:18:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5118, 'learning_rate': 2.9035e-05, 'epoch': 2.24}
653
+
654
+ 05/31/2024 03:19:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5503, 'learning_rate': 2.8889e-05, 'epoch': 2.25}
655
+
656
+ 05/31/2024 03:19:53 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1200
657
+
658
+ 05/31/2024 03:19:53 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1200/tokenizer_config.json
659
+
660
+ 05/31/2024 03:19:53 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1200/special_tokens_map.json
661
+
662
+ 05/31/2024 03:20:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.5345, 'learning_rate': 2.8743e-05, 'epoch': 2.26}
663
+
664
+ 05/31/2024 03:22:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.5940, 'learning_rate': 2.8598e-05, 'epoch': 2.27}
665
+
666
+ 05/31/2024 03:23:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5237, 'learning_rate': 2.8452e-05, 'epoch': 2.28}
667
+
668
+ 05/31/2024 03:24:20 - INFO - llmtuner.extras.callbacks - {'loss': 0.5142, 'learning_rate': 2.8306e-05, 'epoch': 2.29}
669
+
670
+ 05/31/2024 03:25:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.5414, 'learning_rate': 2.8160e-05, 'epoch': 2.30}
671
+
672
+ 05/31/2024 03:26:32 - INFO - llmtuner.extras.callbacks - {'loss': 0.5250, 'learning_rate': 2.8013e-05, 'epoch': 2.31}
673
+
674
+ 05/31/2024 03:27:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5435, 'learning_rate': 2.7867e-05, 'epoch': 2.32}
675
+
676
+ 05/31/2024 03:28:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.5355, 'learning_rate': 2.7721e-05, 'epoch': 2.33}
677
+
678
+ 05/31/2024 03:29:45 - INFO - llmtuner.extras.callbacks - {'loss': 0.5647, 'learning_rate': 2.7574e-05, 'epoch': 2.33}
679
+
680
+ 05/31/2024 03:30:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5580, 'learning_rate': 2.7428e-05, 'epoch': 2.34}
681
+
682
+ 05/31/2024 03:31:58 - INFO - llmtuner.extras.callbacks - {'loss': 0.5683, 'learning_rate': 2.7281e-05, 'epoch': 2.35}
683
+
684
+ 05/31/2024 03:33:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.5172, 'learning_rate': 2.7134e-05, 'epoch': 2.36}
685
+
686
+ 05/31/2024 03:34:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5339, 'learning_rate': 2.6987e-05, 'epoch': 2.37}
687
+
688
+ 05/31/2024 03:35:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.6172, 'learning_rate': 2.6840e-05, 'epoch': 2.38}
689
+
690
+ 05/31/2024 03:36:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5337, 'learning_rate': 2.6693e-05, 'epoch': 2.39}
691
+
692
+ 05/31/2024 03:37:32 - INFO - llmtuner.extras.callbacks - {'loss': 0.5903, 'learning_rate': 2.6546e-05, 'epoch': 2.40}
693
+
694
+ 05/31/2024 03:38:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5693, 'learning_rate': 2.6399e-05, 'epoch': 2.41}
695
+
696
+ 05/31/2024 03:39:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5415, 'learning_rate': 2.6252e-05, 'epoch': 2.42}
697
+
698
+ 05/31/2024 03:40:49 - INFO - llmtuner.extras.callbacks - {'loss': 0.5063, 'learning_rate': 2.6105e-05, 'epoch': 2.43}
699
+
700
+ 05/31/2024 03:41:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5556, 'learning_rate': 2.5958e-05, 'epoch': 2.44}
701
+
702
+ 05/31/2024 03:41:53 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1300
703
+
704
+ 05/31/2024 03:41:53 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1300/tokenizer_config.json
705
+
706
+ 05/31/2024 03:41:53 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1300/special_tokens_map.json
707
+
708
+ 05/31/2024 03:43:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.6267, 'learning_rate': 2.5810e-05, 'epoch': 2.45}
709
+
710
+ 05/31/2024 03:44:09 - INFO - llmtuner.extras.callbacks - {'loss': 0.5637, 'learning_rate': 2.5663e-05, 'epoch': 2.46}
711
+
712
+ 05/31/2024 03:45:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5426, 'learning_rate': 2.5516e-05, 'epoch': 2.47}
713
+
714
+ 05/31/2024 03:46:20 - INFO - llmtuner.extras.callbacks - {'loss': 0.5028, 'learning_rate': 2.5368e-05, 'epoch': 2.48}
715
+
716
+ 05/31/2024 03:47:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.5536, 'learning_rate': 2.5221e-05, 'epoch': 2.48}
717
+
718
+ 05/31/2024 03:48:28 - INFO - llmtuner.extras.callbacks - {'loss': 0.5199, 'learning_rate': 2.5074e-05, 'epoch': 2.49}
719
+
720
+ 05/31/2024 03:49:32 - INFO - llmtuner.extras.callbacks - {'loss': 0.5328, 'learning_rate': 2.4926e-05, 'epoch': 2.50}
721
+
722
+ 05/31/2024 03:50:37 - INFO - llmtuner.extras.callbacks - {'loss': 0.5625, 'learning_rate': 2.4779e-05, 'epoch': 2.51}
723
+
724
+ 05/31/2024 03:51:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.5238, 'learning_rate': 2.4632e-05, 'epoch': 2.52}
725
+
726
+ 05/31/2024 03:52:42 - INFO - llmtuner.extras.callbacks - {'loss': 0.5108, 'learning_rate': 2.4484e-05, 'epoch': 2.53}
727
+
728
+ 05/31/2024 03:53:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5603, 'learning_rate': 2.4337e-05, 'epoch': 2.54}
729
+
730
+ 05/31/2024 03:55:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5740, 'learning_rate': 2.4190e-05, 'epoch': 2.55}
731
+
732
+ 05/31/2024 03:56:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5086, 'learning_rate': 2.4042e-05, 'epoch': 2.56}
733
+
734
+ 05/31/2024 03:57:16 - INFO - llmtuner.extras.callbacks - {'loss': 0.5052, 'learning_rate': 2.3895e-05, 'epoch': 2.57}
735
+
736
+ 05/31/2024 03:58:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5636, 'learning_rate': 2.3748e-05, 'epoch': 2.58}
737
+
738
+ 05/31/2024 03:59:29 - INFO - llmtuner.extras.callbacks - {'loss': 0.5367, 'learning_rate': 2.3601e-05, 'epoch': 2.59}
739
+
740
+ 05/31/2024 04:00:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.5316, 'learning_rate': 2.3454e-05, 'epoch': 2.60}
741
+
742
+ 05/31/2024 04:01:45 - INFO - llmtuner.extras.callbacks - {'loss': 0.5161, 'learning_rate': 2.3307e-05, 'epoch': 2.61}
743
+
744
+ 05/31/2024 04:02:51 - INFO - llmtuner.extras.callbacks - {'loss': 0.5728, 'learning_rate': 2.3160e-05, 'epoch': 2.62}
745
+
746
+ 05/31/2024 04:03:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5436, 'learning_rate': 2.3013e-05, 'epoch': 2.63}
747
+
748
+ 05/31/2024 04:03:56 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1400
749
+
750
+ 05/31/2024 04:03:56 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1400/tokenizer_config.json
751
+
752
+ 05/31/2024 04:03:56 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1400/special_tokens_map.json
753
+
754
+ 05/31/2024 04:05:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5236, 'learning_rate': 2.2866e-05, 'epoch': 2.63}
755
+
756
+ 05/31/2024 04:06:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5078, 'learning_rate': 2.2719e-05, 'epoch': 2.64}
757
+
758
+ 05/31/2024 04:07:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5030, 'learning_rate': 2.2572e-05, 'epoch': 2.65}
759
+
760
+ 05/31/2024 04:08:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5348, 'learning_rate': 2.2426e-05, 'epoch': 2.66}
761
+
762
+ 05/31/2024 04:09:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.5343, 'learning_rate': 2.2279e-05, 'epoch': 2.67}
763
+
764
+ 05/31/2024 04:10:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5586, 'learning_rate': 2.2133e-05, 'epoch': 2.68}
765
+
766
+ 05/31/2024 04:11:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.5474, 'learning_rate': 2.1987e-05, 'epoch': 2.69}
767
+
768
+ 05/31/2024 04:12:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5474, 'learning_rate': 2.1840e-05, 'epoch': 2.70}
769
+
770
+ 05/31/2024 04:13:34 - INFO - llmtuner.extras.callbacks - {'loss': 0.5720, 'learning_rate': 2.1694e-05, 'epoch': 2.71}
771
+
772
+ 05/31/2024 04:14:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.5636, 'learning_rate': 2.1548e-05, 'epoch': 2.72}
773
+
774
+ 05/31/2024 04:15:43 - INFO - llmtuner.extras.callbacks - {'loss': 0.5305, 'learning_rate': 2.1402e-05, 'epoch': 2.73}
775
+
776
+ 05/31/2024 04:16:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5273, 'learning_rate': 2.1257e-05, 'epoch': 2.74}
777
+
778
+ 05/31/2024 04:17:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5383, 'learning_rate': 2.1111e-05, 'epoch': 2.75}
779
+
780
+ 05/31/2024 04:18:59 - INFO - llmtuner.extras.callbacks - {'loss': 0.5482, 'learning_rate': 2.0965e-05, 'epoch': 2.76}
781
+
782
+ 05/31/2024 04:20:05 - INFO - llmtuner.extras.callbacks - {'loss': 0.5226, 'learning_rate': 2.0820e-05, 'epoch': 2.77}
783
+
784
+ 05/31/2024 04:21:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5276, 'learning_rate': 2.0675e-05, 'epoch': 2.78}
785
+
786
+ 05/31/2024 04:22:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5636, 'learning_rate': 2.0530e-05, 'epoch': 2.78}
787
+
788
+ 05/31/2024 04:23:22 - INFO - llmtuner.extras.callbacks - {'loss': 0.5531, 'learning_rate': 2.0385e-05, 'epoch': 2.79}
789
+
790
+ 05/31/2024 04:24:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5709, 'learning_rate': 2.0240e-05, 'epoch': 2.80}
791
+
792
+ 05/31/2024 04:25:34 - INFO - llmtuner.extras.callbacks - {'loss': 0.5300, 'learning_rate': 2.0096e-05, 'epoch': 2.81}
793
+
794
+ 05/31/2024 04:25:34 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1500
795
+
796
+ 05/31/2024 04:25:34 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1500/tokenizer_config.json
797
+
798
+ 05/31/2024 04:25:34 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1500/special_tokens_map.json
799
+
800
+ 05/31/2024 04:26:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.5258, 'learning_rate': 1.9951e-05, 'epoch': 2.82}
801
+
802
+ 05/31/2024 04:27:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5360, 'learning_rate': 1.9807e-05, 'epoch': 2.83}
803
+
804
+ 05/31/2024 04:28:49 - INFO - llmtuner.extras.callbacks - {'loss': 0.5724, 'learning_rate': 1.9663e-05, 'epoch': 2.84}
805
+
806
+ 05/31/2024 04:29:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5325, 'learning_rate': 1.9519e-05, 'epoch': 2.85}
807
+
808
+ 05/31/2024 04:30:59 - INFO - llmtuner.extras.callbacks - {'loss': 0.5505, 'learning_rate': 1.9375e-05, 'epoch': 2.86}
809
+
810
+ 05/31/2024 04:32:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5153, 'learning_rate': 1.9232e-05, 'epoch': 2.87}
811
+
812
+ 05/31/2024 04:33:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5411, 'learning_rate': 1.9089e-05, 'epoch': 2.88}
813
+
814
+ 05/31/2024 04:34:17 - INFO - llmtuner.extras.callbacks - {'loss': 0.5085, 'learning_rate': 1.8946e-05, 'epoch': 2.89}
815
+
816
+ 05/31/2024 04:35:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.5683, 'learning_rate': 1.8803e-05, 'epoch': 2.90}
817
+
818
+ 05/31/2024 04:36:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5393, 'learning_rate': 1.8660e-05, 'epoch': 2.91}
819
+
820
+ 05/31/2024 04:37:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5303, 'learning_rate': 1.8518e-05, 'epoch': 2.92}
821
+
822
+ 05/31/2024 04:38:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.5599, 'learning_rate': 1.8375e-05, 'epoch': 2.93}
823
+
824
+ 05/31/2024 04:39:46 - INFO - llmtuner.extras.callbacks - {'loss': 0.5712, 'learning_rate': 1.8233e-05, 'epoch': 2.93}
825
+
826
+ 05/31/2024 04:40:52 - INFO - llmtuner.extras.callbacks - {'loss': 0.5459, 'learning_rate': 1.8092e-05, 'epoch': 2.94}
827
+
828
+ 05/31/2024 04:41:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5754, 'learning_rate': 1.7950e-05, 'epoch': 2.95}
829
+
830
+ 05/31/2024 04:43:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5615, 'learning_rate': 1.7809e-05, 'epoch': 2.96}
831
+
832
+ 05/31/2024 04:44:05 - INFO - llmtuner.extras.callbacks - {'loss': 0.5135, 'learning_rate': 1.7668e-05, 'epoch': 2.97}
833
+
834
+ 05/31/2024 04:45:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5660, 'learning_rate': 1.7527e-05, 'epoch': 2.98}
835
+
836
+ 05/31/2024 04:46:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5790, 'learning_rate': 1.7387e-05, 'epoch': 2.99}
837
+
838
+ 05/31/2024 04:47:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5286, 'learning_rate': 1.7247e-05, 'epoch': 3.00}
839
+
840
+ 05/31/2024 04:47:21 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1600
841
+
842
+ 05/31/2024 04:47:21 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1600/tokenizer_config.json
843
+
844
+ 05/31/2024 04:47:21 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1600/special_tokens_map.json
845
+
846
+ 05/31/2024 04:48:28 - INFO - llmtuner.extras.callbacks - {'loss': 0.5806, 'learning_rate': 1.7107e-05, 'epoch': 3.01}
847
+
848
+ 05/31/2024 04:49:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.5340, 'learning_rate': 1.6967e-05, 'epoch': 3.02}
849
+
850
+ 05/31/2024 04:50:38 - INFO - llmtuner.extras.callbacks - {'loss': 0.5507, 'learning_rate': 1.6828e-05, 'epoch': 3.03}
851
+
852
+ 05/31/2024 04:51:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5537, 'learning_rate': 1.6688e-05, 'epoch': 3.04}
853
+
854
+ 05/31/2024 04:52:51 - INFO - llmtuner.extras.callbacks - {'loss': 0.5575, 'learning_rate': 1.6550e-05, 'epoch': 3.05}
855
+
856
+ 05/31/2024 04:53:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5043, 'learning_rate': 1.6411e-05, 'epoch': 3.06}
857
+
858
+ 05/31/2024 04:55:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.4903, 'learning_rate': 1.6273e-05, 'epoch': 3.07}
859
+
860
+ 05/31/2024 04:56:09 - INFO - llmtuner.extras.callbacks - {'loss': 0.5934, 'learning_rate': 1.6135e-05, 'epoch': 3.08}
861
+
862
+ 05/31/2024 04:57:16 - INFO - llmtuner.extras.callbacks - {'loss': 0.4922, 'learning_rate': 1.5997e-05, 'epoch': 3.08}
863
+
864
+ 05/31/2024 04:58:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5327, 'learning_rate': 1.5860e-05, 'epoch': 3.09}
865
+
866
+ 05/31/2024 04:59:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.5836, 'learning_rate': 1.5723e-05, 'epoch': 3.10}
867
+
868
+ 05/31/2024 05:00:25 - INFO - llmtuner.extras.callbacks - {'loss': 0.5270, 'learning_rate': 1.5586e-05, 'epoch': 3.11}
869
+
870
+ 05/31/2024 05:01:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5466, 'learning_rate': 1.5450e-05, 'epoch': 3.12}
871
+
872
+ 05/31/2024 05:02:37 - INFO - llmtuner.extras.callbacks - {'loss': 0.5162, 'learning_rate': 1.5314e-05, 'epoch': 3.13}
873
+
874
+ 05/31/2024 05:03:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.4985, 'learning_rate': 1.5178e-05, 'epoch': 3.14}
875
+
876
+ 05/31/2024 05:04:45 - INFO - llmtuner.extras.callbacks - {'loss': 0.5674, 'learning_rate': 1.5043e-05, 'epoch': 3.15}
877
+
878
+ 05/31/2024 05:05:49 - INFO - llmtuner.extras.callbacks - {'loss': 0.5051, 'learning_rate': 1.4908e-05, 'epoch': 3.16}
879
+
880
+ 05/31/2024 05:06:55 - INFO - llmtuner.extras.callbacks - {'loss': 0.5327, 'learning_rate': 1.4773e-05, 'epoch': 3.17}
881
+
882
+ 05/31/2024 05:07:58 - INFO - llmtuner.extras.callbacks - {'loss': 0.5064, 'learning_rate': 1.4639e-05, 'epoch': 3.18}
883
+
884
+ 05/31/2024 05:09:06 - INFO - llmtuner.extras.callbacks - {'loss': 0.5338, 'learning_rate': 1.4505e-05, 'epoch': 3.19}
885
+
886
+ 05/31/2024 05:09:06 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1700
887
+
888
+ 05/31/2024 05:09:06 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1700/tokenizer_config.json
889
+
890
+ 05/31/2024 05:09:06 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1700/special_tokens_map.json
891
+
892
+ 05/31/2024 05:10:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.4959, 'learning_rate': 1.4372e-05, 'epoch': 3.20}
893
+
894
+ 05/31/2024 05:11:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5289, 'learning_rate': 1.4238e-05, 'epoch': 3.21}
895
+
896
+ 05/31/2024 05:12:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5657, 'learning_rate': 1.4106e-05, 'epoch': 3.22}
897
+
898
+ 05/31/2024 05:13:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.5780, 'learning_rate': 1.3973e-05, 'epoch': 3.23}
899
+
900
+ 05/31/2024 05:14:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5262, 'learning_rate': 1.3841e-05, 'epoch': 3.23}
901
+
902
+ 05/31/2024 05:15:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5048, 'learning_rate': 1.3709e-05, 'epoch': 3.24}
903
+
904
+ 05/31/2024 05:16:47 - INFO - llmtuner.extras.callbacks - {'loss': 0.4904, 'learning_rate': 1.3578e-05, 'epoch': 3.25}
905
+
906
+ 05/31/2024 05:17:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5938, 'learning_rate': 1.3447e-05, 'epoch': 3.26}
907
+
908
+ 05/31/2024 05:18:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.5274, 'learning_rate': 1.3317e-05, 'epoch': 3.27}
909
+
910
+ 05/31/2024 05:20:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5083, 'learning_rate': 1.3187e-05, 'epoch': 3.28}
911
+
912
+ 05/31/2024 05:21:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5323, 'learning_rate': 1.3057e-05, 'epoch': 3.29}
913
+
914
+ 05/31/2024 05:22:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.4991, 'learning_rate': 1.2928e-05, 'epoch': 3.30}
915
+
916
+ 05/31/2024 05:23:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.4954, 'learning_rate': 1.2799e-05, 'epoch': 3.31}
917
+
918
+ 05/31/2024 05:24:24 - INFO - llmtuner.extras.callbacks - {'loss': 0.5194, 'learning_rate': 1.2671e-05, 'epoch': 3.32}
919
+
920
+ 05/31/2024 05:25:29 - INFO - llmtuner.extras.callbacks - {'loss': 0.5209, 'learning_rate': 1.2543e-05, 'epoch': 3.33}
921
+
922
+ 05/31/2024 05:26:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5699, 'learning_rate': 1.2415e-05, 'epoch': 3.34}
923
+
924
+ 05/31/2024 05:27:37 - INFO - llmtuner.extras.callbacks - {'loss': 0.5100, 'learning_rate': 1.2288e-05, 'epoch': 3.35}
925
+
926
+ 05/31/2024 05:28:43 - INFO - llmtuner.extras.callbacks - {'loss': 0.5497, 'learning_rate': 1.2161e-05, 'epoch': 3.36}
927
+
928
+ 05/31/2024 05:29:47 - INFO - llmtuner.extras.callbacks - {'loss': 0.5660, 'learning_rate': 1.2035e-05, 'epoch': 3.37}
929
+
930
+ 05/31/2024 05:30:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.5246, 'learning_rate': 1.1909e-05, 'epoch': 3.38}
931
+
932
+ 05/31/2024 05:30:54 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1800
933
+
934
+ 05/31/2024 05:30:54 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1800/tokenizer_config.json
935
+
936
+ 05/31/2024 05:30:54 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1800/special_tokens_map.json
937
+
938
+ 05/31/2024 05:32:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5261, 'learning_rate': 1.1784e-05, 'epoch': 3.38}
939
+
940
+ 05/31/2024 05:33:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5497, 'learning_rate': 1.1659e-05, 'epoch': 3.39}
941
+
942
+ 05/31/2024 05:34:09 - INFO - llmtuner.extras.callbacks - {'loss': 0.5185, 'learning_rate': 1.1535e-05, 'epoch': 3.40}
943
+
944
+ 05/31/2024 05:35:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.5836, 'learning_rate': 1.1411e-05, 'epoch': 3.41}
945
+
946
+ 05/31/2024 05:36:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5318, 'learning_rate': 1.1287e-05, 'epoch': 3.42}
947
+
948
+ 05/31/2024 05:37:25 - INFO - llmtuner.extras.callbacks - {'loss': 0.5141, 'learning_rate': 1.1164e-05, 'epoch': 3.43}
949
+
950
+ 05/31/2024 05:38:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5626, 'learning_rate': 1.1042e-05, 'epoch': 3.44}
951
+
952
+ 05/31/2024 05:39:50 - INFO - llmtuner.extras.callbacks - {'loss': 0.4792, 'learning_rate': 1.0920e-05, 'epoch': 3.45}
953
+
954
+ 05/31/2024 05:41:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5150, 'learning_rate': 1.0798e-05, 'epoch': 3.46}
955
+
956
+ 05/31/2024 05:42:07 - INFO - llmtuner.extras.callbacks - {'loss': 0.5393, 'learning_rate': 1.0677e-05, 'epoch': 3.47}
957
+
958
+ 05/31/2024 05:43:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5096, 'learning_rate': 1.0557e-05, 'epoch': 3.48}
959
+
960
+ 05/31/2024 05:44:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.5266, 'learning_rate': 1.0437e-05, 'epoch': 3.49}
961
+
962
+ 05/31/2024 05:45:22 - INFO - llmtuner.extras.callbacks - {'loss': 0.5003, 'learning_rate': 1.0317e-05, 'epoch': 3.50}
963
+
964
+ 05/31/2024 05:46:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.5228, 'learning_rate': 1.0198e-05, 'epoch': 3.51}
965
+
966
+ 05/31/2024 05:47:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5204, 'learning_rate': 1.0080e-05, 'epoch': 3.52}
967
+
968
+ 05/31/2024 05:48:34 - INFO - llmtuner.extras.callbacks - {'loss': 0.5138, 'learning_rate': 9.9618e-06, 'epoch': 3.53}
969
+
970
+ 05/31/2024 05:49:41 - INFO - llmtuner.extras.callbacks - {'loss': 0.5748, 'learning_rate': 9.8444e-06, 'epoch': 3.53}
971
+
972
+ 05/31/2024 05:50:55 - INFO - llmtuner.extras.callbacks - {'loss': 0.5042, 'learning_rate': 9.7274e-06, 'epoch': 3.54}
973
+
974
+ 05/31/2024 05:52:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.5591, 'learning_rate': 9.6110e-06, 'epoch': 3.55}
975
+
976
+ 05/31/2024 05:53:07 - INFO - llmtuner.extras.callbacks - {'loss': 0.5268, 'learning_rate': 9.4952e-06, 'epoch': 3.56}
977
+
978
+ 05/31/2024 05:53:07 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1900
979
+
980
+ 05/31/2024 05:53:07 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1900/tokenizer_config.json
981
+
982
+ 05/31/2024 05:53:07 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-1900/special_tokens_map.json
983
+
984
+ 05/31/2024 05:54:11 - INFO - llmtuner.extras.callbacks - {'loss': 0.5592, 'learning_rate': 9.3799e-06, 'epoch': 3.57}
985
+
986
+ 05/31/2024 05:55:16 - INFO - llmtuner.extras.callbacks - {'loss': 0.5537, 'learning_rate': 9.2651e-06, 'epoch': 3.58}
987
+
988
+ 05/31/2024 05:56:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5243, 'learning_rate': 9.1508e-06, 'epoch': 3.59}
989
+
990
+ 05/31/2024 05:57:28 - INFO - llmtuner.extras.callbacks - {'loss': 0.5264, 'learning_rate': 9.0372e-06, 'epoch': 3.60}
991
+
992
+ 05/31/2024 05:58:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.5062, 'learning_rate': 8.9240e-06, 'epoch': 3.61}
993
+
994
+ 05/31/2024 05:59:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5729, 'learning_rate': 8.8115e-06, 'epoch': 3.62}
995
+
996
+ 05/31/2024 06:00:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5901, 'learning_rate': 8.6995e-06, 'epoch': 3.63}
997
+
998
+ 05/31/2024 06:01:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.5517, 'learning_rate': 8.5880e-06, 'epoch': 3.64}
999
+
1000
+ 05/31/2024 06:03:01 - INFO - llmtuner.extras.callbacks - {'loss': 0.5617, 'learning_rate': 8.4772e-06, 'epoch': 3.65}
1001
+
1002
+ 05/31/2024 06:04:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5510, 'learning_rate': 8.3669e-06, 'epoch': 3.66}
1003
+
1004
+ 05/31/2024 06:05:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5394, 'learning_rate': 8.2571e-06, 'epoch': 3.67}
1005
+
1006
+ 05/31/2024 06:06:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5532, 'learning_rate': 8.1480e-06, 'epoch': 3.68}
1007
+
1008
+ 05/31/2024 06:07:22 - INFO - llmtuner.extras.callbacks - {'loss': 0.5242, 'learning_rate': 8.0395e-06, 'epoch': 3.68}
1009
+
1010
+ 05/31/2024 06:08:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.6371, 'learning_rate': 7.9315e-06, 'epoch': 3.69}
1011
+
1012
+ 05/31/2024 06:09:42 - INFO - llmtuner.extras.callbacks - {'loss': 0.5144, 'learning_rate': 7.8241e-06, 'epoch': 3.70}
1013
+
1014
+ 05/31/2024 06:10:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5271, 'learning_rate': 7.7173e-06, 'epoch': 3.71}
1015
+
1016
+ 05/31/2024 06:11:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5273, 'learning_rate': 7.6112e-06, 'epoch': 3.72}
1017
+
1018
+ 05/31/2024 06:13:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.5249, 'learning_rate': 7.5056e-06, 'epoch': 3.73}
1019
+
1020
+ 05/31/2024 06:14:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5625, 'learning_rate': 7.4006e-06, 'epoch': 3.74}
1021
+
1022
+ 05/31/2024 06:15:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5701, 'learning_rate': 7.2963e-06, 'epoch': 3.75}
1023
+
1024
+ 05/31/2024 06:15:27 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2000
1025
+
1026
+ 05/31/2024 06:15:27 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2000/tokenizer_config.json
1027
+
1028
+ 05/31/2024 06:15:27 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2000/special_tokens_map.json
1029
+
1030
+ 05/31/2024 06:16:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5830, 'learning_rate': 7.1926e-06, 'epoch': 3.76}
1031
+
1032
+ 05/31/2024 06:17:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.4779, 'learning_rate': 7.0895e-06, 'epoch': 3.77}
1033
+
1034
+ 05/31/2024 06:18:45 - INFO - llmtuner.extras.callbacks - {'loss': 0.4985, 'learning_rate': 6.9870e-06, 'epoch': 3.78}
1035
+
1036
+ 05/31/2024 06:19:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5305, 'learning_rate': 6.8851e-06, 'epoch': 3.79}
1037
+
1038
+ 05/31/2024 06:21:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5356, 'learning_rate': 6.7839e-06, 'epoch': 3.80}
1039
+
1040
+ 05/31/2024 06:22:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5269, 'learning_rate': 6.6833e-06, 'epoch': 3.81}
1041
+
1042
+ 05/31/2024 06:23:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.5563, 'learning_rate': 6.5833e-06, 'epoch': 3.82}
1043
+
1044
+ 05/31/2024 06:24:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5146, 'learning_rate': 6.4840e-06, 'epoch': 3.83}
1045
+
1046
+ 05/31/2024 06:25:24 - INFO - llmtuner.extras.callbacks - {'loss': 0.5614, 'learning_rate': 6.3853e-06, 'epoch': 3.83}
1047
+
1048
+ 05/31/2024 06:26:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.4988, 'learning_rate': 6.2872e-06, 'epoch': 3.84}
1049
+
1050
+ 05/31/2024 06:27:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5621, 'learning_rate': 6.1898e-06, 'epoch': 3.85}
1051
+
1052
+ 05/31/2024 06:28:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.5282, 'learning_rate': 6.0931e-06, 'epoch': 3.86}
1053
+
1054
+ 05/31/2024 06:29:43 - INFO - llmtuner.extras.callbacks - {'loss': 0.5119, 'learning_rate': 5.9970e-06, 'epoch': 3.87}
1055
+
1056
+ 05/31/2024 06:30:47 - INFO - llmtuner.extras.callbacks - {'loss': 0.5431, 'learning_rate': 5.9016e-06, 'epoch': 3.88}
1057
+
1058
+ 05/31/2024 06:31:56 - INFO - llmtuner.extras.callbacks - {'loss': 0.5971, 'learning_rate': 5.8069e-06, 'epoch': 3.89}
1059
+
1060
+ 05/31/2024 06:33:00 - INFO - llmtuner.extras.callbacks - {'loss': 0.5400, 'learning_rate': 5.7128e-06, 'epoch': 3.90}
1061
+
1062
+ 05/31/2024 06:34:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5045, 'learning_rate': 5.6194e-06, 'epoch': 3.91}
1063
+
1064
+ 05/31/2024 06:35:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5086, 'learning_rate': 5.5266e-06, 'epoch': 3.92}
1065
+
1066
+ 05/31/2024 06:36:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.5774, 'learning_rate': 5.4345e-06, 'epoch': 3.93}
1067
+
1068
+ 05/31/2024 06:37:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5541, 'learning_rate': 5.3432e-06, 'epoch': 3.94}
1069
+
1070
+ 05/31/2024 06:37:19 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2100
1071
+
1072
+ 05/31/2024 06:37:19 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2100/tokenizer_config.json
1073
+
1074
+ 05/31/2024 06:37:19 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2100/special_tokens_map.json
1075
+
1076
+ 05/31/2024 06:38:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.5170, 'learning_rate': 5.2524e-06, 'epoch': 3.95}
1077
+
1078
+ 05/31/2024 06:39:38 - INFO - llmtuner.extras.callbacks - {'loss': 0.5474, 'learning_rate': 5.1624e-06, 'epoch': 3.96}
1079
+
1080
+ 05/31/2024 06:40:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5178, 'learning_rate': 5.0731e-06, 'epoch': 3.97}
1081
+
1082
+ 05/31/2024 06:41:50 - INFO - llmtuner.extras.callbacks - {'loss': 0.5671, 'learning_rate': 4.9845e-06, 'epoch': 3.98}
1083
+
1084
+ 05/31/2024 06:42:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5148, 'learning_rate': 4.8965e-06, 'epoch': 3.98}
1085
+
1086
+ 05/31/2024 06:43:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.5418, 'learning_rate': 4.8093e-06, 'epoch': 3.99}
1087
+
1088
+ 05/31/2024 06:45:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5433, 'learning_rate': 4.7227e-06, 'epoch': 4.00}
1089
+
1090
+ 05/31/2024 06:46:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.5810, 'learning_rate': 4.6369e-06, 'epoch': 4.01}
1091
+
1092
+ 05/31/2024 06:47:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5108, 'learning_rate': 4.5518e-06, 'epoch': 4.02}
1093
+
1094
+ 05/31/2024 06:48:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.5533, 'learning_rate': 4.4673e-06, 'epoch': 4.03}
1095
+
1096
+ 05/31/2024 06:49:23 - INFO - llmtuner.extras.callbacks - {'loss': 0.5372, 'learning_rate': 4.3836e-06, 'epoch': 4.04}
1097
+
1098
+ 05/31/2024 06:50:28 - INFO - llmtuner.extras.callbacks - {'loss': 0.5183, 'learning_rate': 4.3006e-06, 'epoch': 4.05}
1099
+
1100
+ 05/31/2024 06:51:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.5426, 'learning_rate': 4.2184e-06, 'epoch': 4.06}
1101
+
1102
+ 05/31/2024 06:52:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5491, 'learning_rate': 4.1368e-06, 'epoch': 4.07}
1103
+
1104
+ 05/31/2024 06:53:41 - INFO - llmtuner.extras.callbacks - {'loss': 0.5401, 'learning_rate': 4.0560e-06, 'epoch': 4.08}
1105
+
1106
+ 05/31/2024 06:54:43 - INFO - llmtuner.extras.callbacks - {'loss': 0.4976, 'learning_rate': 3.9759e-06, 'epoch': 4.09}
1107
+
1108
+ 05/31/2024 06:55:49 - INFO - llmtuner.extras.callbacks - {'loss': 0.5103, 'learning_rate': 3.8965e-06, 'epoch': 4.10}
1109
+
1110
+ 05/31/2024 06:56:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5443, 'learning_rate': 3.8179e-06, 'epoch': 4.11}
1111
+
1112
+ 05/31/2024 06:58:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5377, 'learning_rate': 3.7400e-06, 'epoch': 4.12}
1113
+
1114
+ 05/31/2024 06:59:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.5117, 'learning_rate': 3.6629e-06, 'epoch': 4.13}
1115
+
1116
+ 05/31/2024 06:59:14 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2200
1117
+
1118
+ 05/31/2024 06:59:14 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2200/tokenizer_config.json
1119
+
1120
+ 05/31/2024 06:59:14 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2200/special_tokens_map.json
1121
+
1122
+ 05/31/2024 07:00:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5157, 'learning_rate': 3.5864e-06, 'epoch': 4.14}
1123
+
1124
+ 05/31/2024 07:01:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.5058, 'learning_rate': 3.5108e-06, 'epoch': 4.14}
1125
+
1126
+ 05/31/2024 07:02:29 - INFO - llmtuner.extras.callbacks - {'loss': 0.5233, 'learning_rate': 3.4358e-06, 'epoch': 4.15}
1127
+
1128
+ 05/31/2024 07:03:32 - INFO - llmtuner.extras.callbacks - {'loss': 0.5347, 'learning_rate': 3.3617e-06, 'epoch': 4.16}
1129
+
1130
+ 05/31/2024 07:04:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5124, 'learning_rate': 3.2882e-06, 'epoch': 4.17}
1131
+
1132
+ 05/31/2024 07:05:42 - INFO - llmtuner.extras.callbacks - {'loss': 0.5641, 'learning_rate': 3.2156e-06, 'epoch': 4.18}
1133
+
1134
+ 05/31/2024 07:06:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5416, 'learning_rate': 3.1436e-06, 'epoch': 4.19}
1135
+
1136
+ 05/31/2024 07:07:58 - INFO - llmtuner.extras.callbacks - {'loss': 0.5602, 'learning_rate': 3.0725e-06, 'epoch': 4.20}
1137
+
1138
+ 05/31/2024 07:09:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.4998, 'learning_rate': 3.0021e-06, 'epoch': 4.21}
1139
+
1140
+ 05/31/2024 07:10:07 - INFO - llmtuner.extras.callbacks - {'loss': 0.5699, 'learning_rate': 2.9325e-06, 'epoch': 4.22}
1141
+
1142
+ 05/31/2024 07:11:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.5114, 'learning_rate': 2.8636e-06, 'epoch': 4.23}
1143
+
1144
+ 05/31/2024 07:12:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.5061, 'learning_rate': 2.7955e-06, 'epoch': 4.24}
1145
+
1146
+ 05/31/2024 07:13:24 - INFO - llmtuner.extras.callbacks - {'loss': 0.5334, 'learning_rate': 2.7282e-06, 'epoch': 4.25}
1147
+
1148
+ 05/31/2024 07:14:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5867, 'learning_rate': 2.6616e-06, 'epoch': 4.26}
1149
+
1150
+ 05/31/2024 07:15:36 - INFO - llmtuner.extras.callbacks - {'loss': 0.5287, 'learning_rate': 2.5959e-06, 'epoch': 4.27}
1151
+
1152
+ 05/31/2024 07:16:40 - INFO - llmtuner.extras.callbacks - {'loss': 0.5324, 'learning_rate': 2.5309e-06, 'epoch': 4.28}
1153
+
1154
+ 05/31/2024 07:17:47 - INFO - llmtuner.extras.callbacks - {'loss': 0.5267, 'learning_rate': 2.4667e-06, 'epoch': 4.29}
1155
+
1156
+ 05/31/2024 07:18:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.5480, 'learning_rate': 2.4032e-06, 'epoch': 4.29}
1157
+
1158
+ 05/31/2024 07:19:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.4996, 'learning_rate': 2.3406e-06, 'epoch': 4.30}
1159
+
1160
+ 05/31/2024 07:21:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5294, 'learning_rate': 2.2787e-06, 'epoch': 4.31}
1161
+
1162
+ 05/31/2024 07:21:02 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2300
1163
+
1164
+ 05/31/2024 07:21:02 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2300/tokenizer_config.json
1165
+
1166
+ 05/31/2024 07:21:02 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2300/special_tokens_map.json
1167
+
1168
+ 05/31/2024 07:22:07 - INFO - llmtuner.extras.callbacks - {'loss': 0.4882, 'learning_rate': 2.2176e-06, 'epoch': 4.32}
1169
+
1170
+ 05/31/2024 07:23:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5169, 'learning_rate': 2.1574e-06, 'epoch': 4.33}
1171
+
1172
+ 05/31/2024 07:24:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5325, 'learning_rate': 2.0979e-06, 'epoch': 4.34}
1173
+
1174
+ 05/31/2024 07:25:19 - INFO - llmtuner.extras.callbacks - {'loss': 0.4942, 'learning_rate': 2.0392e-06, 'epoch': 4.35}
1175
+
1176
+ 05/31/2024 07:26:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5117, 'learning_rate': 1.9813e-06, 'epoch': 4.36}
1177
+
1178
+ 05/31/2024 07:27:34 - INFO - llmtuner.extras.callbacks - {'loss': 0.5069, 'learning_rate': 1.9242e-06, 'epoch': 4.37}
1179
+
1180
+ 05/31/2024 07:28:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.5183, 'learning_rate': 1.8679e-06, 'epoch': 4.38}
1181
+
1182
+ 05/31/2024 07:29:52 - INFO - llmtuner.extras.callbacks - {'loss': 0.4979, 'learning_rate': 1.8124e-06, 'epoch': 4.39}
1183
+
1184
+ 05/31/2024 07:30:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.5365, 'learning_rate': 1.7578e-06, 'epoch': 4.40}
1185
+
1186
+ 05/31/2024 07:32:02 - INFO - llmtuner.extras.callbacks - {'loss': 0.5165, 'learning_rate': 1.7039e-06, 'epoch': 4.41}
1187
+
1188
+ 05/31/2024 07:33:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5718, 'learning_rate': 1.6508e-06, 'epoch': 4.42}
1189
+
1190
+ 05/31/2024 07:34:16 - INFO - llmtuner.extras.callbacks - {'loss': 0.5244, 'learning_rate': 1.5986e-06, 'epoch': 4.43}
1191
+
1192
+ 05/31/2024 07:35:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5609, 'learning_rate': 1.5471e-06, 'epoch': 4.44}
1193
+
1194
+ 05/31/2024 07:36:25 - INFO - llmtuner.extras.callbacks - {'loss': 0.4940, 'learning_rate': 1.4965e-06, 'epoch': 4.44}
1195
+
1196
+ 05/31/2024 07:37:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5198, 'learning_rate': 1.4467e-06, 'epoch': 4.45}
1197
+
1198
+ 05/31/2024 07:38:37 - INFO - llmtuner.extras.callbacks - {'loss': 0.5284, 'learning_rate': 1.3977e-06, 'epoch': 4.46}
1199
+
1200
+ 05/31/2024 07:39:44 - INFO - llmtuner.extras.callbacks - {'loss': 0.5223, 'learning_rate': 1.3495e-06, 'epoch': 4.47}
1201
+
1202
+ 05/31/2024 07:40:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5128, 'learning_rate': 1.3022e-06, 'epoch': 4.48}
1203
+
1204
+ 05/31/2024 07:41:59 - INFO - llmtuner.extras.callbacks - {'loss': 0.5634, 'learning_rate': 1.2557e-06, 'epoch': 4.49}
1205
+
1206
+ 05/31/2024 07:43:03 - INFO - llmtuner.extras.callbacks - {'loss': 0.5168, 'learning_rate': 1.2100e-06, 'epoch': 4.50}
1207
+
1208
+ 05/31/2024 07:43:03 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2400
1209
+
1210
+ 05/31/2024 07:43:03 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2400/tokenizer_config.json
1211
+
1212
+ 05/31/2024 07:43:03 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2400/special_tokens_map.json
1213
+
1214
+ 05/31/2024 07:44:05 - INFO - llmtuner.extras.callbacks - {'loss': 0.4968, 'learning_rate': 1.1651e-06, 'epoch': 4.51}
1215
+
1216
+ 05/31/2024 07:45:10 - INFO - llmtuner.extras.callbacks - {'loss': 0.6302, 'learning_rate': 1.1210e-06, 'epoch': 4.52}
1217
+
1218
+ 05/31/2024 07:46:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.5114, 'learning_rate': 1.0778e-06, 'epoch': 4.53}
1219
+
1220
+ 05/31/2024 07:47:17 - INFO - llmtuner.extras.callbacks - {'loss': 0.5136, 'learning_rate': 1.0354e-06, 'epoch': 4.54}
1221
+
1222
+ 05/31/2024 07:48:22 - INFO - llmtuner.extras.callbacks - {'loss': 0.5265, 'learning_rate': 9.9389e-07, 'epoch': 4.55}
1223
+
1224
+ 05/31/2024 07:49:26 - INFO - llmtuner.extras.callbacks - {'loss': 0.5299, 'learning_rate': 9.5317e-07, 'epoch': 4.56}
1225
+
1226
+ 05/31/2024 07:50:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5940, 'learning_rate': 9.1329e-07, 'epoch': 4.57}
1227
+
1228
+ 05/31/2024 07:51:38 - INFO - llmtuner.extras.callbacks - {'loss': 0.5352, 'learning_rate': 8.7424e-07, 'epoch': 4.58}
1229
+
1230
+ 05/31/2024 07:52:43 - INFO - llmtuner.extras.callbacks - {'loss': 0.5262, 'learning_rate': 8.3604e-07, 'epoch': 4.59}
1231
+
1232
+ 05/31/2024 07:53:48 - INFO - llmtuner.extras.callbacks - {'loss': 0.5779, 'learning_rate': 7.9867e-07, 'epoch': 4.59}
1233
+
1234
+ 05/31/2024 07:54:55 - INFO - llmtuner.extras.callbacks - {'loss': 0.5588, 'learning_rate': 7.6214e-07, 'epoch': 4.60}
1235
+
1236
+ 05/31/2024 07:56:01 - INFO - llmtuner.extras.callbacks - {'loss': 0.5025, 'learning_rate': 7.2645e-07, 'epoch': 4.61}
1237
+
1238
+ 05/31/2024 07:57:06 - INFO - llmtuner.extras.callbacks - {'loss': 0.5156, 'learning_rate': 6.9161e-07, 'epoch': 4.62}
1239
+
1240
+ 05/31/2024 07:58:14 - INFO - llmtuner.extras.callbacks - {'loss': 0.5750, 'learning_rate': 6.5761e-07, 'epoch': 4.63}
1241
+
1242
+ 05/31/2024 07:59:27 - INFO - llmtuner.extras.callbacks - {'loss': 0.5262, 'learning_rate': 6.2446e-07, 'epoch': 4.64}
1243
+
1244
+ 05/31/2024 08:00:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.5399, 'learning_rate': 5.9216e-07, 'epoch': 4.65}
1245
+
1246
+ 05/31/2024 08:01:41 - INFO - llmtuner.extras.callbacks - {'loss': 0.5328, 'learning_rate': 5.6070e-07, 'epoch': 4.66}
1247
+
1248
+ 05/31/2024 08:02:46 - INFO - llmtuner.extras.callbacks - {'loss': 0.5403, 'learning_rate': 5.3009e-07, 'epoch': 4.67}
1249
+
1250
+ 05/31/2024 08:03:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5219, 'learning_rate': 5.0033e-07, 'epoch': 4.68}
1251
+
1252
+ 05/31/2024 08:04:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.5515, 'learning_rate': 4.7143e-07, 'epoch': 4.69}
1253
+
1254
+ 05/31/2024 08:04:57 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2500
1255
+
1256
+ 05/31/2024 08:04:57 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2500/tokenizer_config.json
1257
+
1258
+ 05/31/2024 08:04:57 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2500/special_tokens_map.json
1259
+
1260
+ 05/31/2024 08:06:01 - INFO - llmtuner.extras.callbacks - {'loss': 0.5270, 'learning_rate': 4.4337e-07, 'epoch': 4.70}
1261
+
1262
+ 05/31/2024 08:07:06 - INFO - llmtuner.extras.callbacks - {'loss': 0.5766, 'learning_rate': 4.1617e-07, 'epoch': 4.71}
1263
+
1264
+ 05/31/2024 08:08:13 - INFO - llmtuner.extras.callbacks - {'loss': 0.5084, 'learning_rate': 3.8982e-07, 'epoch': 4.72}
1265
+
1266
+ 05/31/2024 08:09:18 - INFO - llmtuner.extras.callbacks - {'loss': 0.5616, 'learning_rate': 3.6433e-07, 'epoch': 4.73}
1267
+
1268
+ 05/31/2024 08:10:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5003, 'learning_rate': 3.3969e-07, 'epoch': 4.74}
1269
+
1270
+ 05/31/2024 08:11:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5369, 'learning_rate': 3.1591e-07, 'epoch': 4.74}
1271
+
1272
+ 05/31/2024 08:12:33 - INFO - llmtuner.extras.callbacks - {'loss': 0.5319, 'learning_rate': 2.9299e-07, 'epoch': 4.75}
1273
+
1274
+ 05/31/2024 08:13:46 - INFO - llmtuner.extras.callbacks - {'loss': 0.4929, 'learning_rate': 2.7093e-07, 'epoch': 4.76}
1275
+
1276
+ 05/31/2024 08:14:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5774, 'learning_rate': 2.4972e-07, 'epoch': 4.77}
1277
+
1278
+ 05/31/2024 08:15:59 - INFO - llmtuner.extras.callbacks - {'loss': 0.5231, 'learning_rate': 2.2937e-07, 'epoch': 4.78}
1279
+
1280
+ 05/31/2024 08:17:06 - INFO - llmtuner.extras.callbacks - {'loss': 0.5563, 'learning_rate': 2.0989e-07, 'epoch': 4.79}
1281
+
1282
+ 05/31/2024 08:18:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5601, 'learning_rate': 1.9127e-07, 'epoch': 4.80}
1283
+
1284
+ 05/31/2024 08:19:16 - INFO - llmtuner.extras.callbacks - {'loss': 0.5116, 'learning_rate': 1.7351e-07, 'epoch': 4.81}
1285
+
1286
+ 05/31/2024 08:20:21 - INFO - llmtuner.extras.callbacks - {'loss': 0.5159, 'learning_rate': 1.5661e-07, 'epoch': 4.82}
1287
+
1288
+ 05/31/2024 08:21:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.5555, 'learning_rate': 1.4057e-07, 'epoch': 4.83}
1289
+
1290
+ 05/31/2024 08:22:37 - INFO - llmtuner.extras.callbacks - {'loss': 0.5433, 'learning_rate': 1.2540e-07, 'epoch': 4.84}
1291
+
1292
+ 05/31/2024 08:23:51 - INFO - llmtuner.extras.callbacks - {'loss': 0.4752, 'learning_rate': 1.1109e-07, 'epoch': 4.85}
1293
+
1294
+ 05/31/2024 08:24:54 - INFO - llmtuner.extras.callbacks - {'loss': 0.4921, 'learning_rate': 9.7646e-08, 'epoch': 4.86}
1295
+
1296
+ 05/31/2024 08:25:57 - INFO - llmtuner.extras.callbacks - {'loss': 0.5313, 'learning_rate': 8.5068e-08, 'epoch': 4.87}
1297
+
1298
+ 05/31/2024 08:26:59 - INFO - llmtuner.extras.callbacks - {'loss': 0.5180, 'learning_rate': 7.3355e-08, 'epoch': 4.88}
1299
+
1300
+ 05/31/2024 08:26:59 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2600
1301
+
1302
+ 05/31/2024 08:27:00 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2600/tokenizer_config.json
1303
+
1304
+ 05/31/2024 08:27:00 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/checkpoint-2600/special_tokens_map.json
1305
+
1306
+ 05/31/2024 08:28:07 - INFO - llmtuner.extras.callbacks - {'loss': 0.5785, 'learning_rate': 6.2508e-08, 'epoch': 4.89}
1307
+
1308
+ 05/31/2024 08:29:12 - INFO - llmtuner.extras.callbacks - {'loss': 0.5209, 'learning_rate': 5.2528e-08, 'epoch': 4.89}
1309
+
1310
+ 05/31/2024 08:30:20 - INFO - llmtuner.extras.callbacks - {'loss': 0.5438, 'learning_rate': 4.3414e-08, 'epoch': 4.90}
1311
+
1312
+ 05/31/2024 08:31:25 - INFO - llmtuner.extras.callbacks - {'loss': 0.4971, 'learning_rate': 3.5167e-08, 'epoch': 4.91}
1313
+
1314
+ 05/31/2024 08:32:31 - INFO - llmtuner.extras.callbacks - {'loss': 0.4950, 'learning_rate': 2.7788e-08, 'epoch': 4.92}
1315
+
1316
+ 05/31/2024 08:33:39 - INFO - llmtuner.extras.callbacks - {'loss': 0.5463, 'learning_rate': 2.1276e-08, 'epoch': 4.93}
1317
+
1318
+ 05/31/2024 08:34:53 - INFO - llmtuner.extras.callbacks - {'loss': 0.5461, 'learning_rate': 1.5632e-08, 'epoch': 4.94}
1319
+
1320
+ 05/31/2024 08:36:04 - INFO - llmtuner.extras.callbacks - {'loss': 0.5319, 'learning_rate': 1.0856e-08, 'epoch': 4.95}
1321
+
1322
+ 05/31/2024 08:37:08 - INFO - llmtuner.extras.callbacks - {'loss': 0.5068, 'learning_rate': 6.9479e-09, 'epoch': 4.96}
1323
+
1324
+ 05/31/2024 08:38:15 - INFO - llmtuner.extras.callbacks - {'loss': 0.5091, 'learning_rate': 3.9083e-09, 'epoch': 4.97}
1325
+
1326
+ 05/31/2024 08:39:24 - INFO - llmtuner.extras.callbacks - {'loss': 0.5747, 'learning_rate': 1.7370e-09, 'epoch': 4.98}
1327
+
1328
+ 05/31/2024 08:40:30 - INFO - llmtuner.extras.callbacks - {'loss': 0.5346, 'learning_rate': 4.3426e-10, 'epoch': 4.99}
1329
+
1330
+ 05/31/2024 08:41:35 - INFO - llmtuner.extras.callbacks - {'loss': 0.5285, 'learning_rate': 0.0000e+00, 'epoch': 5.00}
1331
+
1332
+ 05/31/2024 08:41:35 - INFO - transformers.trainer -
1333
+
1334
+ Training completed. Do not forget to share your model on huggingface.co/models =)
1335
+
1336
+
1337
+
1338
+ 05/31/2024 08:41:35 - INFO - transformers.trainer - Saving model checkpoint to /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat
1339
+
1340
+ 05/31/2024 08:41:35 - INFO - transformers.tokenization_utils_base - tokenizer config file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/tokenizer_config.json
1341
+
1342
+ 05/31/2024 08:41:35 - INFO - transformers.tokenization_utils_base - Special tokens file saved in /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat/special_tokens_map.json
1343
+
1344
+ 05/31/2024 08:41:35 - INFO - transformers.modelcard - Dropping the following result as it does not have all the necessary fields:
1345
+ {'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}
1346
+
special_tokens_map.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<｜begin▁of▁sentence｜>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<｜end▁of▁sentence｜>",
11
+ "lstrip": false,
12
+ "normalized": true,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<｜end▁of▁sentence｜>",
18
+ "lstrip": false,
19
+ "normalized": true,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ }
23
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "100000": {
6
+ "content": "<｜begin▁of▁sentence｜>",
7
+ "lstrip": false,
8
+ "normalized": true,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "100001": {
14
+ "content": "<｜end▁of▁sentence｜>",
15
+ "lstrip": false,
16
+ "normalized": true,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "100002": {
22
+ "content": "ø",
23
+ "lstrip": false,
24
+ "normalized": true,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": false
28
+ },
29
+ "100003": {
30
+ "content": "ö",
31
+ "lstrip": false,
32
+ "normalized": true,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": false
36
+ },
37
+ "100004": {
38
+ "content": "ú",
39
+ "lstrip": false,
40
+ "normalized": true,
41
+ "rstrip": false,
42
+ "single_word": false,
43
+ "special": false
44
+ },
45
+ "100005": {
46
+ "content": "ÿ",
47
+ "lstrip": false,
48
+ "normalized": true,
49
+ "rstrip": false,
50
+ "single_word": false,
51
+ "special": false
52
+ },
53
+ "100006": {
54
+ "content": "õ",
55
+ "lstrip": false,
56
+ "normalized": true,
57
+ "rstrip": false,
58
+ "single_word": false,
59
+ "special": false
60
+ },
61
+ "100007": {
62
+ "content": "÷",
63
+ "lstrip": false,
64
+ "normalized": true,
65
+ "rstrip": false,
66
+ "single_word": false,
67
+ "special": false
68
+ },
69
+ "100008": {
70
+ "content": "û",
71
+ "lstrip": false,
72
+ "normalized": true,
73
+ "rstrip": false,
74
+ "single_word": false,
75
+ "special": false
76
+ },
77
+ "100009": {
78
+ "content": "ý",
79
+ "lstrip": false,
80
+ "normalized": true,
81
+ "rstrip": false,
82
+ "single_word": false,
83
+ "special": false
84
+ },
85
+ "100010": {
86
+ "content": "À",
87
+ "lstrip": false,
88
+ "normalized": true,
89
+ "rstrip": false,
90
+ "single_word": false,
91
+ "special": false
92
+ },
93
+ "100011": {
94
+ "content": "ù",
95
+ "lstrip": false,
96
+ "normalized": true,
97
+ "rstrip": false,
98
+ "single_word": false,
99
+ "special": false
100
+ },
101
+ "100012": {
102
+ "content": "Á",
103
+ "lstrip": false,
104
+ "normalized": true,
105
+ "rstrip": false,
106
+ "single_word": false,
107
+ "special": false
108
+ },
109
+ "100013": {
110
+ "content": "þ",
111
+ "lstrip": false,
112
+ "normalized": true,
113
+ "rstrip": false,
114
+ "single_word": false,
115
+ "special": false
116
+ },
117
+ "100014": {
118
+ "content": "ü",
119
+ "lstrip": false,
120
+ "normalized": true,
121
+ "rstrip": false,
122
+ "single_word": false,
123
+ "special": false
124
+ }
125
+ },
126
+ "bos_token": "<｜begin▁of▁sentence｜>",
127
+ "chat_template": "{% if messages[0]['role'] == 'system' %}{% set system_message = messages[0]['content'] %}{% endif %}{{ '<｜begin▁of▁sentence｜>' + system_message }}{% for message in messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ 'User: ' + content + '\\n\\nAssistant:' }}{% elif message['role'] == 'assistant' %}{{ content + '<｜end▁of▁sentence｜>' }}{% endif %}{% endfor %}",
128
+ "clean_up_tokenization_spaces": false,
129
+ "eos_token": "<｜end▁of▁sentence｜>",
130
+ "legacy": true,
131
+ "model_max_length": 4096,
132
+ "pad_token": "<｜end▁of▁sentence｜>",
133
+ "padding_side": "right",
134
+ "sp_model_kwargs": {},
135
+ "split_special_tokens": false,
136
+ "tokenizer_class": "LlamaTokenizer",
137
+ "unk_token": null,
138
+ "use_default_system_prompt": false
139
+ }
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 4.9976558837318334,
3
+ "total_flos": 1.9473516408636703e+18,
4
+ "train_loss": 0.5572177461119575,
5
+ "train_runtime": 35175.961,
6
+ "train_samples_per_second": 1.213,
7
+ "train_steps_per_second": 0.076
8
+ }
trainer_config.yaml ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bf16: true
2
+ cutoff_len: 4096
3
+ dataset: LangGPT_community,LangGPT_alpaca,LangGPT_seed
4
+ dataset_dir: /datas/wangm/LLM4LangGPT
5
+ do_train: true
6
+ finetuning_type: lora
7
+ flash_attn: auto
8
+ gradient_accumulation_steps: 8
9
+ learning_rate: 5.0e-05
10
+ logging_steps: 5
11
+ lora_alpha: 16
12
+ lora_dropout: 0
13
+ lora_rank: 8
14
+ lora_target: q_proj,v_proj
15
+ lr_scheduler_type: cosine
16
+ max_grad_norm: 1.0
17
+ max_samples: 100000
18
+ model_name_or_path: deepseek-ai/deepseek-llm-7b-chat
19
+ num_train_epochs: 5.0
20
+ optim: adamw_torch
21
+ output_dir: /datas/wangm/LLM4LangGPT/output/deepseek-llm-7b-chat
22
+ packing: false
23
+ per_device_train_batch_size: 2
24
+ report_to: none
25
+ save_steps: 100
26
+ stage: sft
27
+ template: deepseek
28
+ warmup_steps: 0
trainer_log.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
trainer_state.json ADDED
@@ -0,0 +1,3761 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 4.9976558837318334,
5
+ "eval_steps": 500,
6
+ "global_step": 2665,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.009376465072667605,
13
+ "grad_norm": 0.23995593190193176,
14
+ "learning_rate": 4.999956573574533e-05,
15
+ "loss": 0.8631,
16
+ "step": 5
17
+ },
18
+ {
19
+ "epoch": 0.01875293014533521,
20
+ "grad_norm": 0.2710498571395874,
21
+ "learning_rate": 4.999826295806815e-05,
22
+ "loss": 0.8372,
23
+ "step": 10
24
+ },
25
+ {
26
+ "epoch": 0.02812939521800281,
27
+ "grad_norm": 0.20431986451148987,
28
+ "learning_rate": 4.999609171222846e-05,
29
+ "loss": 0.769,
30
+ "step": 15
31
+ },
32
+ {
33
+ "epoch": 0.03750586029067042,
34
+ "grad_norm": 0.2436569333076477,
35
+ "learning_rate": 4.99930520736578e-05,
36
+ "loss": 0.7971,
37
+ "step": 20
38
+ },
39
+ {
40
+ "epoch": 0.04688232536333802,
41
+ "grad_norm": 0.25712528824806213,
42
+ "learning_rate": 4.998914414795668e-05,
43
+ "loss": 0.7329,
44
+ "step": 25
45
+ },
46
+ {
47
+ "epoch": 0.05625879043600562,
48
+ "grad_norm": 0.24791960418224335,
49
+ "learning_rate": 4.99843680708909e-05,
50
+ "loss": 0.7026,
51
+ "step": 30
52
+ },
53
+ {
54
+ "epoch": 0.06563525550867323,
55
+ "grad_norm": 0.27433285117149353,
56
+ "learning_rate": 4.997872400838682e-05,
57
+ "loss": 0.7233,
58
+ "step": 35
59
+ },
60
+ {
61
+ "epoch": 0.07501172058134084,
62
+ "grad_norm": 0.2803635597229004,
63
+ "learning_rate": 4.997221215652562e-05,
64
+ "loss": 0.7185,
65
+ "step": 40
66
+ },
67
+ {
68
+ "epoch": 0.08438818565400844,
69
+ "grad_norm": 0.24648822844028473,
70
+ "learning_rate": 4.9964832741536444e-05,
71
+ "loss": 0.738,
72
+ "step": 45
73
+ },
74
+ {
75
+ "epoch": 0.09376465072667604,
76
+ "grad_norm": 0.3567708730697632,
77
+ "learning_rate": 4.9956586019788584e-05,
78
+ "loss": 0.6857,
79
+ "step": 50
80
+ },
81
+ {
82
+ "epoch": 0.10314111579934365,
83
+ "grad_norm": 0.22938202321529388,
84
+ "learning_rate": 4.9947472277782584e-05,
85
+ "loss": 0.6763,
86
+ "step": 55
87
+ },
88
+ {
89
+ "epoch": 0.11251758087201125,
90
+ "grad_norm": 0.314599871635437,
91
+ "learning_rate": 4.993749183214021e-05,
92
+ "loss": 0.6673,
93
+ "step": 60
94
+ },
95
+ {
96
+ "epoch": 0.12189404594467886,
97
+ "grad_norm": 0.22777864336967468,
98
+ "learning_rate": 4.992664502959351e-05,
99
+ "loss": 0.6489,
100
+ "step": 65
101
+ },
102
+ {
103
+ "epoch": 0.13127051101734646,
104
+ "grad_norm": 0.2409321665763855,
105
+ "learning_rate": 4.991493224697281e-05,
106
+ "loss": 0.6335,
107
+ "step": 70
108
+ },
109
+ {
110
+ "epoch": 0.14064697609001406,
111
+ "grad_norm": 0.25169098377227783,
112
+ "learning_rate": 4.990235389119352e-05,
113
+ "loss": 0.6739,
114
+ "step": 75
115
+ },
116
+ {
117
+ "epoch": 0.15002344116268168,
118
+ "grad_norm": 0.25553980469703674,
119
+ "learning_rate": 4.9888910399242065e-05,
120
+ "loss": 0.6087,
121
+ "step": 80
122
+ },
123
+ {
124
+ "epoch": 0.15939990623534928,
125
+ "grad_norm": 0.2406449317932129,
126
+ "learning_rate": 4.987460223816067e-05,
127
+ "loss": 0.6207,
128
+ "step": 85
129
+ },
130
+ {
131
+ "epoch": 0.16877637130801687,
132
+ "grad_norm": 0.300151526927948,
133
+ "learning_rate": 4.985942990503119e-05,
134
+ "loss": 0.6429,
135
+ "step": 90
136
+ },
137
+ {
138
+ "epoch": 0.1781528363806845,
139
+ "grad_norm": 0.2550767660140991,
140
+ "learning_rate": 4.984339392695777e-05,
141
+ "loss": 0.6033,
142
+ "step": 95
143
+ },
144
+ {
145
+ "epoch": 0.1875293014533521,
146
+ "grad_norm": 0.27587273716926575,
147
+ "learning_rate": 4.9826494861048576e-05,
148
+ "loss": 0.5861,
149
+ "step": 100
150
+ },
151
+ {
152
+ "epoch": 0.19690576652601968,
153
+ "grad_norm": 0.23900988698005676,
154
+ "learning_rate": 4.980873329439644e-05,
155
+ "loss": 0.6604,
156
+ "step": 105
157
+ },
158
+ {
159
+ "epoch": 0.2062822315986873,
160
+ "grad_norm": 0.25917357206344604,
161
+ "learning_rate": 4.979010984405842e-05,
162
+ "loss": 0.614,
163
+ "step": 110
164
+ },
165
+ {
166
+ "epoch": 0.2156586966713549,
167
+ "grad_norm": 0.27334660291671753,
168
+ "learning_rate": 4.9770625157034436e-05,
169
+ "loss": 0.6376,
170
+ "step": 115
171
+ },
172
+ {
173
+ "epoch": 0.2250351617440225,
174
+ "grad_norm": 0.28814154863357544,
175
+ "learning_rate": 4.975027991024473e-05,
176
+ "loss": 0.6502,
177
+ "step": 120
178
+ },
179
+ {
180
+ "epoch": 0.23441162681669012,
181
+ "grad_norm": 0.28324735164642334,
182
+ "learning_rate": 4.972907481050637e-05,
183
+ "loss": 0.6162,
184
+ "step": 125
185
+ },
186
+ {
187
+ "epoch": 0.2437880918893577,
188
+ "grad_norm": 0.326524555683136,
189
+ "learning_rate": 4.970701059450872e-05,
190
+ "loss": 0.6333,
191
+ "step": 130
192
+ },
193
+ {
194
+ "epoch": 0.25316455696202533,
195
+ "grad_norm": 0.2594030797481537,
196
+ "learning_rate": 4.968408802878778e-05,
197
+ "loss": 0.6405,
198
+ "step": 135
199
+ },
200
+ {
201
+ "epoch": 0.26254102203469293,
202
+ "grad_norm": 0.2970665395259857,
203
+ "learning_rate": 4.9660307909699645e-05,
204
+ "loss": 0.5961,
205
+ "step": 140
206
+ },
207
+ {
208
+ "epoch": 0.2719174871073605,
209
+ "grad_norm": 0.274636447429657,
210
+ "learning_rate": 4.963567106339276e-05,
211
+ "loss": 0.5913,
212
+ "step": 145
213
+ },
214
+ {
215
+ "epoch": 0.2812939521800281,
216
+ "grad_norm": 0.30863094329833984,
217
+ "learning_rate": 4.961017834577927e-05,
218
+ "loss": 0.6354,
219
+ "step": 150
220
+ },
221
+ {
222
+ "epoch": 0.2906704172526957,
223
+ "grad_norm": 0.30860957503318787,
224
+ "learning_rate": 4.958383064250525e-05,
225
+ "loss": 0.5861,
226
+ "step": 155
227
+ },
228
+ {
229
+ "epoch": 0.30004688232536336,
230
+ "grad_norm": 0.31627291440963745,
231
+ "learning_rate": 4.955662886891995e-05,
232
+ "loss": 0.6287,
233
+ "step": 160
234
+ },
235
+ {
236
+ "epoch": 0.30942334739803096,
237
+ "grad_norm": 0.2863800525665283,
238
+ "learning_rate": 4.952857397004401e-05,
239
+ "loss": 0.648,
240
+ "step": 165
241
+ },
242
+ {
243
+ "epoch": 0.31879981247069855,
244
+ "grad_norm": 0.30252712965011597,
245
+ "learning_rate": 4.949966692053663e-05,
246
+ "loss": 0.6162,
247
+ "step": 170
248
+ },
249
+ {
250
+ "epoch": 0.32817627754336615,
251
+ "grad_norm": 0.24963121116161346,
252
+ "learning_rate": 4.946990872466164e-05,
253
+ "loss": 0.6344,
254
+ "step": 175
255
+ },
256
+ {
257
+ "epoch": 0.33755274261603374,
258
+ "grad_norm": 0.3113265335559845,
259
+ "learning_rate": 4.943930041625272e-05,
260
+ "loss": 0.6029,
261
+ "step": 180
262
+ },
263
+ {
264
+ "epoch": 0.34692920768870134,
265
+ "grad_norm": 0.28185218572616577,
266
+ "learning_rate": 4.940784305867741e-05,
267
+ "loss": 0.5946,
268
+ "step": 185
269
+ },
270
+ {
271
+ "epoch": 0.356305672761369,
272
+ "grad_norm": 0.3129664659500122,
273
+ "learning_rate": 4.937553774480018e-05,
274
+ "loss": 0.5829,
275
+ "step": 190
276
+ },
277
+ {
278
+ "epoch": 0.3656821378340366,
279
+ "grad_norm": 0.3760225474834442,
280
+ "learning_rate": 4.934238559694448e-05,
281
+ "loss": 0.659,
282
+ "step": 195
283
+ },
284
+ {
285
+ "epoch": 0.3750586029067042,
286
+ "grad_norm": 0.34029099345207214,
287
+ "learning_rate": 4.9308387766853725e-05,
288
+ "loss": 0.6483,
289
+ "step": 200
290
+ },
291
+ {
292
+ "epoch": 0.38443506797937177,
293
+ "grad_norm": 0.3152872920036316,
294
+ "learning_rate": 4.92735454356513e-05,
295
+ "loss": 0.611,
296
+ "step": 205
297
+ },
298
+ {
299
+ "epoch": 0.39381153305203936,
300
+ "grad_norm": 0.32286536693573,
301
+ "learning_rate": 4.9237859813799535e-05,
302
+ "loss": 0.6201,
303
+ "step": 210
304
+ },
305
+ {
306
+ "epoch": 0.40318799812470696,
307
+ "grad_norm": 0.3486417531967163,
308
+ "learning_rate": 4.9201332141057623e-05,
309
+ "loss": 0.5775,
310
+ "step": 215
311
+ },
312
+ {
313
+ "epoch": 0.4125644631973746,
314
+ "grad_norm": 0.33769914507865906,
315
+ "learning_rate": 4.9163963686438575e-05,
316
+ "loss": 0.6115,
317
+ "step": 220
318
+ },
319
+ {
320
+ "epoch": 0.4219409282700422,
321
+ "grad_norm": 0.31967395544052124,
322
+ "learning_rate": 4.912575574816511e-05,
323
+ "loss": 0.5879,
324
+ "step": 225
325
+ },
326
+ {
327
+ "epoch": 0.4313173933427098,
328
+ "grad_norm": 0.40602371096611023,
329
+ "learning_rate": 4.908670965362457e-05,
330
+ "loss": 0.5617,
331
+ "step": 230
332
+ },
333
+ {
334
+ "epoch": 0.4406938584153774,
335
+ "grad_norm": 0.31485116481781006,
336
+ "learning_rate": 4.9046826759322825e-05,
337
+ "loss": 0.6132,
338
+ "step": 235
339
+ },
340
+ {
341
+ "epoch": 0.450070323488045,
342
+ "grad_norm": 0.3902187645435333,
343
+ "learning_rate": 4.9006108450837095e-05,
344
+ "loss": 0.5923,
345
+ "step": 240
346
+ },
347
+ {
348
+ "epoch": 0.45944678856071264,
349
+ "grad_norm": 0.3169301152229309,
350
+ "learning_rate": 4.8964556142767845e-05,
351
+ "loss": 0.6131,
352
+ "step": 245
353
+ },
354
+ {
355
+ "epoch": 0.46882325363338023,
356
+ "grad_norm": 0.31579890847206116,
357
+ "learning_rate": 4.892217127868965e-05,
358
+ "loss": 0.6137,
359
+ "step": 250
360
+ },
361
+ {
362
+ "epoch": 0.4781997187060478,
363
+ "grad_norm": 0.3303227126598358,
364
+ "learning_rate": 4.8878955331101026e-05,
365
+ "loss": 0.6239,
366
+ "step": 255
367
+ },
368
+ {
369
+ "epoch": 0.4875761837787154,
370
+ "grad_norm": 0.33773812651634216,
371
+ "learning_rate": 4.8834909801373264e-05,
372
+ "loss": 0.5665,
373
+ "step": 260
374
+ },
375
+ {
376
+ "epoch": 0.496952648851383,
377
+ "grad_norm": 0.3913642466068268,
378
+ "learning_rate": 4.879003621969831e-05,
379
+ "loss": 0.5801,
380
+ "step": 265
381
+ },
382
+ {
383
+ "epoch": 0.5063291139240507,
384
+ "grad_norm": 0.33499622344970703,
385
+ "learning_rate": 4.874433614503554e-05,
386
+ "loss": 0.5682,
387
+ "step": 270
388
+ },
389
+ {
390
+ "epoch": 0.5157055789967182,
391
+ "grad_norm": 0.3137553930282593,
392
+ "learning_rate": 4.869781116505768e-05,
393
+ "loss": 0.5698,
394
+ "step": 275
395
+ },
396
+ {
397
+ "epoch": 0.5250820440693859,
398
+ "grad_norm": 0.3507164716720581,
399
+ "learning_rate": 4.8650462896095597e-05,
400
+ "loss": 0.5948,
401
+ "step": 280
402
+ },
403
+ {
404
+ "epoch": 0.5344585091420534,
405
+ "grad_norm": 0.33271023631095886,
406
+ "learning_rate": 4.860229298308213e-05,
407
+ "loss": 0.566,
408
+ "step": 285
409
+ },
410
+ {
411
+ "epoch": 0.543834974214721,
412
+ "grad_norm": 0.4269760251045227,
413
+ "learning_rate": 4.8553303099495e-05,
414
+ "loss": 0.5867,
415
+ "step": 290
416
+ },
417
+ {
418
+ "epoch": 0.5532114392873887,
419
+ "grad_norm": 0.37852942943573,
420
+ "learning_rate": 4.8503494947298634e-05,
421
+ "loss": 0.6045,
422
+ "step": 295
423
+ },
424
+ {
425
+ "epoch": 0.5625879043600562,
426
+ "grad_norm": 0.354078084230423,
427
+ "learning_rate": 4.845287025688503e-05,
428
+ "loss": 0.5963,
429
+ "step": 300
430
+ },
431
+ {
432
+ "epoch": 0.5719643694327239,
433
+ "grad_norm": 0.36660170555114746,
434
+ "learning_rate": 4.8401430787013666e-05,
435
+ "loss": 0.6019,
436
+ "step": 305
437
+ },
438
+ {
439
+ "epoch": 0.5813408345053914,
440
+ "grad_norm": 0.3099578320980072,
441
+ "learning_rate": 4.8349178324750387e-05,
442
+ "loss": 0.5811,
443
+ "step": 310
444
+ },
445
+ {
446
+ "epoch": 0.5907172995780591,
447
+ "grad_norm": 0.3528560996055603,
448
+ "learning_rate": 4.8296114685405324e-05,
449
+ "loss": 0.5702,
450
+ "step": 315
451
+ },
452
+ {
453
+ "epoch": 0.6000937646507267,
454
+ "grad_norm": 0.3307196795940399,
455
+ "learning_rate": 4.824224171246981e-05,
456
+ "loss": 0.6386,
457
+ "step": 320
458
+ },
459
+ {
460
+ "epoch": 0.6094702297233943,
461
+ "grad_norm": 0.3812682628631592,
462
+ "learning_rate": 4.8187561277552374e-05,
463
+ "loss": 0.5937,
464
+ "step": 325
465
+ },
466
+ {
467
+ "epoch": 0.6188466947960619,
468
+ "grad_norm": 0.35563403367996216,
469
+ "learning_rate": 4.813207528031366e-05,
470
+ "loss": 0.5537,
471
+ "step": 330
472
+ },
473
+ {
474
+ "epoch": 0.6282231598687295,
475
+ "grad_norm": 0.3535631597042084,
476
+ "learning_rate": 4.807578564840051e-05,
477
+ "loss": 0.6272,
478
+ "step": 335
479
+ },
480
+ {
481
+ "epoch": 0.6375996249413971,
482
+ "grad_norm": 0.3569740951061249,
483
+ "learning_rate": 4.801869433737891e-05,
484
+ "loss": 0.5732,
485
+ "step": 340
486
+ },
487
+ {
488
+ "epoch": 0.6469760900140648,
489
+ "grad_norm": 0.37414535880088806,
490
+ "learning_rate": 4.796080333066613e-05,
491
+ "loss": 0.6062,
492
+ "step": 345
493
+ },
494
+ {
495
+ "epoch": 0.6563525550867323,
496
+ "grad_norm": 0.38426753878593445,
497
+ "learning_rate": 4.790211463946174e-05,
498
+ "loss": 0.6212,
499
+ "step": 350
500
+ },
501
+ {
502
+ "epoch": 0.6657290201593999,
503
+ "grad_norm": 0.46834492683410645,
504
+ "learning_rate": 4.784263030267781e-05,
505
+ "loss": 0.5662,
506
+ "step": 355
507
+ },
508
+ {
509
+ "epoch": 0.6751054852320675,
510
+ "grad_norm": 0.3643794655799866,
511
+ "learning_rate": 4.7782352386868035e-05,
512
+ "loss": 0.6173,
513
+ "step": 360
514
+ },
515
+ {
516
+ "epoch": 0.6844819503047351,
517
+ "grad_norm": 0.36537444591522217,
518
+ "learning_rate": 4.7721282986155945e-05,
519
+ "loss": 0.5543,
520
+ "step": 365
521
+ },
522
+ {
523
+ "epoch": 0.6938584153774027,
524
+ "grad_norm": 0.40244394540786743,
525
+ "learning_rate": 4.7659424222162165e-05,
526
+ "loss": 0.5608,
527
+ "step": 370
528
+ },
529
+ {
530
+ "epoch": 0.7032348804500703,
531
+ "grad_norm": 0.376693457365036,
532
+ "learning_rate": 4.7596778243930694e-05,
533
+ "loss": 0.6113,
534
+ "step": 375
535
+ },
536
+ {
537
+ "epoch": 0.712611345522738,
538
+ "grad_norm": 0.3722762167453766,
539
+ "learning_rate": 4.7533347227854265e-05,
540
+ "loss": 0.604,
541
+ "step": 380
542
+ },
543
+ {
544
+ "epoch": 0.7219878105954055,
545
+ "grad_norm": 0.40300458669662476,
546
+ "learning_rate": 4.7469133377598695e-05,
547
+ "loss": 0.5725,
548
+ "step": 385
549
+ },
550
+ {
551
+ "epoch": 0.7313642756680732,
552
+ "grad_norm": 0.37405309081077576,
553
+ "learning_rate": 4.740413892402639e-05,
554
+ "loss": 0.5792,
555
+ "step": 390
556
+ },
557
+ {
558
+ "epoch": 0.7407407407407407,
559
+ "grad_norm": 0.39936429262161255,
560
+ "learning_rate": 4.7338366125118775e-05,
561
+ "loss": 0.601,
562
+ "step": 395
563
+ },
564
+ {
565
+ "epoch": 0.7501172058134083,
566
+ "grad_norm": 0.398360013961792,
567
+ "learning_rate": 4.727181726589789e-05,
568
+ "loss": 0.6168,
569
+ "step": 400
570
+ },
571
+ {
572
+ "epoch": 0.759493670886076,
573
+ "grad_norm": 0.3800843060016632,
574
+ "learning_rate": 4.7204494658346996e-05,
575
+ "loss": 0.5719,
576
+ "step": 405
577
+ },
578
+ {
579
+ "epoch": 0.7688701359587435,
580
+ "grad_norm": 0.3811691105365753,
581
+ "learning_rate": 4.713640064133025e-05,
582
+ "loss": 0.5798,
583
+ "step": 410
584
+ },
585
+ {
586
+ "epoch": 0.7782466010314112,
587
+ "grad_norm": 0.3757120966911316,
588
+ "learning_rate": 4.706753758051145e-05,
589
+ "loss": 0.5852,
590
+ "step": 415
591
+ },
592
+ {
593
+ "epoch": 0.7876230661040787,
594
+ "grad_norm": 0.4053094983100891,
595
+ "learning_rate": 4.699790786827188e-05,
596
+ "loss": 0.6183,
597
+ "step": 420
598
+ },
599
+ {
600
+ "epoch": 0.7969995311767464,
601
+ "grad_norm": 0.4359457492828369,
602
+ "learning_rate": 4.6927513923627124e-05,
603
+ "loss": 0.59,
604
+ "step": 425
605
+ },
606
+ {
607
+ "epoch": 0.8063759962494139,
608
+ "grad_norm": 0.42008543014526367,
609
+ "learning_rate": 4.68563581921431e-05,
610
+ "loss": 0.5661,
611
+ "step": 430
612
+ },
613
+ {
614
+ "epoch": 0.8157524613220816,
615
+ "grad_norm": 0.43090564012527466,
616
+ "learning_rate": 4.6784443145851074e-05,
617
+ "loss": 0.5222,
618
+ "step": 435
619
+ },
620
+ {
621
+ "epoch": 0.8251289263947492,
622
+ "grad_norm": 0.41035380959510803,
623
+ "learning_rate": 4.671177128316176e-05,
624
+ "loss": 0.5407,
625
+ "step": 440
626
+ },
627
+ {
628
+ "epoch": 0.8345053914674168,
629
+ "grad_norm": 0.38988974690437317,
630
+ "learning_rate": 4.663834512877853e-05,
631
+ "loss": 0.5922,
632
+ "step": 445
633
+ },
634
+ {
635
+ "epoch": 0.8438818565400844,
636
+ "grad_norm": 0.3898136019706726,
637
+ "learning_rate": 4.6564167233609736e-05,
638
+ "loss": 0.5714,
639
+ "step": 450
640
+ },
641
+ {
642
+ "epoch": 0.853258321612752,
643
+ "grad_norm": 0.3754185736179352,
644
+ "learning_rate": 4.648924017468003e-05,
645
+ "loss": 0.5588,
646
+ "step": 455
647
+ },
648
+ {
649
+ "epoch": 0.8626347866854196,
650
+ "grad_norm": 0.42259371280670166,
651
+ "learning_rate": 4.6413566555040896e-05,
652
+ "loss": 0.5935,
653
+ "step": 460
654
+ },
655
+ {
656
+ "epoch": 0.8720112517580872,
657
+ "grad_norm": 0.3977474272251129,
658
+ "learning_rate": 4.633714900368018e-05,
659
+ "loss": 0.5626,
660
+ "step": 465
661
+ },
662
+ {
663
+ "epoch": 0.8813877168307548,
664
+ "grad_norm": 0.3957667052745819,
665
+ "learning_rate": 4.625999017543075e-05,
666
+ "loss": 0.5625,
667
+ "step": 470
668
+ },
669
+ {
670
+ "epoch": 0.8907641819034224,
671
+ "grad_norm": 0.4148056209087372,
672
+ "learning_rate": 4.618209275087829e-05,
673
+ "loss": 0.586,
674
+ "step": 475
675
+ },
676
+ {
677
+ "epoch": 0.90014064697609,
678
+ "grad_norm": 0.4572969079017639,
679
+ "learning_rate": 4.610345943626817e-05,
680
+ "loss": 0.5806,
681
+ "step": 480
682
+ },
683
+ {
684
+ "epoch": 0.9095171120487576,
685
+ "grad_norm": 0.37934327125549316,
686
+ "learning_rate": 4.602409296341141e-05,
687
+ "loss": 0.5682,
688
+ "step": 485
689
+ },
690
+ {
691
+ "epoch": 0.9188935771214253,
692
+ "grad_norm": 0.4195612370967865,
693
+ "learning_rate": 4.5943996089589775e-05,
694
+ "loss": 0.5594,
695
+ "step": 490
696
+ },
697
+ {
698
+ "epoch": 0.9282700421940928,
699
+ "grad_norm": 0.42826300859451294,
700
+ "learning_rate": 4.586317159746001e-05,
701
+ "loss": 0.6643,
702
+ "step": 495
703
+ },
704
+ {
705
+ "epoch": 0.9376465072667605,
706
+ "grad_norm": 0.4025145471096039,
707
+ "learning_rate": 4.5781622294957136e-05,
708
+ "loss": 0.5672,
709
+ "step": 500
710
+ },
711
+ {
712
+ "epoch": 0.947022972339428,
713
+ "grad_norm": 0.37648695707321167,
714
+ "learning_rate": 4.569935101519692e-05,
715
+ "loss": 0.5906,
716
+ "step": 505
717
+ },
718
+ {
719
+ "epoch": 0.9563994374120957,
720
+ "grad_norm": 0.39290428161621094,
721
+ "learning_rate": 4.561636061637745e-05,
722
+ "loss": 0.6222,
723
+ "step": 510
724
+ },
725
+ {
726
+ "epoch": 0.9657759024847632,
727
+ "grad_norm": 0.36130473017692566,
728
+ "learning_rate": 4.553265398167981e-05,
729
+ "loss": 0.5787,
730
+ "step": 515
731
+ },
732
+ {
733
+ "epoch": 0.9751523675574308,
734
+ "grad_norm": 0.409842312335968,
735
+ "learning_rate": 4.5448234019167945e-05,
736
+ "loss": 0.5627,
737
+ "step": 520
738
+ },
739
+ {
740
+ "epoch": 0.9845288326300985,
741
+ "grad_norm": 0.40280935168266296,
742
+ "learning_rate": 4.536310366168763e-05,
743
+ "loss": 0.5704,
744
+ "step": 525
745
+ },
746
+ {
747
+ "epoch": 0.993905297702766,
748
+ "grad_norm": 0.3929194509983063,
749
+ "learning_rate": 4.5277265866764565e-05,
750
+ "loss": 0.5879,
751
+ "step": 530
752
+ },
753
+ {
754
+ "epoch": 1.0032817627754336,
755
+ "grad_norm": 0.5179246664047241,
756
+ "learning_rate": 4.519072361650163e-05,
757
+ "loss": 0.5509,
758
+ "step": 535
759
+ },
760
+ {
761
+ "epoch": 1.0126582278481013,
762
+ "grad_norm": 0.433220773935318,
763
+ "learning_rate": 4.5103479917475286e-05,
764
+ "loss": 0.5293,
765
+ "step": 540
766
+ },
767
+ {
768
+ "epoch": 1.0220346929207689,
769
+ "grad_norm": 0.47541534900665283,
770
+ "learning_rate": 4.501553780063113e-05,
771
+ "loss": 0.5393,
772
+ "step": 545
773
+ },
774
+ {
775
+ "epoch": 1.0314111579934364,
776
+ "grad_norm": 0.3974189758300781,
777
+ "learning_rate": 4.4926900321178595e-05,
778
+ "loss": 0.5949,
779
+ "step": 550
780
+ },
781
+ {
782
+ "epoch": 1.0407876230661042,
783
+ "grad_norm": 0.41597846150398254,
784
+ "learning_rate": 4.483757055848479e-05,
785
+ "loss": 0.5715,
786
+ "step": 555
787
+ },
788
+ {
789
+ "epoch": 1.0501640881387717,
790
+ "grad_norm": 0.42423322796821594,
791
+ "learning_rate": 4.4747551615967534e-05,
792
+ "loss": 0.5321,
793
+ "step": 560
794
+ },
795
+ {
796
+ "epoch": 1.0595405532114393,
797
+ "grad_norm": 0.4616020917892456,
798
+ "learning_rate": 4.4656846620987557e-05,
799
+ "loss": 0.591,
800
+ "step": 565
801
+ },
802
+ {
803
+ "epoch": 1.0689170182841068,
804
+ "grad_norm": 0.39147764444351196,
805
+ "learning_rate": 4.4565458724739825e-05,
806
+ "loss": 0.5834,
807
+ "step": 570
808
+ },
809
+ {
810
+ "epoch": 1.0782934833567746,
811
+ "grad_norm": 0.4559531509876251,
812
+ "learning_rate": 4.447339110214405e-05,
813
+ "loss": 0.5609,
814
+ "step": 575
815
+ },
816
+ {
817
+ "epoch": 1.087669948429442,
818
+ "grad_norm": 0.4997139871120453,
819
+ "learning_rate": 4.438064695173446e-05,
820
+ "loss": 0.5792,
821
+ "step": 580
822
+ },
823
+ {
824
+ "epoch": 1.0970464135021096,
825
+ "grad_norm": 0.45987358689308167,
826
+ "learning_rate": 4.428722949554857e-05,
827
+ "loss": 0.568,
828
+ "step": 585
829
+ },
830
+ {
831
+ "epoch": 1.1064228785747774,
832
+ "grad_norm": 0.3987531065940857,
833
+ "learning_rate": 4.419314197901537e-05,
834
+ "loss": 0.5428,
835
+ "step": 590
836
+ },
837
+ {
838
+ "epoch": 1.115799343647445,
839
+ "grad_norm": 0.40742579102516174,
840
+ "learning_rate": 4.4098387670842466e-05,
841
+ "loss": 0.564,
842
+ "step": 595
843
+ },
844
+ {
845
+ "epoch": 1.1251758087201125,
846
+ "grad_norm": 0.41294044256210327,
847
+ "learning_rate": 4.400296986290258e-05,
848
+ "loss": 0.5625,
849
+ "step": 600
850
+ },
851
+ {
852
+ "epoch": 1.1345522737927802,
853
+ "grad_norm": 0.46300792694091797,
854
+ "learning_rate": 4.390689187011917e-05,
855
+ "loss": 0.5716,
856
+ "step": 605
857
+ },
858
+ {
859
+ "epoch": 1.1439287388654478,
860
+ "grad_norm": 0.45359134674072266,
861
+ "learning_rate": 4.3810157030351276e-05,
862
+ "loss": 0.5394,
863
+ "step": 610
864
+ },
865
+ {
866
+ "epoch": 1.1533052039381153,
867
+ "grad_norm": 0.46484261751174927,
868
+ "learning_rate": 4.371276870427753e-05,
869
+ "loss": 0.5344,
870
+ "step": 615
871
+ },
872
+ {
873
+ "epoch": 1.1626816690107828,
874
+ "grad_norm": 0.42876294255256653,
875
+ "learning_rate": 4.3614730275279457e-05,
876
+ "loss": 0.544,
877
+ "step": 620
878
+ },
879
+ {
880
+ "epoch": 1.1720581340834506,
881
+ "grad_norm": 0.5274311900138855,
882
+ "learning_rate": 4.351604514932387e-05,
883
+ "loss": 0.5481,
884
+ "step": 625
885
+ },
886
+ {
887
+ "epoch": 1.1814345991561181,
888
+ "grad_norm": 0.5408082604408264,
889
+ "learning_rate": 4.341671675484459e-05,
890
+ "loss": 0.5715,
891
+ "step": 630
892
+ },
893
+ {
894
+ "epoch": 1.1908110642287857,
895
+ "grad_norm": 0.41055697202682495,
896
+ "learning_rate": 4.331674854262331e-05,
897
+ "loss": 0.5602,
898
+ "step": 635
899
+ },
900
+ {
901
+ "epoch": 1.2001875293014534,
902
+ "grad_norm": 0.42665359377861023,
903
+ "learning_rate": 4.321614398566972e-05,
904
+ "loss": 0.5441,
905
+ "step": 640
906
+ },
907
+ {
908
+ "epoch": 1.209563994374121,
909
+ "grad_norm": 0.4585810601711273,
910
+ "learning_rate": 4.3114906579100853e-05,
911
+ "loss": 0.5369,
912
+ "step": 645
913
+ },
914
+ {
915
+ "epoch": 1.2189404594467885,
916
+ "grad_norm": 0.4821290969848633,
917
+ "learning_rate": 4.301303984001967e-05,
918
+ "loss": 0.577,
919
+ "step": 650
920
+ },
921
+ {
922
+ "epoch": 1.228316924519456,
923
+ "grad_norm": 0.44844305515289307,
924
+ "learning_rate": 4.291054730739286e-05,
925
+ "loss": 0.5762,
926
+ "step": 655
927
+ },
928
+ {
929
+ "epoch": 1.2376933895921238,
930
+ "grad_norm": 0.47802606225013733,
931
+ "learning_rate": 4.2807432541927865e-05,
932
+ "loss": 0.5726,
933
+ "step": 660
934
+ },
935
+ {
936
+ "epoch": 1.2470698546647914,
937
+ "grad_norm": 0.5414724946022034,
938
+ "learning_rate": 4.2703699125949245e-05,
939
+ "loss": 0.5686,
940
+ "step": 665
941
+ },
942
+ {
943
+ "epoch": 1.256446319737459,
944
+ "grad_norm": 0.4859514832496643,
945
+ "learning_rate": 4.259935066327415e-05,
946
+ "loss": 0.6473,
947
+ "step": 670
948
+ },
949
+ {
950
+ "epoch": 1.2658227848101267,
951
+ "grad_norm": 0.4125962555408478,
952
+ "learning_rate": 4.2494390779087187e-05,
953
+ "loss": 0.6009,
954
+ "step": 675
955
+ },
956
+ {
957
+ "epoch": 1.2751992498827942,
958
+ "grad_norm": 0.45781227946281433,
959
+ "learning_rate": 4.238882311981441e-05,
960
+ "loss": 0.5301,
961
+ "step": 680
962
+ },
963
+ {
964
+ "epoch": 1.2845757149554617,
965
+ "grad_norm": 0.5371187925338745,
966
+ "learning_rate": 4.228265135299669e-05,
967
+ "loss": 0.5715,
968
+ "step": 685
969
+ },
970
+ {
971
+ "epoch": 1.2939521800281293,
972
+ "grad_norm": 0.42323383688926697,
973
+ "learning_rate": 4.2175879167162304e-05,
974
+ "loss": 0.6022,
975
+ "step": 690
976
+ },
977
+ {
978
+ "epoch": 1.303328645100797,
979
+ "grad_norm": 0.5018748641014099,
980
+ "learning_rate": 4.206851027169871e-05,
981
+ "loss": 0.5765,
982
+ "step": 695
983
+ },
984
+ {
985
+ "epoch": 1.3127051101734646,
986
+ "grad_norm": 0.4469924569129944,
987
+ "learning_rate": 4.196054839672382e-05,
988
+ "loss": 0.5521,
989
+ "step": 700
990
+ },
991
+ {
992
+ "epoch": 1.3220815752461323,
993
+ "grad_norm": 0.3689207136631012,
994
+ "learning_rate": 4.1851997292956255e-05,
995
+ "loss": 0.5709,
996
+ "step": 705
997
+ },
998
+ {
999
+ "epoch": 1.3314580403187999,
1000
+ "grad_norm": 0.46595776081085205,
1001
+ "learning_rate": 4.174286073158516e-05,
1002
+ "loss": 0.5751,
1003
+ "step": 710
1004
+ },
1005
+ {
1006
+ "epoch": 1.3408345053914674,
1007
+ "grad_norm": 0.430705189704895,
1008
+ "learning_rate": 4.163314250413913e-05,
1009
+ "loss": 0.5276,
1010
+ "step": 715
1011
+ },
1012
+ {
1013
+ "epoch": 1.350210970464135,
1014
+ "grad_norm": 0.45362386107444763,
1015
+ "learning_rate": 4.152284642235452e-05,
1016
+ "loss": 0.5654,
1017
+ "step": 720
1018
+ },
1019
+ {
1020
+ "epoch": 1.3595874355368025,
1021
+ "grad_norm": 0.4694269895553589,
1022
+ "learning_rate": 4.141197631804298e-05,
1023
+ "loss": 0.5771,
1024
+ "step": 725
1025
+ },
1026
+ {
1027
+ "epoch": 1.3689639006094703,
1028
+ "grad_norm": 0.4499427080154419,
1029
+ "learning_rate": 4.1300536042958354e-05,
1030
+ "loss": 0.5849,
1031
+ "step": 730
1032
+ },
1033
+ {
1034
+ "epoch": 1.3783403656821378,
1035
+ "grad_norm": 0.4603768587112427,
1036
+ "learning_rate": 4.118852946866291e-05,
1037
+ "loss": 0.5539,
1038
+ "step": 735
1039
+ },
1040
+ {
1041
+ "epoch": 1.3877168307548056,
1042
+ "grad_norm": 0.5671654343605042,
1043
+ "learning_rate": 4.107596048639274e-05,
1044
+ "loss": 0.5408,
1045
+ "step": 740
1046
+ },
1047
+ {
1048
+ "epoch": 1.397093295827473,
1049
+ "grad_norm": 0.4729969799518585,
1050
+ "learning_rate": 4.0962833006922675e-05,
1051
+ "loss": 0.5717,
1052
+ "step": 745
1053
+ },
1054
+ {
1055
+ "epoch": 1.4064697609001406,
1056
+ "grad_norm": 0.4194209575653076,
1057
+ "learning_rate": 4.0849150960430356e-05,
1058
+ "loss": 0.5612,
1059
+ "step": 750
1060
+ },
1061
+ {
1062
+ "epoch": 1.4158462259728082,
1063
+ "grad_norm": 0.43975749611854553,
1064
+ "learning_rate": 4.0734918296359716e-05,
1065
+ "loss": 0.5587,
1066
+ "step": 755
1067
+ },
1068
+ {
1069
+ "epoch": 1.4252226910454757,
1070
+ "grad_norm": 0.3893047273159027,
1071
+ "learning_rate": 4.0620138983283785e-05,
1072
+ "loss": 0.5965,
1073
+ "step": 760
1074
+ },
1075
+ {
1076
+ "epoch": 1.4345991561181435,
1077
+ "grad_norm": 0.4223731458187103,
1078
+ "learning_rate": 4.050481700876677e-05,
1079
+ "loss": 0.5576,
1080
+ "step": 765
1081
+ },
1082
+ {
1083
+ "epoch": 1.443975621190811,
1084
+ "grad_norm": 0.504723310470581,
1085
+ "learning_rate": 4.038895637922559e-05,
1086
+ "loss": 0.5481,
1087
+ "step": 770
1088
+ },
1089
+ {
1090
+ "epoch": 1.4533520862634788,
1091
+ "grad_norm": 0.4618123173713684,
1092
+ "learning_rate": 4.027256111979063e-05,
1093
+ "loss": 0.5343,
1094
+ "step": 775
1095
+ },
1096
+ {
1097
+ "epoch": 1.4627285513361463,
1098
+ "grad_norm": 0.4357120990753174,
1099
+ "learning_rate": 4.015563527416595e-05,
1100
+ "loss": 0.5677,
1101
+ "step": 780
1102
+ },
1103
+ {
1104
+ "epoch": 1.4721050164088139,
1105
+ "grad_norm": 0.4497866928577423,
1106
+ "learning_rate": 4.003818290448876e-05,
1107
+ "loss": 0.565,
1108
+ "step": 785
1109
+ },
1110
+ {
1111
+ "epoch": 1.4814814814814814,
1112
+ "grad_norm": 0.4233000874519348,
1113
+ "learning_rate": 3.992020809118832e-05,
1114
+ "loss": 0.6299,
1115
+ "step": 790
1116
+ },
1117
+ {
1118
+ "epoch": 1.4908579465541492,
1119
+ "grad_norm": 0.522912859916687,
1120
+ "learning_rate": 3.980171493284418e-05,
1121
+ "loss": 0.5588,
1122
+ "step": 795
1123
+ },
1124
+ {
1125
+ "epoch": 1.5002344116268167,
1126
+ "grad_norm": 0.5034630298614502,
1127
+ "learning_rate": 3.9682707546043785e-05,
1128
+ "loss": 0.6042,
1129
+ "step": 800
1130
+ },
1131
+ {
1132
+ "epoch": 1.5096108766994845,
1133
+ "grad_norm": 0.47972697019577026,
1134
+ "learning_rate": 3.9563190065239474e-05,
1135
+ "loss": 0.5613,
1136
+ "step": 805
1137
+ },
1138
+ {
1139
+ "epoch": 1.518987341772152,
1140
+ "grad_norm": 0.4823983311653137,
1141
+ "learning_rate": 3.9443166642604814e-05,
1142
+ "loss": 0.5466,
1143
+ "step": 810
1144
+ },
1145
+ {
1146
+ "epoch": 1.5283638068448195,
1147
+ "grad_norm": 0.4230092465877533,
1148
+ "learning_rate": 3.932264144789038e-05,
1149
+ "loss": 0.5305,
1150
+ "step": 815
1151
+ },
1152
+ {
1153
+ "epoch": 1.537740271917487,
1154
+ "grad_norm": 0.42191463708877563,
1155
+ "learning_rate": 3.920161866827889e-05,
1156
+ "loss": 0.5822,
1157
+ "step": 820
1158
+ },
1159
+ {
1160
+ "epoch": 1.5471167369901546,
1161
+ "grad_norm": 0.4600556194782257,
1162
+ "learning_rate": 3.908010250823972e-05,
1163
+ "loss": 0.5585,
1164
+ "step": 825
1165
+ },
1166
+ {
1167
+ "epoch": 1.5564932020628222,
1168
+ "grad_norm": 0.5136891603469849,
1169
+ "learning_rate": 3.895809718938283e-05,
1170
+ "loss": 0.5693,
1171
+ "step": 830
1172
+ },
1173
+ {
1174
+ "epoch": 1.56586966713549,
1175
+ "grad_norm": 0.4428097903728485,
1176
+ "learning_rate": 3.883560695031213e-05,
1177
+ "loss": 0.5137,
1178
+ "step": 835
1179
+ },
1180
+ {
1181
+ "epoch": 1.5752461322081577,
1182
+ "grad_norm": 0.4338129758834839,
1183
+ "learning_rate": 3.871263604647822e-05,
1184
+ "loss": 0.5622,
1185
+ "step": 840
1186
+ },
1187
+ {
1188
+ "epoch": 1.5846225972808252,
1189
+ "grad_norm": 0.4927155077457428,
1190
+ "learning_rate": 3.858918875003053e-05,
1191
+ "loss": 0.5321,
1192
+ "step": 845
1193
+ },
1194
+ {
1195
+ "epoch": 1.5939990623534928,
1196
+ "grad_norm": 0.44332149624824524,
1197
+ "learning_rate": 3.846526934966891e-05,
1198
+ "loss": 0.5623,
1199
+ "step": 850
1200
+ },
1201
+ {
1202
+ "epoch": 1.6033755274261603,
1203
+ "grad_norm": 0.5348272919654846,
1204
+ "learning_rate": 3.834088215049464e-05,
1205
+ "loss": 0.5781,
1206
+ "step": 855
1207
+ },
1208
+ {
1209
+ "epoch": 1.6127519924988278,
1210
+ "grad_norm": 0.4220760464668274,
1211
+ "learning_rate": 3.821603147386088e-05,
1212
+ "loss": 0.5724,
1213
+ "step": 860
1214
+ },
1215
+ {
1216
+ "epoch": 1.6221284575714956,
1217
+ "grad_norm": 0.47724246978759766,
1218
+ "learning_rate": 3.80907216572225e-05,
1219
+ "loss": 0.5351,
1220
+ "step": 865
1221
+ },
1222
+ {
1223
+ "epoch": 1.6315049226441631,
1224
+ "grad_norm": 0.5851244926452637,
1225
+ "learning_rate": 3.796495705398544e-05,
1226
+ "loss": 0.5726,
1227
+ "step": 870
1228
+ },
1229
+ {
1230
+ "epoch": 1.640881387716831,
1231
+ "grad_norm": 0.45955443382263184,
1232
+ "learning_rate": 3.783874203335542e-05,
1233
+ "loss": 0.5326,
1234
+ "step": 875
1235
+ },
1236
+ {
1237
+ "epoch": 1.6502578527894984,
1238
+ "grad_norm": 0.528683066368103,
1239
+ "learning_rate": 3.77120809801862e-05,
1240
+ "loss": 0.5384,
1241
+ "step": 880
1242
+ },
1243
+ {
1244
+ "epoch": 1.659634317862166,
1245
+ "grad_norm": 0.42244866490364075,
1246
+ "learning_rate": 3.758497829482721e-05,
1247
+ "loss": 0.5808,
1248
+ "step": 885
1249
+ },
1250
+ {
1251
+ "epoch": 1.6690107829348335,
1252
+ "grad_norm": 0.4697413742542267,
1253
+ "learning_rate": 3.7457438392970686e-05,
1254
+ "loss": 0.5123,
1255
+ "step": 890
1256
+ },
1257
+ {
1258
+ "epoch": 1.678387248007501,
1259
+ "grad_norm": 0.47603949904441833,
1260
+ "learning_rate": 3.732946570549825e-05,
1261
+ "loss": 0.5437,
1262
+ "step": 895
1263
+ },
1264
+ {
1265
+ "epoch": 1.6877637130801688,
1266
+ "grad_norm": 0.5042173862457275,
1267
+ "learning_rate": 3.720106467832701e-05,
1268
+ "loss": 0.6003,
1269
+ "step": 900
1270
+ },
1271
+ {
1272
+ "epoch": 1.6971401781528364,
1273
+ "grad_norm": 0.4678844213485718,
1274
+ "learning_rate": 3.707223977225507e-05,
1275
+ "loss": 0.5941,
1276
+ "step": 905
1277
+ },
1278
+ {
1279
+ "epoch": 1.7065166432255041,
1280
+ "grad_norm": 0.4053237736225128,
1281
+ "learning_rate": 3.694299546280657e-05,
1282
+ "loss": 0.6019,
1283
+ "step": 910
1284
+ },
1285
+ {
1286
+ "epoch": 1.7158931082981717,
1287
+ "grad_norm": 0.49923238158226013,
1288
+ "learning_rate": 3.681333624007623e-05,
1289
+ "loss": 0.5774,
1290
+ "step": 915
1291
+ },
1292
+ {
1293
+ "epoch": 1.7252695733708392,
1294
+ "grad_norm": 0.5050910711288452,
1295
+ "learning_rate": 3.6683266608573286e-05,
1296
+ "loss": 0.6598,
1297
+ "step": 920
1298
+ },
1299
+ {
1300
+ "epoch": 1.7346460384435067,
1301
+ "grad_norm": 0.4897356331348419,
1302
+ "learning_rate": 3.6552791087065075e-05,
1303
+ "loss": 0.5536,
1304
+ "step": 925
1305
+ },
1306
+ {
1307
+ "epoch": 1.7440225035161743,
1308
+ "grad_norm": 0.4682961702346802,
1309
+ "learning_rate": 3.642191420842e-05,
1310
+ "loss": 0.5606,
1311
+ "step": 930
1312
+ },
1313
+ {
1314
+ "epoch": 1.753398968588842,
1315
+ "grad_norm": 0.46142104268074036,
1316
+ "learning_rate": 3.6290640519450074e-05,
1317
+ "loss": 0.5553,
1318
+ "step": 935
1319
+ },
1320
+ {
1321
+ "epoch": 1.7627754336615096,
1322
+ "grad_norm": 0.4685068726539612,
1323
+ "learning_rate": 3.6158974580752954e-05,
1324
+ "loss": 0.5974,
1325
+ "step": 940
1326
+ },
1327
+ {
1328
+ "epoch": 1.7721518987341773,
1329
+ "grad_norm": 0.4957576394081116,
1330
+ "learning_rate": 3.60269209665535e-05,
1331
+ "loss": 0.5381,
1332
+ "step": 945
1333
+ },
1334
+ {
1335
+ "epoch": 1.7815283638068449,
1336
+ "grad_norm": 0.5038308501243591,
1337
+ "learning_rate": 3.589448426454486e-05,
1338
+ "loss": 0.5858,
1339
+ "step": 950
1340
+ },
1341
+ {
1342
+ "epoch": 1.7909048288795124,
1343
+ "grad_norm": 0.5150238871574402,
1344
+ "learning_rate": 3.5761669075729084e-05,
1345
+ "loss": 0.552,
1346
+ "step": 955
1347
+ },
1348
+ {
1349
+ "epoch": 1.80028129395218,
1350
+ "grad_norm": 0.4681142568588257,
1351
+ "learning_rate": 3.562848001425729e-05,
1352
+ "loss": 0.6042,
1353
+ "step": 960
1354
+ },
1355
+ {
1356
+ "epoch": 1.8096577590248475,
1357
+ "grad_norm": 0.4891737401485443,
1358
+ "learning_rate": 3.549492170726937e-05,
1359
+ "loss": 0.5418,
1360
+ "step": 965
1361
+ },
1362
+ {
1363
+ "epoch": 1.8190342240975153,
1364
+ "grad_norm": 0.523688554763794,
1365
+ "learning_rate": 3.53609987947332e-05,
1366
+ "loss": 0.5743,
1367
+ "step": 970
1368
+ },
1369
+ {
1370
+ "epoch": 1.8284106891701828,
1371
+ "grad_norm": 0.46937504410743713,
1372
+ "learning_rate": 3.5226715929283506e-05,
1373
+ "loss": 0.5245,
1374
+ "step": 975
1375
+ },
1376
+ {
1377
+ "epoch": 1.8377871542428506,
1378
+ "grad_norm": 0.4652925729751587,
1379
+ "learning_rate": 3.509207777606013e-05,
1380
+ "loss": 0.5232,
1381
+ "step": 980
1382
+ },
1383
+ {
1384
+ "epoch": 1.847163619315518,
1385
+ "grad_norm": 0.5381861925125122,
1386
+ "learning_rate": 3.49570890125461e-05,
1387
+ "loss": 0.5354,
1388
+ "step": 985
1389
+ },
1390
+ {
1391
+ "epoch": 1.8565400843881856,
1392
+ "grad_norm": 0.47162288427352905,
1393
+ "learning_rate": 3.482175432840495e-05,
1394
+ "loss": 0.5296,
1395
+ "step": 990
1396
+ },
1397
+ {
1398
+ "epoch": 1.8659165494608532,
1399
+ "grad_norm": 0.5244145393371582,
1400
+ "learning_rate": 3.468607842531797e-05,
1401
+ "loss": 0.5486,
1402
+ "step": 995
1403
+ },
1404
+ {
1405
+ "epoch": 1.8752930145335207,
1406
+ "grad_norm": 0.5489354729652405,
1407
+ "learning_rate": 3.455006601682075e-05,
1408
+ "loss": 0.5339,
1409
+ "step": 1000
1410
+ },
1411
+ {
1412
+ "epoch": 1.8846694796061885,
1413
+ "grad_norm": 0.48319703340530396,
1414
+ "learning_rate": 3.441372182813946e-05,
1415
+ "loss": 0.5105,
1416
+ "step": 1005
1417
+ },
1418
+ {
1419
+ "epoch": 1.8940459446788562,
1420
+ "grad_norm": 0.4515131413936615,
1421
+ "learning_rate": 3.427705059602671e-05,
1422
+ "loss": 0.5353,
1423
+ "step": 1010
1424
+ },
1425
+ {
1426
+ "epoch": 1.9034224097515238,
1427
+ "grad_norm": 0.49319255352020264,
1428
+ "learning_rate": 3.414005706859693e-05,
1429
+ "loss": 0.574,
1430
+ "step": 1015
1431
+ },
1432
+ {
1433
+ "epoch": 1.9127988748241913,
1434
+ "grad_norm": 0.47591525316238403,
1435
+ "learning_rate": 3.400274600516152e-05,
1436
+ "loss": 0.558,
1437
+ "step": 1020
1438
+ },
1439
+ {
1440
+ "epoch": 1.9221753398968588,
1441
+ "grad_norm": 0.4464595913887024,
1442
+ "learning_rate": 3.386512217606339e-05,
1443
+ "loss": 0.5312,
1444
+ "step": 1025
1445
+ },
1446
+ {
1447
+ "epoch": 1.9315518049695264,
1448
+ "grad_norm": 0.5216926336288452,
1449
+ "learning_rate": 3.372719036251132e-05,
1450
+ "loss": 0.5401,
1451
+ "step": 1030
1452
+ },
1453
+ {
1454
+ "epoch": 1.9409282700421941,
1455
+ "grad_norm": 0.44332537055015564,
1456
+ "learning_rate": 3.3588955356413795e-05,
1457
+ "loss": 0.5468,
1458
+ "step": 1035
1459
+ },
1460
+ {
1461
+ "epoch": 1.9503047351148617,
1462
+ "grad_norm": 0.5241976976394653,
1463
+ "learning_rate": 3.3450421960212566e-05,
1464
+ "loss": 0.5533,
1465
+ "step": 1040
1466
+ },
1467
+ {
1468
+ "epoch": 1.9596812001875294,
1469
+ "grad_norm": 0.5050984621047974,
1470
+ "learning_rate": 3.3311594986715814e-05,
1471
+ "loss": 0.5615,
1472
+ "step": 1045
1473
+ },
1474
+ {
1475
+ "epoch": 1.969057665260197,
1476
+ "grad_norm": 0.5221999883651733,
1477
+ "learning_rate": 3.317247925893089e-05,
1478
+ "loss": 0.5201,
1479
+ "step": 1050
1480
+ },
1481
+ {
1482
+ "epoch": 1.9784341303328645,
1483
+ "grad_norm": 0.4861835241317749,
1484
+ "learning_rate": 3.3033079609896834e-05,
1485
+ "loss": 0.5404,
1486
+ "step": 1055
1487
+ },
1488
+ {
1489
+ "epoch": 1.987810595405532,
1490
+ "grad_norm": 0.5304378867149353,
1491
+ "learning_rate": 3.289340088251642e-05,
1492
+ "loss": 0.5162,
1493
+ "step": 1060
1494
+ },
1495
+ {
1496
+ "epoch": 1.9971870604781996,
1497
+ "grad_norm": 0.4507236182689667,
1498
+ "learning_rate": 3.275344792938791e-05,
1499
+ "loss": 0.543,
1500
+ "step": 1065
1501
+ },
1502
+ {
1503
+ "epoch": 2.006563525550867,
1504
+ "grad_norm": 0.42468950152397156,
1505
+ "learning_rate": 3.2613225612636525e-05,
1506
+ "loss": 0.4996,
1507
+ "step": 1070
1508
+ },
1509
+ {
1510
+ "epoch": 2.015939990623535,
1511
+ "grad_norm": 0.4971221089363098,
1512
+ "learning_rate": 3.247273880374542e-05,
1513
+ "loss": 0.5008,
1514
+ "step": 1075
1515
+ },
1516
+ {
1517
+ "epoch": 2.0253164556962027,
1518
+ "grad_norm": 0.5252302885055542,
1519
+ "learning_rate": 3.2331992383386566e-05,
1520
+ "loss": 0.5758,
1521
+ "step": 1080
1522
+ },
1523
+ {
1524
+ "epoch": 2.03469292076887,
1525
+ "grad_norm": 0.450898140668869,
1526
+ "learning_rate": 3.21909912412511e-05,
1527
+ "loss": 0.5323,
1528
+ "step": 1085
1529
+ },
1530
+ {
1531
+ "epoch": 2.0440693858415377,
1532
+ "grad_norm": 0.480165034532547,
1533
+ "learning_rate": 3.2049740275879493e-05,
1534
+ "loss": 0.5572,
1535
+ "step": 1090
1536
+ },
1537
+ {
1538
+ "epoch": 2.0534458509142053,
1539
+ "grad_norm": 0.5092324018478394,
1540
+ "learning_rate": 3.190824439449137e-05,
1541
+ "loss": 0.5761,
1542
+ "step": 1095
1543
+ },
1544
+ {
1545
+ "epoch": 2.062822315986873,
1546
+ "grad_norm": 0.5136324167251587,
1547
+ "learning_rate": 3.176650851281499e-05,
1548
+ "loss": 0.5328,
1549
+ "step": 1100
1550
+ },
1551
+ {
1552
+ "epoch": 2.0721987810595404,
1553
+ "grad_norm": 0.47322919964790344,
1554
+ "learning_rate": 3.162453755491655e-05,
1555
+ "loss": 0.4987,
1556
+ "step": 1105
1557
+ },
1558
+ {
1559
+ "epoch": 2.0815752461322083,
1560
+ "grad_norm": 0.4827199876308441,
1561
+ "learning_rate": 3.1482336453028986e-05,
1562
+ "loss": 0.5158,
1563
+ "step": 1110
1564
+ },
1565
+ {
1566
+ "epoch": 2.090951711204876,
1567
+ "grad_norm": 0.4925857484340668,
1568
+ "learning_rate": 3.133991014738076e-05,
1569
+ "loss": 0.5462,
1570
+ "step": 1115
1571
+ },
1572
+ {
1573
+ "epoch": 2.1003281762775434,
1574
+ "grad_norm": 0.4654436409473419,
1575
+ "learning_rate": 3.1197263586024155e-05,
1576
+ "loss": 0.5337,
1577
+ "step": 1120
1578
+ },
1579
+ {
1580
+ "epoch": 2.109704641350211,
1581
+ "grad_norm": 0.5248768925666809,
1582
+ "learning_rate": 3.105440172466337e-05,
1583
+ "loss": 0.5591,
1584
+ "step": 1125
1585
+ },
1586
+ {
1587
+ "epoch": 2.1190811064228785,
1588
+ "grad_norm": 0.5586119294166565,
1589
+ "learning_rate": 3.09113295264824e-05,
1590
+ "loss": 0.5473,
1591
+ "step": 1130
1592
+ },
1593
+ {
1594
+ "epoch": 2.128457571495546,
1595
+ "grad_norm": 0.500682532787323,
1596
+ "learning_rate": 3.076805196197255e-05,
1597
+ "loss": 0.5381,
1598
+ "step": 1135
1599
+ },
1600
+ {
1601
+ "epoch": 2.1378340365682136,
1602
+ "grad_norm": 0.5898781418800354,
1603
+ "learning_rate": 3.0624574008759805e-05,
1604
+ "loss": 0.6433,
1605
+ "step": 1140
1606
+ },
1607
+ {
1608
+ "epoch": 2.1472105016408816,
1609
+ "grad_norm": 0.5896307826042175,
1610
+ "learning_rate": 3.0480900651431876e-05,
1611
+ "loss": 0.5256,
1612
+ "step": 1145
1613
+ },
1614
+ {
1615
+ "epoch": 2.156586966713549,
1616
+ "grad_norm": 0.5270710587501526,
1617
+ "learning_rate": 3.0337036881365045e-05,
1618
+ "loss": 0.512,
1619
+ "step": 1150
1620
+ },
1621
+ {
1622
+ "epoch": 2.1659634317862166,
1623
+ "grad_norm": 0.5556356906890869,
1624
+ "learning_rate": 3.0192987696550746e-05,
1625
+ "loss": 0.5319,
1626
+ "step": 1155
1627
+ },
1628
+ {
1629
+ "epoch": 2.175339896858884,
1630
+ "grad_norm": 0.5308244228363037,
1631
+ "learning_rate": 3.0048758101421914e-05,
1632
+ "loss": 0.536,
1633
+ "step": 1160
1634
+ },
1635
+ {
1636
+ "epoch": 2.1847163619315517,
1637
+ "grad_norm": 0.5032731294631958,
1638
+ "learning_rate": 2.9904353106679178e-05,
1639
+ "loss": 0.544,
1640
+ "step": 1165
1641
+ },
1642
+ {
1643
+ "epoch": 2.1940928270042193,
1644
+ "grad_norm": 0.4815014898777008,
1645
+ "learning_rate": 2.975977772911671e-05,
1646
+ "loss": 0.5989,
1647
+ "step": 1170
1648
+ },
1649
+ {
1650
+ "epoch": 2.2034692920768872,
1651
+ "grad_norm": 0.5544810891151428,
1652
+ "learning_rate": 2.9615036991448015e-05,
1653
+ "loss": 0.5598,
1654
+ "step": 1175
1655
+ },
1656
+ {
1657
+ "epoch": 2.212845757149555,
1658
+ "grad_norm": 0.5214317440986633,
1659
+ "learning_rate": 2.947013592213137e-05,
1660
+ "loss": 0.5667,
1661
+ "step": 1180
1662
+ },
1663
+ {
1664
+ "epoch": 2.2222222222222223,
1665
+ "grad_norm": 0.47054922580718994,
1666
+ "learning_rate": 2.9325079555195163e-05,
1667
+ "loss": 0.5661,
1668
+ "step": 1185
1669
+ },
1670
+ {
1671
+ "epoch": 2.23159868729489,
1672
+ "grad_norm": 0.5608205795288086,
1673
+ "learning_rate": 2.9179872930063e-05,
1674
+ "loss": 0.5312,
1675
+ "step": 1190
1676
+ },
1677
+ {
1678
+ "epoch": 2.2409751523675574,
1679
+ "grad_norm": 0.5608832240104675,
1680
+ "learning_rate": 2.9034521091378635e-05,
1681
+ "loss": 0.5118,
1682
+ "step": 1195
1683
+ },
1684
+ {
1685
+ "epoch": 2.250351617440225,
1686
+ "grad_norm": 0.5235047936439514,
1687
+ "learning_rate": 2.8889029088830686e-05,
1688
+ "loss": 0.5503,
1689
+ "step": 1200
1690
+ },
1691
+ {
1692
+ "epoch": 2.2597280825128925,
1693
+ "grad_norm": 0.5191235542297363,
1694
+ "learning_rate": 2.8743401976977257e-05,
1695
+ "loss": 0.5345,
1696
+ "step": 1205
1697
+ },
1698
+ {
1699
+ "epoch": 2.2691045475855605,
1700
+ "grad_norm": 0.5068000555038452,
1701
+ "learning_rate": 2.8597644815070263e-05,
1702
+ "loss": 0.594,
1703
+ "step": 1210
1704
+ },
1705
+ {
1706
+ "epoch": 2.278481012658228,
1707
+ "grad_norm": 0.525265097618103,
1708
+ "learning_rate": 2.845176266687974e-05,
1709
+ "loss": 0.5237,
1710
+ "step": 1215
1711
+ },
1712
+ {
1713
+ "epoch": 2.2878574777308955,
1714
+ "grad_norm": 0.5695721507072449,
1715
+ "learning_rate": 2.8305760600517862e-05,
1716
+ "loss": 0.5142,
1717
+ "step": 1220
1718
+ },
1719
+ {
1720
+ "epoch": 2.297233942803563,
1721
+ "grad_norm": 0.5257561802864075,
1722
+ "learning_rate": 2.815964368826292e-05,
1723
+ "loss": 0.5414,
1724
+ "step": 1225
1725
+ },
1726
+ {
1727
+ "epoch": 2.3066104078762306,
1728
+ "grad_norm": 0.5726068019866943,
1729
+ "learning_rate": 2.8013417006383076e-05,
1730
+ "loss": 0.525,
1731
+ "step": 1230
1732
+ },
1733
+ {
1734
+ "epoch": 2.315986872948898,
1735
+ "grad_norm": 0.555002748966217,
1736
+ "learning_rate": 2.7867085634960016e-05,
1737
+ "loss": 0.5435,
1738
+ "step": 1235
1739
+ },
1740
+ {
1741
+ "epoch": 2.3253633380215657,
1742
+ "grad_norm": 0.6023123860359192,
1743
+ "learning_rate": 2.772065465771244e-05,
1744
+ "loss": 0.5355,
1745
+ "step": 1240
1746
+ },
1747
+ {
1748
+ "epoch": 2.3347398030942337,
1749
+ "grad_norm": 0.6386926174163818,
1750
+ "learning_rate": 2.7574129161819495e-05,
1751
+ "loss": 0.5647,
1752
+ "step": 1245
1753
+ },
1754
+ {
1755
+ "epoch": 2.344116268166901,
1756
+ "grad_norm": 0.4695705771446228,
1757
+ "learning_rate": 2.7427514237744e-05,
1758
+ "loss": 0.558,
1759
+ "step": 1250
1760
+ },
1761
+ {
1762
+ "epoch": 2.3534927332395688,
1763
+ "grad_norm": 0.5911107063293457,
1764
+ "learning_rate": 2.7280814979055612e-05,
1765
+ "loss": 0.5683,
1766
+ "step": 1255
1767
+ },
1768
+ {
1769
+ "epoch": 2.3628691983122363,
1770
+ "grad_norm": 0.4722149074077606,
1771
+ "learning_rate": 2.713403648225388e-05,
1772
+ "loss": 0.5172,
1773
+ "step": 1260
1774
+ },
1775
+ {
1776
+ "epoch": 2.372245663384904,
1777
+ "grad_norm": 0.5362224578857422,
1778
+ "learning_rate": 2.698718384659114e-05,
1779
+ "loss": 0.5339,
1780
+ "step": 1265
1781
+ },
1782
+ {
1783
+ "epoch": 2.3816221284575714,
1784
+ "grad_norm": 0.5780048370361328,
1785
+ "learning_rate": 2.684026217389544e-05,
1786
+ "loss": 0.6172,
1787
+ "step": 1270
1788
+ },
1789
+ {
1790
+ "epoch": 2.390998593530239,
1791
+ "grad_norm": 0.4645563066005707,
1792
+ "learning_rate": 2.6693276568393245e-05,
1793
+ "loss": 0.5337,
1794
+ "step": 1275
1795
+ },
1796
+ {
1797
+ "epoch": 2.400375058602907,
1798
+ "grad_norm": 0.48818090558052063,
1799
+ "learning_rate": 2.6546232136532083e-05,
1800
+ "loss": 0.5903,
1801
+ "step": 1280
1802
+ },
1803
+ {
1804
+ "epoch": 2.4097515236755744,
1805
+ "grad_norm": 0.5511724948883057,
1806
+ "learning_rate": 2.639913398680322e-05,
1807
+ "loss": 0.5693,
1808
+ "step": 1285
1809
+ },
1810
+ {
1811
+ "epoch": 2.419127988748242,
1812
+ "grad_norm": 0.506675124168396,
1813
+ "learning_rate": 2.6251987229564123e-05,
1814
+ "loss": 0.5415,
1815
+ "step": 1290
1816
+ },
1817
+ {
1818
+ "epoch": 2.4285044538209095,
1819
+ "grad_norm": 0.5370191335678101,
1820
+ "learning_rate": 2.610479697686093e-05,
1821
+ "loss": 0.5063,
1822
+ "step": 1295
1823
+ },
1824
+ {
1825
+ "epoch": 2.437880918893577,
1826
+ "grad_norm": 0.5342317223548889,
1827
+ "learning_rate": 2.595756834225089e-05,
1828
+ "loss": 0.5556,
1829
+ "step": 1300
1830
+ },
1831
+ {
1832
+ "epoch": 2.4472573839662446,
1833
+ "grad_norm": 0.5420209765434265,
1834
+ "learning_rate": 2.5810306440624644e-05,
1835
+ "loss": 0.6267,
1836
+ "step": 1305
1837
+ },
1838
+ {
1839
+ "epoch": 2.456633849038912,
1840
+ "grad_norm": 0.5854496359825134,
1841
+ "learning_rate": 2.566301638802861e-05,
1842
+ "loss": 0.5637,
1843
+ "step": 1310
1844
+ },
1845
+ {
1846
+ "epoch": 2.46601031411158,
1847
+ "grad_norm": 0.5205966234207153,
1848
+ "learning_rate": 2.551570330148716e-05,
1849
+ "loss": 0.5426,
1850
+ "step": 1315
1851
+ },
1852
+ {
1853
+ "epoch": 2.4753867791842477,
1854
+ "grad_norm": 0.5490741729736328,
1855
+ "learning_rate": 2.5368372298824922e-05,
1856
+ "loss": 0.5028,
1857
+ "step": 1320
1858
+ },
1859
+ {
1860
+ "epoch": 2.484763244256915,
1861
+ "grad_norm": 0.5535454750061035,
1862
+ "learning_rate": 2.5221028498488947e-05,
1863
+ "loss": 0.5536,
1864
+ "step": 1325
1865
+ },
1866
+ {
1867
+ "epoch": 2.4941397093295827,
1868
+ "grad_norm": 0.624588668346405,
1869
+ "learning_rate": 2.507367701937087e-05,
1870
+ "loss": 0.5199,
1871
+ "step": 1330
1872
+ },
1873
+ {
1874
+ "epoch": 2.5035161744022503,
1875
+ "grad_norm": 0.5129014253616333,
1876
+ "learning_rate": 2.492632298062913e-05,
1877
+ "loss": 0.5328,
1878
+ "step": 1335
1879
+ },
1880
+ {
1881
+ "epoch": 2.512892639474918,
1882
+ "grad_norm": 0.5573890805244446,
1883
+ "learning_rate": 2.4778971501511063e-05,
1884
+ "loss": 0.5625,
1885
+ "step": 1340
1886
+ },
1887
+ {
1888
+ "epoch": 2.5222691045475853,
1889
+ "grad_norm": 0.5620486736297607,
1890
+ "learning_rate": 2.4631627701175084e-05,
1891
+ "loss": 0.5238,
1892
+ "step": 1345
1893
+ },
1894
+ {
1895
+ "epoch": 2.5316455696202533,
1896
+ "grad_norm": 0.5784332752227783,
1897
+ "learning_rate": 2.448429669851285e-05,
1898
+ "loss": 0.5108,
1899
+ "step": 1350
1900
+ },
1901
+ {
1902
+ "epoch": 2.541022034692921,
1903
+ "grad_norm": 0.6034047603607178,
1904
+ "learning_rate": 2.43369836119714e-05,
1905
+ "loss": 0.5603,
1906
+ "step": 1355
1907
+ },
1908
+ {
1909
+ "epoch": 2.5503984997655884,
1910
+ "grad_norm": 0.5075374841690063,
1911
+ "learning_rate": 2.4189693559375365e-05,
1912
+ "loss": 0.574,
1913
+ "step": 1360
1914
+ },
1915
+ {
1916
+ "epoch": 2.559774964838256,
1917
+ "grad_norm": 0.5321151614189148,
1918
+ "learning_rate": 2.4042431657749117e-05,
1919
+ "loss": 0.5086,
1920
+ "step": 1365
1921
+ },
1922
+ {
1923
+ "epoch": 2.5691514299109235,
1924
+ "grad_norm": 0.5753352642059326,
1925
+ "learning_rate": 2.3895203023139073e-05,
1926
+ "loss": 0.5052,
1927
+ "step": 1370
1928
+ },
1929
+ {
1930
+ "epoch": 2.578527894983591,
1931
+ "grad_norm": 0.552044153213501,
1932
+ "learning_rate": 2.3748012770435883e-05,
1933
+ "loss": 0.5636,
1934
+ "step": 1375
1935
+ },
1936
+ {
1937
+ "epoch": 2.5879043600562586,
1938
+ "grad_norm": 0.5452053546905518,
1939
+ "learning_rate": 2.3600866013196787e-05,
1940
+ "loss": 0.5367,
1941
+ "step": 1380
1942
+ },
1943
+ {
1944
+ "epoch": 2.5972808251289266,
1945
+ "grad_norm": 0.5029872059822083,
1946
+ "learning_rate": 2.3453767863467923e-05,
1947
+ "loss": 0.5316,
1948
+ "step": 1385
1949
+ },
1950
+ {
1951
+ "epoch": 2.606657290201594,
1952
+ "grad_norm": 0.5490592122077942,
1953
+ "learning_rate": 2.3306723431606758e-05,
1954
+ "loss": 0.5161,
1955
+ "step": 1390
1956
+ },
1957
+ {
1958
+ "epoch": 2.6160337552742616,
1959
+ "grad_norm": 0.5136232376098633,
1960
+ "learning_rate": 2.3159737826104565e-05,
1961
+ "loss": 0.5728,
1962
+ "step": 1395
1963
+ },
1964
+ {
1965
+ "epoch": 2.625410220346929,
1966
+ "grad_norm": 0.5669593811035156,
1967
+ "learning_rate": 2.3012816153408863e-05,
1968
+ "loss": 0.5436,
1969
+ "step": 1400
1970
+ },
1971
+ {
1972
+ "epoch": 2.6347866854195967,
1973
+ "grad_norm": 0.5253113508224487,
1974
+ "learning_rate": 2.286596351774613e-05,
1975
+ "loss": 0.5236,
1976
+ "step": 1405
1977
+ },
1978
+ {
1979
+ "epoch": 2.6441631504922647,
1980
+ "grad_norm": 0.6063436269760132,
1981
+ "learning_rate": 2.271918502094439e-05,
1982
+ "loss": 0.5078,
1983
+ "step": 1410
1984
+ },
1985
+ {
1986
+ "epoch": 2.653539615564932,
1987
+ "grad_norm": 0.5364498496055603,
1988
+ "learning_rate": 2.2572485762256005e-05,
1989
+ "loss": 0.503,
1990
+ "step": 1415
1991
+ },
1992
+ {
1993
+ "epoch": 2.6629160806375998,
1994
+ "grad_norm": 0.5893564820289612,
1995
+ "learning_rate": 2.2425870838180507e-05,
1996
+ "loss": 0.5348,
1997
+ "step": 1420
1998
+ },
1999
+ {
2000
+ "epoch": 2.6722925457102673,
2001
+ "grad_norm": 0.523727536201477,
2002
+ "learning_rate": 2.2279345342287567e-05,
2003
+ "loss": 0.5343,
2004
+ "step": 1425
2005
+ },
2006
+ {
2007
+ "epoch": 2.681669010782935,
2008
+ "grad_norm": 0.5448949337005615,
2009
+ "learning_rate": 2.2132914365039993e-05,
2010
+ "loss": 0.5586,
2011
+ "step": 1430
2012
+ },
2013
+ {
2014
+ "epoch": 2.6910454758556024,
2015
+ "grad_norm": 0.5735951662063599,
2016
+ "learning_rate": 2.1986582993616926e-05,
2017
+ "loss": 0.5474,
2018
+ "step": 1435
2019
+ },
2020
+ {
2021
+ "epoch": 2.70042194092827,
2022
+ "grad_norm": 0.5605931878089905,
2023
+ "learning_rate": 2.1840356311737084e-05,
2024
+ "loss": 0.5474,
2025
+ "step": 1440
2026
+ },
2027
+ {
2028
+ "epoch": 2.709798406000938,
2029
+ "grad_norm": 0.48015716671943665,
2030
+ "learning_rate": 2.169423939948215e-05,
2031
+ "loss": 0.572,
2032
+ "step": 1445
2033
+ },
2034
+ {
2035
+ "epoch": 2.719174871073605,
2036
+ "grad_norm": 0.5914106965065002,
2037
+ "learning_rate": 2.154823733312027e-05,
2038
+ "loss": 0.5636,
2039
+ "step": 1450
2040
+ },
2041
+ {
2042
+ "epoch": 2.728551336146273,
2043
+ "grad_norm": 0.5666556358337402,
2044
+ "learning_rate": 2.140235518492975e-05,
2045
+ "loss": 0.5305,
2046
+ "step": 1455
2047
+ },
2048
+ {
2049
+ "epoch": 2.7379278012189405,
2050
+ "grad_norm": 0.6020416617393494,
2051
+ "learning_rate": 2.125659802302275e-05,
2052
+ "loss": 0.5273,
2053
+ "step": 1460
2054
+ },
2055
+ {
2056
+ "epoch": 2.747304266291608,
2057
+ "grad_norm": 0.46794259548187256,
2058
+ "learning_rate": 2.1110970911169316e-05,
2059
+ "loss": 0.5383,
2060
+ "step": 1465
2061
+ },
2062
+ {
2063
+ "epoch": 2.7566807313642756,
2064
+ "grad_norm": 0.525830864906311,
2065
+ "learning_rate": 2.096547890862137e-05,
2066
+ "loss": 0.5482,
2067
+ "step": 1470
2068
+ },
2069
+ {
2070
+ "epoch": 2.766057196436943,
2071
+ "grad_norm": 0.5674699544906616,
2072
+ "learning_rate": 2.0820127069937008e-05,
2073
+ "loss": 0.5226,
2074
+ "step": 1475
2075
+ },
2076
+ {
2077
+ "epoch": 2.775433661509611,
2078
+ "grad_norm": 0.5397214293479919,
2079
+ "learning_rate": 2.0674920444804847e-05,
2080
+ "loss": 0.5276,
2081
+ "step": 1480
2082
+ },
2083
+ {
2084
+ "epoch": 2.7848101265822782,
2085
+ "grad_norm": 0.5620929002761841,
2086
+ "learning_rate": 2.0529864077868643e-05,
2087
+ "loss": 0.5636,
2088
+ "step": 1485
2089
+ },
2090
+ {
2091
+ "epoch": 2.794186591654946,
2092
+ "grad_norm": 0.5328047275543213,
2093
+ "learning_rate": 2.0384963008551995e-05,
2094
+ "loss": 0.5531,
2095
+ "step": 1490
2096
+ },
2097
+ {
2098
+ "epoch": 2.8035630567276137,
2099
+ "grad_norm": 0.5161858797073364,
2100
+ "learning_rate": 2.0240222270883288e-05,
2101
+ "loss": 0.5709,
2102
+ "step": 1495
2103
+ },
2104
+ {
2105
+ "epoch": 2.8129395218002813,
2106
+ "grad_norm": 0.547215461730957,
2107
+ "learning_rate": 2.0095646893320828e-05,
2108
+ "loss": 0.53,
2109
+ "step": 1500
2110
+ },
2111
+ {
2112
+ "epoch": 2.822315986872949,
2113
+ "grad_norm": 0.5471156239509583,
2114
+ "learning_rate": 1.9951241898578085e-05,
2115
+ "loss": 0.5258,
2116
+ "step": 1505
2117
+ },
2118
+ {
2119
+ "epoch": 2.8316924519456164,
2120
+ "grad_norm": 0.576733410358429,
2121
+ "learning_rate": 1.980701230344926e-05,
2122
+ "loss": 0.536,
2123
+ "step": 1510
2124
+ },
2125
+ {
2126
+ "epoch": 2.8410689170182843,
2127
+ "grad_norm": 0.5285231471061707,
2128
+ "learning_rate": 1.9662963118634954e-05,
2129
+ "loss": 0.5724,
2130
+ "step": 1515
2131
+ },
2132
+ {
2133
+ "epoch": 2.8504453820909514,
2134
+ "grad_norm": 0.571106493473053,
2135
+ "learning_rate": 1.9519099348568127e-05,
2136
+ "loss": 0.5325,
2137
+ "step": 1520
2138
+ },
2139
+ {
2140
+ "epoch": 2.8598218471636194,
2141
+ "grad_norm": 0.5687730312347412,
2142
+ "learning_rate": 1.93754259912402e-05,
2143
+ "loss": 0.5505,
2144
+ "step": 1525
2145
+ },
2146
+ {
2147
+ "epoch": 2.869198312236287,
2148
+ "grad_norm": 0.5169468522071838,
2149
+ "learning_rate": 1.9231948038027462e-05,
2150
+ "loss": 0.5153,
2151
+ "step": 1530
2152
+ },
2153
+ {
2154
+ "epoch": 2.8785747773089545,
2155
+ "grad_norm": 0.5131816267967224,
2156
+ "learning_rate": 1.9088670473517605e-05,
2157
+ "loss": 0.5411,
2158
+ "step": 1535
2159
+ },
2160
+ {
2161
+ "epoch": 2.887951242381622,
2162
+ "grad_norm": 0.509089469909668,
2163
+ "learning_rate": 1.8945598275336633e-05,
2164
+ "loss": 0.5085,
2165
+ "step": 1540
2166
+ },
2167
+ {
2168
+ "epoch": 2.8973277074542896,
2169
+ "grad_norm": 0.5635603070259094,
2170
+ "learning_rate": 1.8802736413975844e-05,
2171
+ "loss": 0.5683,
2172
+ "step": 1545
2173
+ },
2174
+ {
2175
+ "epoch": 2.9067041725269576,
2176
+ "grad_norm": 0.6164025068283081,
2177
+ "learning_rate": 1.866008985261924e-05,
2178
+ "loss": 0.5393,
2179
+ "step": 1550
2180
+ },
2181
+ {
2182
+ "epoch": 2.916080637599625,
2183
+ "grad_norm": 0.5791330337524414,
2184
+ "learning_rate": 1.8517663546971013e-05,
2185
+ "loss": 0.5303,
2186
+ "step": 1555
2187
+ },
2188
+ {
2189
+ "epoch": 2.9254571026722926,
2190
+ "grad_norm": 0.5574399828910828,
2191
+ "learning_rate": 1.8375462445083464e-05,
2192
+ "loss": 0.5599,
2193
+ "step": 1560
2194
+ },
2195
+ {
2196
+ "epoch": 2.93483356774496,
2197
+ "grad_norm": 0.5197755098342896,
2198
+ "learning_rate": 1.8233491487185006e-05,
2199
+ "loss": 0.5712,
2200
+ "step": 1565
2201
+ },
2202
+ {
2203
+ "epoch": 2.9442100328176277,
2204
+ "grad_norm": 0.5005416870117188,
2205
+ "learning_rate": 1.8091755605508643e-05,
2206
+ "loss": 0.5459,
2207
+ "step": 1570
2208
+ },
2209
+ {
2210
+ "epoch": 2.9535864978902953,
2211
+ "grad_norm": 0.6060538291931152,
2212
+ "learning_rate": 1.7950259724120512e-05,
2213
+ "loss": 0.5754,
2214
+ "step": 1575
2215
+ },
2216
+ {
2217
+ "epoch": 2.962962962962963,
2218
+ "grad_norm": 0.570067286491394,
2219
+ "learning_rate": 1.7809008758748913e-05,
2220
+ "loss": 0.5615,
2221
+ "step": 1580
2222
+ },
2223
+ {
2224
+ "epoch": 2.972339428035631,
2225
+ "grad_norm": 0.6312186121940613,
2226
+ "learning_rate": 1.766800761661344e-05,
2227
+ "loss": 0.5135,
2228
+ "step": 1585
2229
+ },
2230
+ {
2231
+ "epoch": 2.9817158931082983,
2232
+ "grad_norm": 0.6070550084114075,
2233
+ "learning_rate": 1.752726119625459e-05,
2234
+ "loss": 0.566,
2235
+ "step": 1590
2236
+ },
2237
+ {
2238
+ "epoch": 2.991092358180966,
2239
+ "grad_norm": 0.5160552859306335,
2240
+ "learning_rate": 1.7386774387363484e-05,
2241
+ "loss": 0.579,
2242
+ "step": 1595
2243
+ },
2244
+ {
2245
+ "epoch": 3.0004688232536334,
2246
+ "grad_norm": 0.5943719148635864,
2247
+ "learning_rate": 1.724655207061209e-05,
2248
+ "loss": 0.5286,
2249
+ "step": 1600
2250
+ },
2251
+ {
2252
+ "epoch": 3.009845288326301,
2253
+ "grad_norm": 0.5501561760902405,
2254
+ "learning_rate": 1.710659911748359e-05,
2255
+ "loss": 0.5806,
2256
+ "step": 1605
2257
+ },
2258
+ {
2259
+ "epoch": 3.0192217533989685,
2260
+ "grad_norm": 0.5667310953140259,
2261
+ "learning_rate": 1.696692039010317e-05,
2262
+ "loss": 0.534,
2263
+ "step": 1610
2264
+ },
2265
+ {
2266
+ "epoch": 3.028598218471636,
2267
+ "grad_norm": 0.5230310559272766,
2268
+ "learning_rate": 1.6827520741069118e-05,
2269
+ "loss": 0.5507,
2270
+ "step": 1615
2271
+ },
2272
+ {
2273
+ "epoch": 3.037974683544304,
2274
+ "grad_norm": 0.6031038165092468,
2275
+ "learning_rate": 1.6688405013284192e-05,
2276
+ "loss": 0.5537,
2277
+ "step": 1620
2278
+ },
2279
+ {
2280
+ "epoch": 3.0473511486169715,
2281
+ "grad_norm": 0.4670465290546417,
2282
+ "learning_rate": 1.6549578039787436e-05,
2283
+ "loss": 0.5575,
2284
+ "step": 1625
2285
+ },
2286
+ {
2287
+ "epoch": 3.056727613689639,
2288
+ "grad_norm": 0.5443038940429688,
2289
+ "learning_rate": 1.6411044643586204e-05,
2290
+ "loss": 0.5043,
2291
+ "step": 1630
2292
+ },
2293
+ {
2294
+ "epoch": 3.0661040787623066,
2295
+ "grad_norm": 0.6336117386817932,
2296
+ "learning_rate": 1.627280963748869e-05,
2297
+ "loss": 0.4903,
2298
+ "step": 1635
2299
+ },
2300
+ {
2301
+ "epoch": 3.075480543834974,
2302
+ "grad_norm": 0.49661606550216675,
2303
+ "learning_rate": 1.613487782393661e-05,
2304
+ "loss": 0.5934,
2305
+ "step": 1640
2306
+ },
2307
+ {
2308
+ "epoch": 3.0848570089076417,
2309
+ "grad_norm": 0.5555146932601929,
2310
+ "learning_rate": 1.5997253994838484e-05,
2311
+ "loss": 0.4922,
2312
+ "step": 1645
2313
+ },
2314
+ {
2315
+ "epoch": 3.0942334739803092,
2316
+ "grad_norm": 0.6635964512825012,
2317
+ "learning_rate": 1.5859942931403072e-05,
2318
+ "loss": 0.5327,
2319
+ "step": 1650
2320
+ },
2321
+ {
2322
+ "epoch": 3.103609939052977,
2323
+ "grad_norm": 0.5776465535163879,
2324
+ "learning_rate": 1.5722949403973308e-05,
2325
+ "loss": 0.5836,
2326
+ "step": 1655
2327
+ },
2328
+ {
2329
+ "epoch": 3.1129864041256448,
2330
+ "grad_norm": 0.6424173712730408,
2331
+ "learning_rate": 1.5586278171860546e-05,
2332
+ "loss": 0.527,
2333
+ "step": 1660
2334
+ },
2335
+ {
2336
+ "epoch": 3.1223628691983123,
2337
+ "grad_norm": 0.5594909191131592,
2338
+ "learning_rate": 1.5449933983179256e-05,
2339
+ "loss": 0.5466,
2340
+ "step": 1665
2341
+ },
2342
+ {
2343
+ "epoch": 3.13173933427098,
2344
+ "grad_norm": 0.5368645787239075,
2345
+ "learning_rate": 1.5313921574682032e-05,
2346
+ "loss": 0.5162,
2347
+ "step": 1670
2348
+ },
2349
+ {
2350
+ "epoch": 3.1411157993436474,
2351
+ "grad_norm": 0.5256815552711487,
2352
+ "learning_rate": 1.517824567159506e-05,
2353
+ "loss": 0.4985,
2354
+ "step": 1675
2355
+ },
2356
+ {
2357
+ "epoch": 3.150492264416315,
2358
+ "grad_norm": 0.5784279108047485,
2359
+ "learning_rate": 1.5042910987453909e-05,
2360
+ "loss": 0.5674,
2361
+ "step": 1680
2362
+ },
2363
+ {
2364
+ "epoch": 3.1598687294889825,
2365
+ "grad_norm": 0.5585960149765015,
2366
+ "learning_rate": 1.4907922223939874e-05,
2367
+ "loss": 0.5051,
2368
+ "step": 1685
2369
+ },
2370
+ {
2371
+ "epoch": 3.1692451945616504,
2372
+ "grad_norm": 0.5591123104095459,
2373
+ "learning_rate": 1.4773284070716503e-05,
2374
+ "loss": 0.5327,
2375
+ "step": 1690
2376
+ },
2377
+ {
2378
+ "epoch": 3.178621659634318,
2379
+ "grad_norm": 0.5815097689628601,
2380
+ "learning_rate": 1.4639001205266803e-05,
2381
+ "loss": 0.5064,
2382
+ "step": 1695
2383
+ },
2384
+ {
2385
+ "epoch": 3.1879981247069855,
2386
+ "grad_norm": 0.5162126421928406,
2387
+ "learning_rate": 1.4505078292730632e-05,
2388
+ "loss": 0.5338,
2389
+ "step": 1700
2390
+ },
2391
+ {
2392
+ "epoch": 3.197374589779653,
2393
+ "grad_norm": 0.6133378744125366,
2394
+ "learning_rate": 1.4371519985742715e-05,
2395
+ "loss": 0.4959,
2396
+ "step": 1705
2397
+ },
2398
+ {
2399
+ "epoch": 3.2067510548523206,
2400
+ "grad_norm": 0.6034416556358337,
2401
+ "learning_rate": 1.4238330924270927e-05,
2402
+ "loss": 0.5289,
2403
+ "step": 1710
2404
+ },
2405
+ {
2406
+ "epoch": 3.216127519924988,
2407
+ "grad_norm": 0.5579488277435303,
2408
+ "learning_rate": 1.4105515735455149e-05,
2409
+ "loss": 0.5657,
2410
+ "step": 1715
2411
+ },
2412
+ {
2413
+ "epoch": 3.2255039849976557,
2414
+ "grad_norm": 0.5845019221305847,
2415
+ "learning_rate": 1.3973079033446501e-05,
2416
+ "loss": 0.578,
2417
+ "step": 1720
2418
+ },
2419
+ {
2420
+ "epoch": 3.2348804500703237,
2421
+ "grad_norm": 0.6357687711715698,
2422
+ "learning_rate": 1.3841025419247045e-05,
2423
+ "loss": 0.5262,
2424
+ "step": 1725
2425
+ },
2426
+ {
2427
+ "epoch": 3.244256915142991,
2428
+ "grad_norm": 0.6028162240982056,
2429
+ "learning_rate": 1.3709359480549932e-05,
2430
+ "loss": 0.5048,
2431
+ "step": 1730
2432
+ },
2433
+ {
2434
+ "epoch": 3.2536333802156587,
2435
+ "grad_norm": 0.6072187423706055,
2436
+ "learning_rate": 1.3578085791580008e-05,
2437
+ "loss": 0.4904,
2438
+ "step": 1735
2439
+ },
2440
+ {
2441
+ "epoch": 3.2630098452883263,
2442
+ "grad_norm": 0.5650848150253296,
2443
+ "learning_rate": 1.3447208912934927e-05,
2444
+ "loss": 0.5938,
2445
+ "step": 1740
2446
+ },
2447
+ {
2448
+ "epoch": 3.272386310360994,
2449
+ "grad_norm": 0.6589064002037048,
2450
+ "learning_rate": 1.3316733391426716e-05,
2451
+ "loss": 0.5274,
2452
+ "step": 1745
2453
+ },
2454
+ {
2455
+ "epoch": 3.2817627754336613,
2456
+ "grad_norm": 0.5519670844078064,
2457
+ "learning_rate": 1.3186663759923782e-05,
2458
+ "loss": 0.5083,
2459
+ "step": 1750
2460
+ },
2461
+ {
2462
+ "epoch": 3.291139240506329,
2463
+ "grad_norm": 0.5798940658569336,
2464
+ "learning_rate": 1.3057004537193423e-05,
2465
+ "loss": 0.5323,
2466
+ "step": 1755
2467
+ },
2468
+ {
2469
+ "epoch": 3.300515705578997,
2470
+ "grad_norm": 0.5830232501029968,
2471
+ "learning_rate": 1.2927760227744943e-05,
2472
+ "loss": 0.4991,
2473
+ "step": 1760
2474
+ },
2475
+ {
2476
+ "epoch": 3.3098921706516644,
2477
+ "grad_norm": 0.583799421787262,
2478
+ "learning_rate": 1.2798935321673e-05,
2479
+ "loss": 0.4954,
2480
+ "step": 1765
2481
+ },
2482
+ {
2483
+ "epoch": 3.319268635724332,
2484
+ "grad_norm": 0.5336604714393616,
2485
+ "learning_rate": 1.2670534294501756e-05,
2486
+ "loss": 0.5194,
2487
+ "step": 1770
2488
+ },
2489
+ {
2490
+ "epoch": 3.3286451007969995,
2491
+ "grad_norm": 0.6230494976043701,
2492
+ "learning_rate": 1.2542561607029322e-05,
2493
+ "loss": 0.5209,
2494
+ "step": 1775
2495
+ },
2496
+ {
2497
+ "epoch": 3.338021565869667,
2498
+ "grad_norm": 0.5860636234283447,
2499
+ "learning_rate": 1.2415021705172799e-05,
2500
+ "loss": 0.5699,
2501
+ "step": 1780
2502
+ },
2503
+ {
2504
+ "epoch": 3.3473980309423346,
2505
+ "grad_norm": 0.5983242392539978,
2506
+ "learning_rate": 1.2287919019813807e-05,
2507
+ "loss": 0.51,
2508
+ "step": 1785
2509
+ },
2510
+ {
2511
+ "epoch": 3.356774496015002,
2512
+ "grad_norm": 0.6714468002319336,
2513
+ "learning_rate": 1.2161257966644588e-05,
2514
+ "loss": 0.5497,
2515
+ "step": 1790
2516
+ },
2517
+ {
2518
+ "epoch": 3.36615096108767,
2519
+ "grad_norm": 0.5774706602096558,
2520
+ "learning_rate": 1.2035042946014572e-05,
2521
+ "loss": 0.566,
2522
+ "step": 1795
2523
+ },
2524
+ {
2525
+ "epoch": 3.3755274261603376,
2526
+ "grad_norm": 0.5805652737617493,
2527
+ "learning_rate": 1.1909278342777513e-05,
2528
+ "loss": 0.5246,
2529
+ "step": 1800
2530
+ },
2531
+ {
2532
+ "epoch": 3.384903891233005,
2533
+ "grad_norm": 0.5634963512420654,
2534
+ "learning_rate": 1.1783968526139121e-05,
2535
+ "loss": 0.5261,
2536
+ "step": 1805
2537
+ },
2538
+ {
2539
+ "epoch": 3.3942803563056727,
2540
+ "grad_norm": 0.6619728207588196,
2541
+ "learning_rate": 1.1659117849505367e-05,
2542
+ "loss": 0.5497,
2543
+ "step": 1810
2544
+ },
2545
+ {
2546
+ "epoch": 3.4036568213783402,
2547
+ "grad_norm": 0.5979011058807373,
2548
+ "learning_rate": 1.1534730650331096e-05,
2549
+ "loss": 0.5185,
2550
+ "step": 1815
2551
+ },
2552
+ {
2553
+ "epoch": 3.413033286451008,
2554
+ "grad_norm": 0.6057260632514954,
2555
+ "learning_rate": 1.1410811249969475e-05,
2556
+ "loss": 0.5836,
2557
+ "step": 1820
2558
+ },
2559
+ {
2560
+ "epoch": 3.4224097515236753,
2561
+ "grad_norm": 0.5888515114784241,
2562
+ "learning_rate": 1.1287363953521779e-05,
2563
+ "loss": 0.5318,
2564
+ "step": 1825
2565
+ },
2566
+ {
2567
+ "epoch": 3.4317862165963433,
2568
+ "grad_norm": 0.5773415565490723,
2569
+ "learning_rate": 1.1164393049687868e-05,
2570
+ "loss": 0.5141,
2571
+ "step": 1830
2572
+ },
2573
+ {
2574
+ "epoch": 3.441162681669011,
2575
+ "grad_norm": 0.542140007019043,
2576
+ "learning_rate": 1.104190281061718e-05,
2577
+ "loss": 0.5626,
2578
+ "step": 1835
2579
+ },
2580
+ {
2581
+ "epoch": 3.4505391467416784,
2582
+ "grad_norm": 0.5948598980903625,
2583
+ "learning_rate": 1.0919897491760279e-05,
2584
+ "loss": 0.4792,
2585
+ "step": 1840
2586
+ },
2587
+ {
2588
+ "epoch": 3.459915611814346,
2589
+ "grad_norm": 0.5241517424583435,
2590
+ "learning_rate": 1.0798381331721109e-05,
2591
+ "loss": 0.515,
2592
+ "step": 1845
2593
+ },
2594
+ {
2595
+ "epoch": 3.4692920768870135,
2596
+ "grad_norm": 0.5624334812164307,
2597
+ "learning_rate": 1.0677358552109618e-05,
2598
+ "loss": 0.5393,
2599
+ "step": 1850
2600
+ },
2601
+ {
2602
+ "epoch": 3.4786685419596814,
2603
+ "grad_norm": 0.5150102376937866,
2604
+ "learning_rate": 1.0556833357395188e-05,
2605
+ "loss": 0.5096,
2606
+ "step": 1855
2607
+ },
2608
+ {
2609
+ "epoch": 3.488045007032349,
2610
+ "grad_norm": 0.5694672465324402,
2611
+ "learning_rate": 1.0436809934760527e-05,
2612
+ "loss": 0.5266,
2613
+ "step": 1860
2614
+ },
2615
+ {
2616
+ "epoch": 3.4974214721050165,
2617
+ "grad_norm": 0.6179835200309753,
2618
+ "learning_rate": 1.031729245395622e-05,
2619
+ "loss": 0.5003,
2620
+ "step": 1865
2621
+ },
2622
+ {
2623
+ "epoch": 3.506797937177684,
2624
+ "grad_norm": 0.6380405426025391,
2625
+ "learning_rate": 1.0198285067155827e-05,
2626
+ "loss": 0.5228,
2627
+ "step": 1870
2628
+ },
2629
+ {
2630
+ "epoch": 3.5161744022503516,
2631
+ "grad_norm": 0.6013153791427612,
2632
+ "learning_rate": 1.0079791908811683e-05,
2633
+ "loss": 0.5204,
2634
+ "step": 1875
2635
+ },
2636
+ {
2637
+ "epoch": 3.525550867323019,
2638
+ "grad_norm": 0.5853320956230164,
2639
+ "learning_rate": 9.961817095511242e-06,
2640
+ "loss": 0.5138,
2641
+ "step": 1880
2642
+ },
2643
+ {
2644
+ "epoch": 3.5349273323956867,
2645
+ "grad_norm": 0.5283975601196289,
2646
+ "learning_rate": 9.844364725834057e-06,
2647
+ "loss": 0.5748,
2648
+ "step": 1885
2649
+ },
2650
+ {
2651
+ "epoch": 3.5443037974683547,
2652
+ "grad_norm": 0.7122249603271484,
2653
+ "learning_rate": 9.727438880209366e-06,
2654
+ "loss": 0.5042,
2655
+ "step": 1890
2656
+ },
2657
+ {
2658
+ "epoch": 3.5536802625410218,
2659
+ "grad_norm": 0.6497731804847717,
2660
+ "learning_rate": 9.611043620774419e-06,
2661
+ "loss": 0.5591,
2662
+ "step": 1895
2663
+ },
2664
+ {
2665
+ "epoch": 3.5630567276136897,
2666
+ "grad_norm": 0.5497525930404663,
2667
+ "learning_rate": 9.495182991233236e-06,
2668
+ "loss": 0.5268,
2669
+ "step": 1900
2670
+ },
2671
+ {
2672
+ "epoch": 3.5724331926863573,
2673
+ "grad_norm": 0.6197280287742615,
2674
+ "learning_rate": 9.379861016716224e-06,
2675
+ "loss": 0.5592,
2676
+ "step": 1905
2677
+ },
2678
+ {
2679
+ "epoch": 3.581809657759025,
2680
+ "grad_norm": 0.564769446849823,
2681
+ "learning_rate": 9.265081703640285e-06,
2682
+ "loss": 0.5537,
2683
+ "step": 1910
2684
+ },
2685
+ {
2686
+ "epoch": 3.5911861228316924,
2687
+ "grad_norm": 0.5738486051559448,
2688
+ "learning_rate": 9.150849039569655e-06,
2689
+ "loss": 0.5243,
2690
+ "step": 1915
2691
+ },
2692
+ {
2693
+ "epoch": 3.60056258790436,
2694
+ "grad_norm": 0.5415158867835999,
2695
+ "learning_rate": 9.037166993077337e-06,
2696
+ "loss": 0.5264,
2697
+ "step": 1920
2698
+ },
2699
+ {
2700
+ "epoch": 3.609939052977028,
2701
+ "grad_norm": 0.5810761451721191,
2702
+ "learning_rate": 8.92403951360726e-06,
2703
+ "loss": 0.5062,
2704
+ "step": 1925
2705
+ },
2706
+ {
2707
+ "epoch": 3.6193155180496954,
2708
+ "grad_norm": 0.5113641619682312,
2709
+ "learning_rate": 8.811470531337102e-06,
2710
+ "loss": 0.5729,
2711
+ "step": 1930
2712
+ },
2713
+ {
2714
+ "epoch": 3.628691983122363,
2715
+ "grad_norm": 0.5926602482795715,
2716
+ "learning_rate": 8.699463957041649e-06,
2717
+ "loss": 0.5901,
2718
+ "step": 1935
2719
+ },
2720
+ {
2721
+ "epoch": 3.6380684481950305,
2722
+ "grad_norm": 0.5843651294708252,
2723
+ "learning_rate": 8.588023681957028e-06,
2724
+ "loss": 0.5517,
2725
+ "step": 1940
2726
+ },
2727
+ {
2728
+ "epoch": 3.647444913267698,
2729
+ "grad_norm": 0.6659519672393799,
2730
+ "learning_rate": 8.477153577645481e-06,
2731
+ "loss": 0.5617,
2732
+ "step": 1945
2733
+ },
2734
+ {
2735
+ "epoch": 3.6568213783403656,
2736
+ "grad_norm": 0.5458998084068298,
2737
+ "learning_rate": 8.36685749586087e-06,
2738
+ "loss": 0.551,
2739
+ "step": 1950
2740
+ },
2741
+ {
2742
+ "epoch": 3.666197843413033,
2743
+ "grad_norm": 0.6160162687301636,
2744
+ "learning_rate": 8.257139268414844e-06,
2745
+ "loss": 0.5394,
2746
+ "step": 1955
2747
+ },
2748
+ {
2749
+ "epoch": 3.675574308485701,
2750
+ "grad_norm": 0.5762403011322021,
2751
+ "learning_rate": 8.14800270704375e-06,
2752
+ "loss": 0.5532,
2753
+ "step": 1960
2754
+ },
2755
+ {
2756
+ "epoch": 3.6849507735583686,
2757
+ "grad_norm": 0.6198619604110718,
2758
+ "learning_rate": 8.039451603276185e-06,
2759
+ "loss": 0.5242,
2760
+ "step": 1965
2761
+ },
2762
+ {
2763
+ "epoch": 3.694327238631036,
2764
+ "grad_norm": 0.5094100832939148,
2765
+ "learning_rate": 7.931489728301292e-06,
2766
+ "loss": 0.6371,
2767
+ "step": 1970
2768
+ },
2769
+ {
2770
+ "epoch": 3.7037037037037037,
2771
+ "grad_norm": 0.6113427877426147,
2772
+ "learning_rate": 7.8241208328377e-06,
2773
+ "loss": 0.5144,
2774
+ "step": 1975
2775
+ },
2776
+ {
2777
+ "epoch": 3.7130801687763713,
2778
+ "grad_norm": 0.48081642389297485,
2779
+ "learning_rate": 7.71734864700331e-06,
2780
+ "loss": 0.5271,
2781
+ "step": 1980
2782
+ },
2783
+ {
2784
+ "epoch": 3.722456633849039,
2785
+ "grad_norm": 0.6206328868865967,
2786
+ "learning_rate": 7.611176880185597e-06,
2787
+ "loss": 0.5273,
2788
+ "step": 1985
2789
+ },
2790
+ {
2791
+ "epoch": 3.7318330989217063,
2792
+ "grad_norm": 0.5632244944572449,
2793
+ "learning_rate": 7.505609220912821e-06,
2794
+ "loss": 0.5249,
2795
+ "step": 1990
2796
+ },
2797
+ {
2798
+ "epoch": 3.7412095639943743,
2799
+ "grad_norm": 0.5980995893478394,
2800
+ "learning_rate": 7.4006493367258515e-06,
2801
+ "loss": 0.5625,
2802
+ "step": 1995
2803
+ },
2804
+ {
2805
+ "epoch": 3.750586029067042,
2806
+ "grad_norm": 0.6244497895240784,
2807
+ "learning_rate": 7.2963008740507656e-06,
2808
+ "loss": 0.5701,
2809
+ "step": 2000
2810
+ },
2811
+ {
2812
+ "epoch": 3.7599624941397094,
2813
+ "grad_norm": 0.589423656463623,
2814
+ "learning_rate": 7.192567458072138e-06,
2815
+ "loss": 0.583,
2816
+ "step": 2005
2817
+ },
2818
+ {
2819
+ "epoch": 3.769338959212377,
2820
+ "grad_norm": 0.5806391835212708,
2821
+ "learning_rate": 7.089452692607146e-06,
2822
+ "loss": 0.4779,
2823
+ "step": 2010
2824
+ },
2825
+ {
2826
+ "epoch": 3.7787154242850445,
2827
+ "grad_norm": 0.6201165914535522,
2828
+ "learning_rate": 6.986960159980327e-06,
2829
+ "loss": 0.4985,
2830
+ "step": 2015
2831
+ },
2832
+ {
2833
+ "epoch": 3.788091889357712,
2834
+ "grad_norm": 0.5155811905860901,
2835
+ "learning_rate": 6.885093420899152e-06,
2836
+ "loss": 0.5305,
2837
+ "step": 2020
2838
+ },
2839
+ {
2840
+ "epoch": 3.7974683544303796,
2841
+ "grad_norm": 0.6702911257743835,
2842
+ "learning_rate": 6.783856014330281e-06,
2843
+ "loss": 0.5356,
2844
+ "step": 2025
2845
+ },
2846
+ {
2847
+ "epoch": 3.8068448195030475,
2848
+ "grad_norm": 0.6424790024757385,
2849
+ "learning_rate": 6.68325145737669e-06,
2850
+ "loss": 0.5269,
2851
+ "step": 2030
2852
+ },
2853
+ {
2854
+ "epoch": 3.816221284575715,
2855
+ "grad_norm": 0.6156381368637085,
2856
+ "learning_rate": 6.583283245155414e-06,
2857
+ "loss": 0.5563,
2858
+ "step": 2035
2859
+ },
2860
+ {
2861
+ "epoch": 3.8255977496483826,
2862
+ "grad_norm": 0.7311144471168518,
2863
+ "learning_rate": 6.483954850676133e-06,
2864
+ "loss": 0.5146,
2865
+ "step": 2040
2866
+ },
2867
+ {
2868
+ "epoch": 3.83497421472105,
2869
+ "grad_norm": 0.5719829797744751,
2870
+ "learning_rate": 6.385269724720547e-06,
2871
+ "loss": 0.5614,
2872
+ "step": 2045
2873
+ },
2874
+ {
2875
+ "epoch": 3.8443506797937177,
2876
+ "grad_norm": 0.6313838958740234,
2877
+ "learning_rate": 6.28723129572247e-06,
2878
+ "loss": 0.4988,
2879
+ "step": 2050
2880
+ },
2881
+ {
2882
+ "epoch": 3.8537271448663852,
2883
+ "grad_norm": 0.649053156375885,
2884
+ "learning_rate": 6.189842969648737e-06,
2885
+ "loss": 0.5621,
2886
+ "step": 2055
2887
+ },
2888
+ {
2889
+ "epoch": 3.8631036099390528,
2890
+ "grad_norm": 0.5733803510665894,
2891
+ "learning_rate": 6.0931081298808316e-06,
2892
+ "loss": 0.5282,
2893
+ "step": 2060
2894
+ },
2895
+ {
2896
+ "epoch": 3.8724800750117208,
2897
+ "grad_norm": 0.6120590567588806,
2898
+ "learning_rate": 5.997030137097426e-06,
2899
+ "loss": 0.5119,
2900
+ "step": 2065
2901
+ },
2902
+ {
2903
+ "epoch": 3.8818565400843883,
2904
+ "grad_norm": 0.5763484239578247,
2905
+ "learning_rate": 5.901612329157535e-06,
2906
+ "loss": 0.5431,
2907
+ "step": 2070
2908
+ },
2909
+ {
2910
+ "epoch": 3.891233005157056,
2911
+ "grad_norm": 0.5271406173706055,
2912
+ "learning_rate": 5.806858020984629e-06,
2913
+ "loss": 0.5971,
2914
+ "step": 2075
2915
+ },
2916
+ {
2917
+ "epoch": 3.9006094702297234,
2918
+ "grad_norm": 0.6094108819961548,
2919
+ "learning_rate": 5.712770504451426e-06,
2920
+ "loss": 0.54,
2921
+ "step": 2080
2922
+ },
2923
+ {
2924
+ "epoch": 3.909985935302391,
2925
+ "grad_norm": 0.6518940329551697,
2926
+ "learning_rate": 5.619353048265552e-06,
2927
+ "loss": 0.5045,
2928
+ "step": 2085
2929
+ },
2930
+ {
2931
+ "epoch": 3.9193624003750585,
2932
+ "grad_norm": 0.5773950815200806,
2933
+ "learning_rate": 5.526608897855953e-06,
2934
+ "loss": 0.5086,
2935
+ "step": 2090
2936
+ },
2937
+ {
2938
+ "epoch": 3.928738865447726,
2939
+ "grad_norm": 0.5974025726318359,
2940
+ "learning_rate": 5.434541275260182e-06,
2941
+ "loss": 0.5774,
2942
+ "step": 2095
2943
+ },
2944
+ {
2945
+ "epoch": 3.938115330520394,
2946
+ "grad_norm": 0.5835773348808289,
2947
+ "learning_rate": 5.343153379012444e-06,
2948
+ "loss": 0.5541,
2949
+ "step": 2100
2950
+ },
2951
+ {
2952
+ "epoch": 3.9474917955930615,
2953
+ "grad_norm": 0.5908766388893127,
2954
+ "learning_rate": 5.252448384032471e-06,
2955
+ "loss": 0.517,
2956
+ "step": 2105
2957
+ },
2958
+ {
2959
+ "epoch": 3.956868260665729,
2960
+ "grad_norm": 0.6290919780731201,
2961
+ "learning_rate": 5.162429441515221e-06,
2962
+ "loss": 0.5474,
2963
+ "step": 2110
2964
+ },
2965
+ {
2966
+ "epoch": 3.9662447257383966,
2967
+ "grad_norm": 0.5998512506484985,
2968
+ "learning_rate": 5.073099678821413e-06,
2969
+ "loss": 0.5178,
2970
+ "step": 2115
2971
+ },
2972
+ {
2973
+ "epoch": 3.975621190811064,
2974
+ "grad_norm": 0.5630286335945129,
2975
+ "learning_rate": 4.984462199368872e-06,
2976
+ "loss": 0.5671,
2977
+ "step": 2120
2978
+ },
2979
+ {
2980
+ "epoch": 3.9849976558837317,
2981
+ "grad_norm": 0.6709501147270203,
2982
+ "learning_rate": 4.8965200825247245e-06,
2983
+ "loss": 0.5148,
2984
+ "step": 2125
2985
+ },
2986
+ {
2987
+ "epoch": 3.994374120956399,
2988
+ "grad_norm": 0.5447418689727783,
2989
+ "learning_rate": 4.809276383498376e-06,
2990
+ "loss": 0.5418,
2991
+ "step": 2130
2992
+ },
2993
+ {
2994
+ "epoch": 4.003750586029067,
2995
+ "grad_norm": 0.5624414086341858,
2996
+ "learning_rate": 4.722734133235438e-06,
2997
+ "loss": 0.5433,
2998
+ "step": 2135
2999
+ },
3000
+ {
3001
+ "epoch": 4.013127051101734,
3002
+ "grad_norm": 0.6009880304336548,
3003
+ "learning_rate": 4.636896338312374e-06,
3004
+ "loss": 0.581,
3005
+ "step": 2140
3006
+ },
3007
+ {
3008
+ "epoch": 4.022503516174402,
3009
+ "grad_norm": 0.6356287002563477,
3010
+ "learning_rate": 4.551765980832059e-06,
3011
+ "loss": 0.5108,
3012
+ "step": 2145
3013
+ },
3014
+ {
3015
+ "epoch": 4.03187998124707,
3016
+ "grad_norm": 0.6711658835411072,
3017
+ "learning_rate": 4.467346018320198e-06,
3018
+ "loss": 0.5533,
3019
+ "step": 2150
3020
+ },
3021
+ {
3022
+ "epoch": 4.041256446319737,
3023
+ "grad_norm": 0.6147199869155884,
3024
+ "learning_rate": 4.383639383622557e-06,
3025
+ "loss": 0.5372,
3026
+ "step": 2155
3027
+ },
3028
+ {
3029
+ "epoch": 4.050632911392405,
3030
+ "grad_norm": 0.6129254102706909,
3031
+ "learning_rate": 4.300648984803085e-06,
3032
+ "loss": 0.5183,
3033
+ "step": 2160
3034
+ },
3035
+ {
3036
+ "epoch": 4.060009376465072,
3037
+ "grad_norm": 0.5274255275726318,
3038
+ "learning_rate": 4.218377705042867e-06,
3039
+ "loss": 0.5426,
3040
+ "step": 2165
3041
+ },
3042
+ {
3043
+ "epoch": 4.06938584153774,
3044
+ "grad_norm": 0.5687923431396484,
3045
+ "learning_rate": 4.1368284025399965e-06,
3046
+ "loss": 0.5491,
3047
+ "step": 2170
3048
+ },
3049
+ {
3050
+ "epoch": 4.0787623066104075,
3051
+ "grad_norm": 0.5000860095024109,
3052
+ "learning_rate": 4.0560039104102305e-06,
3053
+ "loss": 0.5401,
3054
+ "step": 2175
3055
+ },
3056
+ {
3057
+ "epoch": 4.0881387716830755,
3058
+ "grad_norm": 0.5673547387123108,
3059
+ "learning_rate": 3.975907036588594e-06,
3060
+ "loss": 0.4976,
3061
+ "step": 2180
3062
+ },
3063
+ {
3064
+ "epoch": 4.0975152367557435,
3065
+ "grad_norm": 0.6658757925033569,
3066
+ "learning_rate": 3.8965405637318294e-06,
3067
+ "loss": 0.5103,
3068
+ "step": 2185
3069
+ },
3070
+ {
3071
+ "epoch": 4.106891701828411,
3072
+ "grad_norm": 0.6456134915351868,
3073
+ "learning_rate": 3.817907249121713e-06,
3074
+ "loss": 0.5443,
3075
+ "step": 2190
3076
+ },
3077
+ {
3078
+ "epoch": 4.1162681669010786,
3079
+ "grad_norm": 0.5926840901374817,
3080
+ "learning_rate": 3.7400098245692572e-06,
3081
+ "loss": 0.5377,
3082
+ "step": 2195
3083
+ },
3084
+ {
3085
+ "epoch": 4.125644631973746,
3086
+ "grad_norm": 0.5508164763450623,
3087
+ "learning_rate": 3.662850996319825e-06,
3088
+ "loss": 0.5117,
3089
+ "step": 2200
3090
+ },
3091
+ {
3092
+ "epoch": 4.135021097046414,
3093
+ "grad_norm": 0.6228732466697693,
3094
+ "learning_rate": 3.586433444959103e-06,
3095
+ "loss": 0.5157,
3096
+ "step": 2205
3097
+ },
3098
+ {
3099
+ "epoch": 4.144397562119081,
3100
+ "grad_norm": 0.6101076006889343,
3101
+ "learning_rate": 3.5107598253199758e-06,
3102
+ "loss": 0.5058,
3103
+ "step": 2210
3104
+ },
3105
+ {
3106
+ "epoch": 4.153774027191749,
3107
+ "grad_norm": 0.6002482175827026,
3108
+ "learning_rate": 3.4358327663902677e-06,
3109
+ "loss": 0.5233,
3110
+ "step": 2215
3111
+ },
3112
+ {
3113
+ "epoch": 4.163150492264417,
3114
+ "grad_norm": 0.6422848701477051,
3115
+ "learning_rate": 3.3616548712214756e-06,
3116
+ "loss": 0.5347,
3117
+ "step": 2220
3118
+ },
3119
+ {
3120
+ "epoch": 4.172526957337084,
3121
+ "grad_norm": 0.5175875425338745,
3122
+ "learning_rate": 3.288228716838246e-06,
3123
+ "loss": 0.5124,
3124
+ "step": 2225
3125
+ },
3126
+ {
3127
+ "epoch": 4.181903422409752,
3128
+ "grad_norm": 0.6100782155990601,
3129
+ "learning_rate": 3.2155568541489268e-06,
3130
+ "loss": 0.5641,
3131
+ "step": 2230
3132
+ },
3133
+ {
3134
+ "epoch": 4.191279887482419,
3135
+ "grad_norm": 0.5398646593093872,
3136
+ "learning_rate": 3.143641807856898e-06,
3137
+ "loss": 0.5416,
3138
+ "step": 2235
3139
+ },
3140
+ {
3141
+ "epoch": 4.200656352555087,
3142
+ "grad_norm": 0.5987186431884766,
3143
+ "learning_rate": 3.0724860763728767e-06,
3144
+ "loss": 0.5602,
3145
+ "step": 2240
3146
+ },
3147
+ {
3148
+ "epoch": 4.210032817627754,
3149
+ "grad_norm": 0.5340943336486816,
3150
+ "learning_rate": 3.0020921317281264e-06,
3151
+ "loss": 0.4998,
3152
+ "step": 2245
3153
+ },
3154
+ {
3155
+ "epoch": 4.219409282700422,
3156
+ "grad_norm": 0.6151329874992371,
3157
+ "learning_rate": 2.9324624194885436e-06,
3158
+ "loss": 0.5699,
3159
+ "step": 2250
3160
+ },
3161
+ {
3162
+ "epoch": 4.22878574777309,
3163
+ "grad_norm": 0.6071544885635376,
3164
+ "learning_rate": 2.8635993586697553e-06,
3165
+ "loss": 0.5114,
3166
+ "step": 2255
3167
+ },
3168
+ {
3169
+ "epoch": 4.238162212845757,
3170
+ "grad_norm": 0.5918383002281189,
3171
+ "learning_rate": 2.795505341653007e-06,
3172
+ "loss": 0.5061,
3173
+ "step": 2260
3174
+ },
3175
+ {
3176
+ "epoch": 4.247538677918425,
3177
+ "grad_norm": 0.5735620260238647,
3178
+ "learning_rate": 2.728182734102111e-06,
3179
+ "loss": 0.5334,
3180
+ "step": 2265
3181
+ },
3182
+ {
3183
+ "epoch": 4.256915142991092,
3184
+ "grad_norm": 0.579006016254425,
3185
+ "learning_rate": 2.6616338748812255e-06,
3186
+ "loss": 0.5867,
3187
+ "step": 2270
3188
+ },
3189
+ {
3190
+ "epoch": 4.26629160806376,
3191
+ "grad_norm": 0.5297543406486511,
3192
+ "learning_rate": 2.595861075973613e-06,
3193
+ "loss": 0.5287,
3194
+ "step": 2275
3195
+ },
3196
+ {
3197
+ "epoch": 4.275668073136427,
3198
+ "grad_norm": 0.5488983392715454,
3199
+ "learning_rate": 2.530866622401304e-06,
3200
+ "loss": 0.5324,
3201
+ "step": 2280
3202
+ },
3203
+ {
3204
+ "epoch": 4.285044538209095,
3205
+ "grad_norm": 0.5938432812690735,
3206
+ "learning_rate": 2.4666527721457416e-06,
3207
+ "loss": 0.5267,
3208
+ "step": 2285
3209
+ },
3210
+ {
3211
+ "epoch": 4.294421003281763,
3212
+ "grad_norm": 0.5408592224121094,
3213
+ "learning_rate": 2.40322175606931e-06,
3214
+ "loss": 0.548,
3215
+ "step": 2290
3216
+ },
3217
+ {
3218
+ "epoch": 4.30379746835443,
3219
+ "grad_norm": 0.6493216753005981,
3220
+ "learning_rate": 2.3405757778378445e-06,
3221
+ "loss": 0.4996,
3222
+ "step": 2295
3223
+ },
3224
+ {
3225
+ "epoch": 4.313173933427098,
3226
+ "grad_norm": 0.5764236450195312,
3227
+ "learning_rate": 2.278717013844059e-06,
3228
+ "loss": 0.5294,
3229
+ "step": 2300
3230
+ },
3231
+ {
3232
+ "epoch": 4.322550398499765,
3233
+ "grad_norm": 0.5868895053863525,
3234
+ "learning_rate": 2.2176476131319707e-06,
3235
+ "loss": 0.4882,
3236
+ "step": 2305
3237
+ },
3238
+ {
3239
+ "epoch": 4.331926863572433,
3240
+ "grad_norm": 0.7049744725227356,
3241
+ "learning_rate": 2.1573696973221922e-06,
3242
+ "loss": 0.5169,
3243
+ "step": 2310
3244
+ },
3245
+ {
3246
+ "epoch": 4.3413033286451,
3247
+ "grad_norm": 0.5863852500915527,
3248
+ "learning_rate": 2.0978853605382624e-06,
3249
+ "loss": 0.5325,
3250
+ "step": 2315
3251
+ },
3252
+ {
3253
+ "epoch": 4.350679793717768,
3254
+ "grad_norm": 0.5573306679725647,
3255
+ "learning_rate": 2.0391966693338733e-06,
3256
+ "loss": 0.4942,
3257
+ "step": 2320
3258
+ },
3259
+ {
3260
+ "epoch": 4.360056258790436,
3261
+ "grad_norm": 0.6028510928153992,
3262
+ "learning_rate": 1.9813056626210886e-06,
3263
+ "loss": 0.5117,
3264
+ "step": 2325
3265
+ },
3266
+ {
3267
+ "epoch": 4.369432723863103,
3268
+ "grad_norm": 0.5050864815711975,
3269
+ "learning_rate": 1.9242143515994933e-06,
3270
+ "loss": 0.5069,
3271
+ "step": 2330
3272
+ },
3273
+ {
3274
+ "epoch": 4.378809188935771,
3275
+ "grad_norm": 0.5868462920188904,
3276
+ "learning_rate": 1.8679247196863425e-06,
3277
+ "loss": 0.5183,
3278
+ "step": 2335
3279
+ },
3280
+ {
3281
+ "epoch": 4.3881856540084385,
3282
+ "grad_norm": 0.5458810925483704,
3283
+ "learning_rate": 1.8124387224476347e-06,
3284
+ "loss": 0.4979,
3285
+ "step": 2340
3286
+ },
3287
+ {
3288
+ "epoch": 4.3975621190811065,
3289
+ "grad_norm": 0.5342989563941956,
3290
+ "learning_rate": 1.757758287530195e-06,
3291
+ "loss": 0.5365,
3292
+ "step": 2345
3293
+ },
3294
+ {
3295
+ "epoch": 4.4069385841537745,
3296
+ "grad_norm": 0.5850876569747925,
3297
+ "learning_rate": 1.7038853145946804e-06,
3298
+ "loss": 0.5165,
3299
+ "step": 2350
3300
+ },
3301
+ {
3302
+ "epoch": 4.416315049226442,
3303
+ "grad_norm": 0.6274101734161377,
3304
+ "learning_rate": 1.6508216752496141e-06,
3305
+ "loss": 0.5718,
3306
+ "step": 2355
3307
+ },
3308
+ {
3309
+ "epoch": 4.42569151429911,
3310
+ "grad_norm": 0.6325822472572327,
3311
+ "learning_rate": 1.5985692129863395e-06,
3312
+ "loss": 0.5244,
3313
+ "step": 2360
3314
+ },
3315
+ {
3316
+ "epoch": 4.435067979371777,
3317
+ "grad_norm": 0.5776230692863464,
3318
+ "learning_rate": 1.547129743114978e-06,
3319
+ "loss": 0.5609,
3320
+ "step": 2365
3321
+ },
3322
+ {
3323
+ "epoch": 4.444444444444445,
3324
+ "grad_norm": 0.5962603092193604,
3325
+ "learning_rate": 1.496505052701372e-06,
3326
+ "loss": 0.494,
3327
+ "step": 2370
3328
+ },
3329
+ {
3330
+ "epoch": 4.453820909517112,
3331
+ "grad_norm": 0.5872151851654053,
3332
+ "learning_rate": 1.4466969005050013e-06,
3333
+ "loss": 0.5198,
3334
+ "step": 2375
3335
+ },
3336
+ {
3337
+ "epoch": 4.46319737458978,
3338
+ "grad_norm": 0.5666322112083435,
3339
+ "learning_rate": 1.3977070169178763e-06,
3340
+ "loss": 0.5284,
3341
+ "step": 2380
3342
+ },
3343
+ {
3344
+ "epoch": 4.472573839662447,
3345
+ "grad_norm": 0.5479874610900879,
3346
+ "learning_rate": 1.349537103904408e-06,
3347
+ "loss": 0.5223,
3348
+ "step": 2385
3349
+ },
3350
+ {
3351
+ "epoch": 4.481950304735115,
3352
+ "grad_norm": 0.598061740398407,
3353
+ "learning_rate": 1.3021888349423222e-06,
3354
+ "loss": 0.5128,
3355
+ "step": 2390
3356
+ },
3357
+ {
3358
+ "epoch": 4.491326769807783,
3359
+ "grad_norm": 0.5555986762046814,
3360
+ "learning_rate": 1.2556638549644644e-06,
3361
+ "loss": 0.5634,
3362
+ "step": 2395
3363
+ },
3364
+ {
3365
+ "epoch": 4.50070323488045,
3366
+ "grad_norm": 0.5764344930648804,
3367
+ "learning_rate": 1.2099637803016983e-06,
3368
+ "loss": 0.5168,
3369
+ "step": 2400
3370
+ },
3371
+ {
3372
+ "epoch": 4.510079699953118,
3373
+ "grad_norm": 0.5597651600837708,
3374
+ "learning_rate": 1.1650901986267365e-06,
3375
+ "loss": 0.4968,
3376
+ "step": 2405
3377
+ },
3378
+ {
3379
+ "epoch": 4.519456165025785,
3380
+ "grad_norm": 0.6328330039978027,
3381
+ "learning_rate": 1.1210446688989768e-06,
3382
+ "loss": 0.6302,
3383
+ "step": 2410
3384
+ },
3385
+ {
3386
+ "epoch": 4.528832630098453,
3387
+ "grad_norm": 0.6118944883346558,
3388
+ "learning_rate": 1.0778287213103478e-06,
3389
+ "loss": 0.5114,
3390
+ "step": 2415
3391
+ },
3392
+ {
3393
+ "epoch": 4.538209095171121,
3394
+ "grad_norm": 0.6393660306930542,
3395
+ "learning_rate": 1.0354438572321546e-06,
3396
+ "loss": 0.5136,
3397
+ "step": 2420
3398
+ },
3399
+ {
3400
+ "epoch": 4.547585560243788,
3401
+ "grad_norm": 0.5955559611320496,
3402
+ "learning_rate": 9.938915491629063e-07,
3403
+ "loss": 0.5265,
3404
+ "step": 2425
3405
+ },
3406
+ {
3407
+ "epoch": 4.556962025316456,
3408
+ "grad_norm": 0.635802686214447,
3409
+ "learning_rate": 9.531732406771771e-07,
3410
+ "loss": 0.5299,
3411
+ "step": 2430
3412
+ },
3413
+ {
3414
+ "epoch": 4.566338490389123,
3415
+ "grad_norm": 0.6105532050132751,
3416
+ "learning_rate": 9.132903463754256e-07,
3417
+ "loss": 0.594,
3418
+ "step": 2435
3419
+ },
3420
+ {
3421
+ "epoch": 4.575714955461791,
3422
+ "grad_norm": 0.5851595997810364,
3423
+ "learning_rate": 8.742442518348965e-07,
3424
+ "loss": 0.5352,
3425
+ "step": 2440
3426
+ },
3427
+ {
3428
+ "epoch": 4.585091420534458,
3429
+ "grad_norm": 0.573544442653656,
3430
+ "learning_rate": 8.360363135614307e-07,
3431
+ "loss": 0.5262,
3432
+ "step": 2445
3433
+ },
3434
+ {
3435
+ "epoch": 4.594467885607126,
3436
+ "grad_norm": 0.6063082814216614,
3437
+ "learning_rate": 7.986678589423758e-07,
3438
+ "loss": 0.5779,
3439
+ "step": 2450
3440
+ },
3441
+ {
3442
+ "epoch": 4.603844350679793,
3443
+ "grad_norm": 0.5579631328582764,
3444
+ "learning_rate": 7.621401862004634e-07,
3445
+ "loss": 0.5588,
3446
+ "step": 2455
3447
+ },
3448
+ {
3449
+ "epoch": 4.613220815752461,
3450
+ "grad_norm": 0.6332941651344299,
3451
+ "learning_rate": 7.264545643486997e-07,
3452
+ "loss": 0.5025,
3453
+ "step": 2460
3454
+ },
3455
+ {
3456
+ "epoch": 4.622597280825129,
3457
+ "grad_norm": 0.564562201499939,
3458
+ "learning_rate": 6.916122331462799e-07,
3459
+ "loss": 0.5156,
3460
+ "step": 2465
3461
+ },
3462
+ {
3463
+ "epoch": 4.631973745897796,
3464
+ "grad_norm": 0.5456069111824036,
3465
+ "learning_rate": 6.576144030555259e-07,
3466
+ "loss": 0.575,
3467
+ "step": 2470
3468
+ },
3469
+ {
3470
+ "epoch": 4.641350210970464,
3471
+ "grad_norm": 0.5309081077575684,
3472
+ "learning_rate": 6.244622551998203e-07,
3473
+ "loss": 0.5262,
3474
+ "step": 2475
3475
+ },
3476
+ {
3477
+ "epoch": 4.650726676043131,
3478
+ "grad_norm": 0.5958854556083679,
3479
+ "learning_rate": 5.921569413225913e-07,
3480
+ "loss": 0.5399,
3481
+ "step": 2480
3482
+ },
3483
+ {
3484
+ "epoch": 4.660103141115799,
3485
+ "grad_norm": 0.6013832092285156,
3486
+ "learning_rate": 5.606995837472817e-07,
3487
+ "loss": 0.5328,
3488
+ "step": 2485
3489
+ },
3490
+ {
3491
+ "epoch": 4.669479606188467,
3492
+ "grad_norm": 0.5663477182388306,
3493
+ "learning_rate": 5.300912753383625e-07,
3494
+ "loss": 0.5403,
3495
+ "step": 2490
3496
+ },
3497
+ {
3498
+ "epoch": 4.6788560712611345,
3499
+ "grad_norm": 0.539078950881958,
3500
+ "learning_rate": 5.003330794633776e-07,
3501
+ "loss": 0.5219,
3502
+ "step": 2495
3503
+ },
3504
+ {
3505
+ "epoch": 4.688232536333802,
3506
+ "grad_norm": 0.5885345339775085,
3507
+ "learning_rate": 4.714260299559875e-07,
3508
+ "loss": 0.5515,
3509
+ "step": 2500
3510
+ },
3511
+ {
3512
+ "epoch": 4.6976090014064695,
3513
+ "grad_norm": 0.6007012724876404,
3514
+ "learning_rate": 4.4337113108005314e-07,
3515
+ "loss": 0.527,
3516
+ "step": 2505
3517
+ },
3518
+ {
3519
+ "epoch": 4.7069854664791375,
3520
+ "grad_norm": 0.5390775799751282,
3521
+ "learning_rate": 4.161693574947556e-07,
3522
+ "loss": 0.5766,
3523
+ "step": 2510
3524
+ },
3525
+ {
3526
+ "epoch": 4.716361931551805,
3527
+ "grad_norm": 0.5515893697738647,
3528
+ "learning_rate": 3.8982165422073445e-07,
3529
+ "loss": 0.5084,
3530
+ "step": 2515
3531
+ },
3532
+ {
3533
+ "epoch": 4.725738396624473,
3534
+ "grad_norm": 0.5834362506866455,
3535
+ "learning_rate": 3.6432893660723886e-07,
3536
+ "loss": 0.5616,
3537
+ "step": 2520
3538
+ },
3539
+ {
3540
+ "epoch": 4.73511486169714,
3541
+ "grad_norm": 0.5748853087425232,
3542
+ "learning_rate": 3.396920903003559e-07,
3543
+ "loss": 0.5003,
3544
+ "step": 2525
3545
+ },
3546
+ {
3547
+ "epoch": 4.744491326769808,
3548
+ "grad_norm": 0.6511205434799194,
3549
+ "learning_rate": 3.1591197121222107e-07,
3550
+ "loss": 0.5369,
3551
+ "step": 2530
3552
+ },
3553
+ {
3554
+ "epoch": 4.753867791842476,
3555
+ "grad_norm": 0.6432448625564575,
3556
+ "learning_rate": 2.9298940549128964e-07,
3557
+ "loss": 0.5319,
3558
+ "step": 2535
3559
+ },
3560
+ {
3561
+ "epoch": 4.763244256915143,
3562
+ "grad_norm": 0.5946859121322632,
3563
+ "learning_rate": 2.7092518949362875e-07,
3564
+ "loss": 0.4929,
3565
+ "step": 2540
3566
+ },
3567
+ {
3568
+ "epoch": 4.772620721987811,
3569
+ "grad_norm": 0.5316891670227051,
3570
+ "learning_rate": 2.4972008975527593e-07,
3571
+ "loss": 0.5774,
3572
+ "step": 2545
3573
+ },
3574
+ {
3575
+ "epoch": 4.781997187060478,
3576
+ "grad_norm": 0.6336256265640259,
3577
+ "learning_rate": 2.2937484296556566e-07,
3578
+ "loss": 0.5231,
3579
+ "step": 2550
3580
+ },
3581
+ {
3582
+ "epoch": 4.791373652133146,
3583
+ "grad_norm": 0.5824437141418457,
3584
+ "learning_rate": 2.0989015594158058e-07,
3585
+ "loss": 0.5563,
3586
+ "step": 2555
3587
+ },
3588
+ {
3589
+ "epoch": 4.800750117205814,
3590
+ "grad_norm": 0.6376898288726807,
3591
+ "learning_rate": 1.9126670560356553e-07,
3592
+ "loss": 0.5601,
3593
+ "step": 2560
3594
+ },
3595
+ {
3596
+ "epoch": 4.810126582278481,
3597
+ "grad_norm": 0.610390305519104,
3598
+ "learning_rate": 1.735051389514214e-07,
3599
+ "loss": 0.5116,
3600
+ "step": 2565
3601
+ },
3602
+ {
3603
+ "epoch": 4.819503047351149,
3604
+ "grad_norm": 0.6232736706733704,
3605
+ "learning_rate": 1.5660607304223141e-07,
3606
+ "loss": 0.5159,
3607
+ "step": 2570
3608
+ },
3609
+ {
3610
+ "epoch": 4.828879512423816,
3611
+ "grad_norm": 0.6081536412239075,
3612
+ "learning_rate": 1.4057009496881158e-07,
3613
+ "loss": 0.5555,
3614
+ "step": 2575
3615
+ },
3616
+ {
3617
+ "epoch": 4.838255977496484,
3618
+ "grad_norm": 0.5945781469345093,
3619
+ "learning_rate": 1.2539776183932982e-07,
3620
+ "loss": 0.5433,
3621
+ "step": 2580
3622
+ },
3623
+ {
3624
+ "epoch": 4.847632442569151,
3625
+ "grad_norm": 0.574899435043335,
3626
+ "learning_rate": 1.1108960075794372e-07,
3627
+ "loss": 0.4752,
3628
+ "step": 2585
3629
+ },
3630
+ {
3631
+ "epoch": 4.857008907641819,
3632
+ "grad_norm": 0.5889243483543396,
3633
+ "learning_rate": 9.764610880648451e-08,
3634
+ "loss": 0.4921,
3635
+ "step": 2590
3636
+ },
3637
+ {
3638
+ "epoch": 4.866385372714487,
3639
+ "grad_norm": 0.5796253681182861,
3640
+ "learning_rate": 8.506775302719039e-08,
3641
+ "loss": 0.5313,
3642
+ "step": 2595
3643
+ },
3644
+ {
3645
+ "epoch": 4.875761837787154,
3646
+ "grad_norm": 0.5696360468864441,
3647
+ "learning_rate": 7.335497040648898e-08,
3648
+ "loss": 0.518,
3649
+ "step": 2600
3650
+ },
3651
+ {
3652
+ "epoch": 4.885138302859822,
3653
+ "grad_norm": 0.5847044587135315,
3654
+ "learning_rate": 6.250816785980385e-08,
3655
+ "loss": 0.5785,
3656
+ "step": 2605
3657
+ },
3658
+ {
3659
+ "epoch": 4.894514767932489,
3660
+ "grad_norm": 0.6327594518661499,
3661
+ "learning_rate": 5.2527722217421416e-08,
3662
+ "loss": 0.5209,
3663
+ "step": 2610
3664
+ },
3665
+ {
3666
+ "epoch": 4.903891233005157,
3667
+ "grad_norm": 0.6338078379631042,
3668
+ "learning_rate": 4.3413980211412516e-08,
3669
+ "loss": 0.5438,
3670
+ "step": 2615
3671
+ },
3672
+ {
3673
+ "epoch": 4.913267698077824,
3674
+ "grad_norm": 0.6505990624427795,
3675
+ "learning_rate": 3.516725846355873e-08,
3676
+ "loss": 0.4971,
3677
+ "step": 2620
3678
+ },
3679
+ {
3680
+ "epoch": 4.922644163150492,
3681
+ "grad_norm": 0.6756799817085266,
3682
+ "learning_rate": 2.7787843474386123e-08,
3683
+ "loss": 0.495,
3684
+ "step": 2625
3685
+ },
3686
+ {
3687
+ "epoch": 4.93202062822316,
3688
+ "grad_norm": 0.5888404250144958,
3689
+ "learning_rate": 2.127599161318161e-08,
3690
+ "loss": 0.5463,
3691
+ "step": 2630
3692
+ },
3693
+ {
3694
+ "epoch": 4.941397093295827,
3695
+ "grad_norm": 0.5684934258460999,
3696
+ "learning_rate": 1.5631929109102828e-08,
3697
+ "loss": 0.5461,
3698
+ "step": 2635
3699
+ },
3700
+ {
3701
+ "epoch": 4.950773558368495,
3702
+ "grad_norm": 0.5537897348403931,
3703
+ "learning_rate": 1.0855852043323289e-08,
3704
+ "loss": 0.5319,
3705
+ "step": 2640
3706
+ },
3707
+ {
3708
+ "epoch": 4.960150023441162,
3709
+ "grad_norm": 0.5860295295715332,
3710
+ "learning_rate": 6.947926342204536e-09,
3711
+ "loss": 0.5068,
3712
+ "step": 2645
3713
+ },
3714
+ {
3715
+ "epoch": 4.96952648851383,
3716
+ "grad_norm": 0.566766619682312,
3717
+ "learning_rate": 3.908287771542396e-09,
3718
+ "loss": 0.5091,
3719
+ "step": 2650
3720
+ },
3721
+ {
3722
+ "epoch": 4.978902953586498,
3723
+ "grad_norm": 0.5283293724060059,
3724
+ "learning_rate": 1.737041931845762e-09,
3725
+ "loss": 0.5747,
3726
+ "step": 2655
3727
+ },
3728
+ {
3729
+ "epoch": 4.9882794186591655,
3730
+ "grad_norm": 0.5808627009391785,
3731
+ "learning_rate": 4.3426425467008035e-10,
3732
+ "loss": 0.5346,
3733
+ "step": 2660
3734
+ },
3735
+ {
3736
+ "epoch": 4.9976558837318334,
3737
+ "grad_norm": 0.5620869398117065,
3738
+ "learning_rate": 0.0,
3739
+ "loss": 0.5285,
3740
+ "step": 2665
3741
+ },
3742
+ {
3743
+ "epoch": 4.9976558837318334,
3744
+ "step": 2665,
3745
+ "total_flos": 1.9473516408636703e+18,
3746
+ "train_loss": 0.5572177461119575,
3747
+ "train_runtime": 35175.961,
3748
+ "train_samples_per_second": 1.213,
3749
+ "train_steps_per_second": 0.076
3750
+ }
3751
+ ],
3752
+ "logging_steps": 5,
3753
+ "max_steps": 2665,
3754
+ "num_input_tokens_seen": 0,
3755
+ "num_train_epochs": 5,
3756
+ "save_steps": 100,
3757
+ "total_flos": 1.9473516408636703e+18,
3758
+ "train_batch_size": 2,
3759
+ "trial_name": null,
3760
+ "trial_params": null
3761
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a82fd09e2242cdb3d6f2d3f8992288a595cd1a118f30d9b7d514949202d87979
3
+ size 5176