Upload LoRA adapter folder

.gitattributes

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-1386/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-2079/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-2772/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-3465/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-4158/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-4851/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-5544/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-6237/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-6240/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-693/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+final/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,209 @@
+---
+base_model: Salesforce/Llama-xLAM-2-8b-fc-r
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Salesforce/Llama-xLAM-2-8b-fc-r
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Salesforce/Llama-xLAM-2-8b-fc-r",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc84d3c192316680e398885d49252ed59766fd8a5337572745617778ddf8f39a
+size 54560368

checkpoint-1386/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Salesforce/Llama-xLAM-2-8b-fc-r
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Salesforce/Llama-xLAM-2-8b-fc-r
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-1386/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Salesforce/Llama-xLAM-2-8b-fc-r",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-1386/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a9a14054811160af822edc69c6476acd551d7dcfa3224eeaa020cb3474b7ead
+size 54560368

checkpoint-1386/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f553d2a1f9b72fc48bbdef1c15cb90c7622f68c6ad8d3f15513441f361649f1
+size 109267450

checkpoint-1386/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbcc239b407c93a542893777935dc907a3d194df2baa5a175b5e4b536166f8be
+size 14244

checkpoint-1386/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d44021ca2b43e13461b056e55151a436de1268d01c30903cf6d8fa0b508b4c0d
+size 1064

checkpoint-1386/special_tokens_map.json

@@ -0,0 +1,27 @@
+{
+  "additional_special_tokens": [
+    "<|eot_id|>",
+    "<|eom_id|>"
+  ],
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1386/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

checkpoint-1386/tokenizer_config.json

@@ -0,0 +1,2070 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|eot_id|>",
+    "<|eom_id|>"
+  ],
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- Extract system message #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content'] | trim %}\n    {%- set messages = messages[1:] %}\n    {{- system_message + \"\\n\" }}\n{%- else %}\n    {%- set system_message = \"You are a helpful assistant that can use tools. You are developed by Salesforce xLAM team.\" %}\n    {% set format_instruction %}You have access to a set of tools. When using tools, make calls in a single JSON array: \n\n[{\"name\": \"tool_call_name\", \"arguments\": {\"arg1\": \"value1\", \"arg2\": \"value2\"}}, ... (additional parallel tool calls as needed)]\n\nIf no tool is suitable, state that explicitly. If the user's input lacks required parameters, ask for clarification. Do not interpret or respond until tool results are returned. Once they are available, process them or make additional calls if needed. For tasks that don't require tools, such as casual conversation or general advice, respond directly in plain text. The available tools are:{% endset %}\n    {{- system_message + \"\\n\" }}\n    {%- if tools is not none %}\n        {{- format_instruction + \"\\n\\n\" }}\n    {%- endif %}\n{%- endif %}\n\n\n{%- if tools is not none %}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- \"<|eot_id|>\" }}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n        {%- if message['tool_calls'] %}\n            {{- \"[\" }}\n            {%- for tool_call_function in message.tool_calls %}\n                {%- set tool_call = tool_call_function.function %}\n                {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n                {{- '\"arguments\": ' }}\n                {{- tool_call.arguments | tojson }}\n                {{- \"}\" }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n            {%- endfor %}\n            {{- \"]\" }}\n            {{- \"<|eot_id|>\" }}\n        {%- elif message['content'] %}\n            {{- message['content'] | trim + '<|eot_id|>' }}\n        {%- else %}\n            {{- \"[]\\n\" + '<|eot_id|>' }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>\" + \"ipython\" + \"<|end_header_id|>\\n\\n\" }}\n        {%- set content = message[\"content\"] %}\n        {%- if content is mapping or (content is iterable and content is not string) %}\n            {{- content | tojson }}\n        {%- else %}\n            {{- content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 16384,
+  "pad_token": "<|eot_id|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

checkpoint-1386/trainer_state.json

@@ -0,0 +1,532 @@
+{
+  "best_metric": 0.11805504560470581,
+  "best_model_checkpoint": "./xlam_lora_new_2560_1_delete_over_size_3epoch_multi/checkpoint-1386",
+  "epoch": 0.666226040978189,
+  "eval_steps": 693,
+  "global_step": 1386,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.009613651384966653,
+      "grad_norm": 1.1487505435943604,
+      "learning_rate": 1.0638297872340427e-06,
+      "loss": 0.7127,
+      "step": 20
+    },
+    {
+      "epoch": 0.019227302769933306,
+      "grad_norm": 1.1384060382843018,
+      "learning_rate": 2.1276595744680853e-06,
+      "loss": 0.682,
+      "step": 40
+    },
+    {
+      "epoch": 0.02884095415489996,
+      "grad_norm": 0.6661604046821594,
+      "learning_rate": 3.191489361702128e-06,
+      "loss": 0.7262,
+      "step": 60
+    },
+    {
+      "epoch": 0.03845460553986661,
+      "grad_norm": 0.4899628460407257,
+      "learning_rate": 4.255319148936171e-06,
+      "loss": 0.6003,
+      "step": 80
+    },
+    {
+      "epoch": 0.04806825692483326,
+      "grad_norm": 1.2069792747497559,
+      "learning_rate": 5.319148936170213e-06,
+      "loss": 0.5991,
+      "step": 100
+    },
+    {
+      "epoch": 0.05768190830979992,
+      "grad_norm": 1.0019866228103638,
+      "learning_rate": 6.382978723404256e-06,
+      "loss": 0.5028,
+      "step": 120
+    },
+    {
+      "epoch": 0.06729555969476657,
+      "grad_norm": 1.056580662727356,
+      "learning_rate": 7.446808510638298e-06,
+      "loss": 0.433,
+      "step": 140
+    },
+    {
+      "epoch": 0.07690921107973323,
+      "grad_norm": 0.6433275938034058,
+      "learning_rate": 8.510638297872341e-06,
+      "loss": 0.336,
+      "step": 160
+    },
+    {
+      "epoch": 0.08652286246469987,
+      "grad_norm": 0.33471840620040894,
+      "learning_rate": 9.574468085106385e-06,
+      "loss": 0.248,
+      "step": 180
+    },
+    {
+      "epoch": 0.09613651384966652,
+      "grad_norm": 0.5343720316886902,
+      "learning_rate": 9.999902993015826e-06,
+      "loss": 0.2601,
+      "step": 200
+    },
+    {
+      "epoch": 0.10575016523463318,
+      "grad_norm": 0.9687429070472717,
+      "learning_rate": 9.999310186188382e-06,
+      "loss": 0.2559,
+      "step": 220
+    },
+    {
+      "epoch": 0.11536381661959984,
+      "grad_norm": 0.4758259356021881,
+      "learning_rate": 9.998178529120666e-06,
+      "loss": 0.2103,
+      "step": 240
+    },
+    {
+      "epoch": 0.12497746800456648,
+      "grad_norm": 0.782993495464325,
+      "learning_rate": 9.996508143788239e-06,
+      "loss": 0.2466,
+      "step": 260
+    },
+    {
+      "epoch": 0.13459111938953314,
+      "grad_norm": 0.621163010597229,
+      "learning_rate": 9.994299210233451e-06,
+      "loss": 0.2037,
+      "step": 280
+    },
+    {
+      "epoch": 0.14420477077449978,
+      "grad_norm": 0.41969236731529236,
+      "learning_rate": 9.991551966546023e-06,
+      "loss": 0.1842,
+      "step": 300
+    },
+    {
+      "epoch": 0.15381842215946645,
+      "grad_norm": 0.493594765663147,
+      "learning_rate": 9.988266708837393e-06,
+      "loss": 0.1475,
+      "step": 320
+    },
+    {
+      "epoch": 0.1634320735444331,
+      "grad_norm": 1.0131076574325562,
+      "learning_rate": 9.984443791208795e-06,
+      "loss": 0.2276,
+      "step": 340
+    },
+    {
+      "epoch": 0.17304572492939974,
+      "grad_norm": 0.9529913663864136,
+      "learning_rate": 9.98008362571309e-06,
+      "loss": 0.1857,
+      "step": 360
+    },
+    {
+      "epoch": 0.1826593763143664,
+      "grad_norm": 0.7426656484603882,
+      "learning_rate": 9.975186682310364e-06,
+      "loss": 0.1898,
+      "step": 380
+    },
+    {
+      "epoch": 0.19227302769933305,
+      "grad_norm": 0.867158830165863,
+      "learning_rate": 9.969753488817263e-06,
+      "loss": 0.1766,
+      "step": 400
+    },
+    {
+      "epoch": 0.20188667908429972,
+      "grad_norm": 0.7929105162620544,
+      "learning_rate": 9.963784630850099e-06,
+      "loss": 0.2066,
+      "step": 420
+    },
+    {
+      "epoch": 0.21150033046926636,
+      "grad_norm": 0.6801859736442566,
+      "learning_rate": 9.957280751761743e-06,
+      "loss": 0.1975,
+      "step": 440
+    },
+    {
+      "epoch": 0.221113981854233,
+      "grad_norm": 0.9697579145431519,
+      "learning_rate": 9.950242552572272e-06,
+      "loss": 0.1587,
+      "step": 460
+    },
+    {
+      "epoch": 0.23072763323919968,
+      "grad_norm": 0.9611632227897644,
+      "learning_rate": 9.942670791893412e-06,
+      "loss": 0.1772,
+      "step": 480
+    },
+    {
+      "epoch": 0.24034128462416632,
+      "grad_norm": 0.9096260070800781,
+      "learning_rate": 9.934566285846767e-06,
+      "loss": 0.1352,
+      "step": 500
+    },
+    {
+      "epoch": 0.24995493600913296,
+      "grad_norm": 0.8181589245796204,
+      "learning_rate": 9.925929907975864e-06,
+      "loss": 0.1402,
+      "step": 520
+    },
+    {
+      "epoch": 0.2595685873940996,
+      "grad_norm": 0.8003833889961243,
+      "learning_rate": 9.91676258915198e-06,
+      "loss": 0.1882,
+      "step": 540
+    },
+    {
+      "epoch": 0.2691822387790663,
+      "grad_norm": 0.5620806217193604,
+      "learning_rate": 9.907065317473832e-06,
+      "loss": 0.1663,
+      "step": 560
+    },
+    {
+      "epoch": 0.27879589016403294,
+      "grad_norm": 1.0216277837753296,
+      "learning_rate": 9.896839138161053e-06,
+      "loss": 0.1341,
+      "step": 580
+    },
+    {
+      "epoch": 0.28840954154899956,
+      "grad_norm": 0.6821846961975098,
+      "learning_rate": 9.886085153441549e-06,
+      "loss": 0.1953,
+      "step": 600
+    },
+    {
+      "epoch": 0.29802319293396623,
+      "grad_norm": 1.1055521965026855,
+      "learning_rate": 9.87480452243268e-06,
+      "loss": 0.1493,
+      "step": 620
+    },
+    {
+      "epoch": 0.3076368443189329,
+      "grad_norm": 0.7052333950996399,
+      "learning_rate": 9.86299846101634e-06,
+      "loss": 0.2156,
+      "step": 640
+    },
+    {
+      "epoch": 0.3172504957038995,
+      "grad_norm": 0.7262466549873352,
+      "learning_rate": 9.850668241707897e-06,
+      "loss": 0.122,
+      "step": 660
+    },
+    {
+      "epoch": 0.3268641470888662,
+      "grad_norm": 1.1921933889389038,
+      "learning_rate": 9.837815193519031e-06,
+      "loss": 0.1534,
+      "step": 680
+    },
+    {
+      "epoch": 0.3331130204890945,
+      "eval_loss": 0.13637545704841614,
+      "eval_runtime": 206.7827,
+      "eval_samples_per_second": 5.184,
+      "eval_steps_per_second": 5.184,
+      "step": 693
+    },
+    {
+      "epoch": 0.33647779847383286,
+      "grad_norm": 0.8965498208999634,
+      "learning_rate": 9.824440701814491e-06,
+      "loss": 0.206,
+      "step": 700
+    },
+    {
+      "epoch": 0.34609144985879947,
+      "grad_norm": 1.2437098026275635,
+      "learning_rate": 9.810546208162776e-06,
+      "loss": 0.1498,
+      "step": 720
+    },
+    {
+      "epoch": 0.35570510124376614,
+      "grad_norm": 0.823193371295929,
+      "learning_rate": 9.796133210180748e-06,
+      "loss": 0.142,
+      "step": 740
+    },
+    {
+      "epoch": 0.3653187526287328,
+      "grad_norm": 1.1031694412231445,
+      "learning_rate": 9.781203261372216e-06,
+      "loss": 0.1411,
+      "step": 760
+    },
+    {
+      "epoch": 0.3749324040136994,
+      "grad_norm": 0.788958728313446,
+      "learning_rate": 9.765757970960496e-06,
+      "loss": 0.1354,
+      "step": 780
+    },
+    {
+      "epoch": 0.3845460553986661,
+      "grad_norm": 0.3245677351951599,
+      "learning_rate": 9.749799003714954e-06,
+      "loss": 0.1547,
+      "step": 800
+    },
+    {
+      "epoch": 0.39415970678363277,
+      "grad_norm": 0.6731569766998291,
+      "learning_rate": 9.733328079771573e-06,
+      "loss": 0.1158,
+      "step": 820
+    },
+    {
+      "epoch": 0.40377335816859944,
+      "grad_norm": 1.334695816040039,
+      "learning_rate": 9.71634697444754e-06,
+      "loss": 0.1606,
+      "step": 840
+    },
+    {
+      "epoch": 0.41338700955356605,
+      "grad_norm": 1.1176824569702148,
+      "learning_rate": 9.698857518049913e-06,
+      "loss": 0.1659,
+      "step": 860
+    },
+    {
+      "epoch": 0.4230006609385327,
+      "grad_norm": 1.074682593345642,
+      "learning_rate": 9.68086159567832e-06,
+      "loss": 0.1382,
+      "step": 880
+    },
+    {
+      "epoch": 0.4326143123234994,
+      "grad_norm": 0.8198431730270386,
+      "learning_rate": 9.66236114702178e-06,
+      "loss": 0.1431,
+      "step": 900
+    },
+    {
+      "epoch": 0.442227963708466,
+      "grad_norm": 0.26566508412361145,
+      "learning_rate": 9.643358166149646e-06,
+      "loss": 0.1671,
+      "step": 920
+    },
+    {
+      "epoch": 0.4518416150934327,
+      "grad_norm": 1.1486155986785889,
+      "learning_rate": 9.623854701296658e-06,
+      "loss": 0.1302,
+      "step": 940
+    },
+    {
+      "epoch": 0.46145526647839935,
+      "grad_norm": 0.9278470873832703,
+      "learning_rate": 9.603852854642183e-06,
+      "loss": 0.1884,
+      "step": 960
+    },
+    {
+      "epoch": 0.47106891786336597,
+      "grad_norm": 2.031203508377075,
+      "learning_rate": 9.58335478208363e-06,
+      "loss": 0.1672,
+      "step": 980
+    },
+    {
+      "epoch": 0.48068256924833264,
+      "grad_norm": 0.7224697470664978,
+      "learning_rate": 9.562362693004076e-06,
+      "loss": 0.1947,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4902962206332993,
+      "grad_norm": 1.2762720584869385,
+      "learning_rate": 9.54087885003413e-06,
+      "loss": 0.125,
+      "step": 1020
+    },
+    {
+      "epoch": 0.4999098720182659,
+      "grad_norm": 1.0751471519470215,
+      "learning_rate": 9.51890556880805e-06,
+      "loss": 0.1866,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5095235234032326,
+      "grad_norm": 1.022700309753418,
+      "learning_rate": 9.49644521771416e-06,
+      "loss": 0.1377,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5191371747881992,
+      "grad_norm": 1.0384761095046997,
+      "learning_rate": 9.473500217639571e-06,
+      "loss": 0.1275,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5287508261731659,
+      "grad_norm": 1.288801908493042,
+      "learning_rate": 9.450073041709243e-06,
+      "loss": 0.1563,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5383644775581325,
+      "grad_norm": 1.2543052434921265,
+      "learning_rate": 9.426166215019417e-06,
+      "loss": 0.1878,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5479781289430992,
+      "grad_norm": 1.3297594785690308,
+      "learning_rate": 9.401782314365458e-06,
+      "loss": 0.1533,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5575917803280659,
+      "grad_norm": 0.599759042263031,
+      "learning_rate": 9.376923967964108e-06,
+      "loss": 0.1473,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5672054317130325,
+      "grad_norm": 0.18831883370876312,
+      "learning_rate": 9.351593855170204e-06,
+      "loss": 0.1076,
+      "step": 1180
+    },
+    {
+      "epoch": 0.5768190830979991,
+      "grad_norm": 1.0437901020050049,
+      "learning_rate": 9.325794706187885e-06,
+      "loss": 0.1644,
+      "step": 1200
+    },
+    {
+      "epoch": 0.5864327344829658,
+      "grad_norm": 0.46901631355285645,
+      "learning_rate": 9.29952930177632e-06,
+      "loss": 0.0936,
+      "step": 1220
+    },
+    {
+      "epoch": 0.5960463858679325,
+      "grad_norm": 1.486289620399475,
+      "learning_rate": 9.272800472949976e-06,
+      "loss": 0.1931,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6056600372528991,
+      "grad_norm": 0.6040721535682678,
+      "learning_rate": 9.245611100673488e-06,
+      "loss": 0.1397,
+      "step": 1260
+    },
+    {
+      "epoch": 0.6152736886378658,
+      "grad_norm": 0.18753597140312195,
+      "learning_rate": 9.217964115551127e-06,
+      "loss": 0.1169,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6248873400228324,
+      "grad_norm": 0.9620321989059448,
+      "learning_rate": 9.189862497510927e-06,
+      "loss": 0.1573,
+      "step": 1300
+    },
+    {
+      "epoch": 0.634500991407799,
+      "grad_norm": 1.2345646619796753,
+      "learning_rate": 9.161309275483491e-06,
+      "loss": 0.1587,
+      "step": 1320
+    },
+    {
+      "epoch": 0.6441146427927658,
+      "grad_norm": 1.7545924186706543,
+      "learning_rate": 9.132307527075531e-06,
+      "loss": 0.1384,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6537282941777324,
+      "grad_norm": 1.2889124155044556,
+      "learning_rate": 9.102860378238127e-06,
+      "loss": 0.1425,
+      "step": 1360
+    },
+    {
+      "epoch": 0.663341945562699,
+      "grad_norm": 0.955100953578949,
+      "learning_rate": 9.072971002929812e-06,
+      "loss": 0.1375,
+      "step": 1380
+    },
+    {
+      "epoch": 0.666226040978189,
+      "eval_loss": 0.11805504560470581,
+      "eval_runtime": 207.1597,
+      "eval_samples_per_second": 5.175,
+      "eval_steps_per_second": 5.175,
+      "step": 1386
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 6240,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 693,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.033069762772992e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1386/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:228958bc198813ec3c7334720d792de981c3477375cb092239e1e5c068b39593
+size 5624

checkpoint-2079/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Salesforce/Llama-xLAM-2-8b-fc-r
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Salesforce/Llama-xLAM-2-8b-fc-r
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-2079/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Salesforce/Llama-xLAM-2-8b-fc-r",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2079/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36f58889ec23696446d07eac18c5f935199a2487c67d833d5bf0aef3550a099a
+size 54560368

checkpoint-2079/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95b92d6d479811342bd95900d76c1ba5c89f2e313bd9eb9120d2a76f19d2916e
+size 109267450

checkpoint-2079/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7cb827d5e78dc82f6b76994cd2f3db4b847dcae7791a1b6b0a133b40fd259e7d
+size 14244

checkpoint-2079/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbeffe88b661a41e8db2ec5e2c1896fa034bf40716f9b75791c9a4ebc706af27
+size 1064

checkpoint-2079/special_tokens_map.json

@@ -0,0 +1,27 @@
+{
+  "additional_special_tokens": [
+    "<|eot_id|>",
+    "<|eom_id|>"
+  ],
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-2079/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

checkpoint-2079/tokenizer_config.json

@@ -0,0 +1,2070 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|eot_id|>",
+    "<|eom_id|>"
+  ],
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- Extract system message #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content'] | trim %}\n    {%- set messages = messages[1:] %}\n    {{- system_message + \"\\n\" }}\n{%- else %}\n    {%- set system_message = \"You are a helpful assistant that can use tools. You are developed by Salesforce xLAM team.\" %}\n    {% set format_instruction %}You have access to a set of tools. When using tools, make calls in a single JSON array: \n\n[{\"name\": \"tool_call_name\", \"arguments\": {\"arg1\": \"value1\", \"arg2\": \"value2\"}}, ... (additional parallel tool calls as needed)]\n\nIf no tool is suitable, state that explicitly. If the user's input lacks required parameters, ask for clarification. Do not interpret or respond until tool results are returned. Once they are available, process them or make additional calls if needed. For tasks that don't require tools, such as casual conversation or general advice, respond directly in plain text. The available tools are:{% endset %}\n    {{- system_message + \"\\n\" }}\n    {%- if tools is not none %}\n        {{- format_instruction + \"\\n\\n\" }}\n    {%- endif %}\n{%- endif %}\n\n\n{%- if tools is not none %}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- \"<|eot_id|>\" }}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n        {%- if message['tool_calls'] %}\n            {{- \"[\" }}\n            {%- for tool_call_function in message.tool_calls %}\n                {%- set tool_call = tool_call_function.function %}\n                {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n                {{- '\"arguments\": ' }}\n                {{- tool_call.arguments | tojson }}\n                {{- \"}\" }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n            {%- endfor %}\n            {{- \"]\" }}\n            {{- \"<|eot_id|>\" }}\n        {%- elif message['content'] %}\n            {{- message['content'] | trim + '<|eot_id|>' }}\n        {%- else %}\n            {{- \"[]\\n\" + '<|eot_id|>' }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>\" + \"ipython\" + \"<|end_header_id|>\\n\\n\" }}\n        {%- set content = message[\"content\"] %}\n        {%- if content is mapping or (content is iterable and content is not string) %}\n            {{- content | tojson }}\n        {%- else %}\n            {{- content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 16384,
+  "pad_token": "<|eot_id|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

checkpoint-2079/trainer_state.json

@@ -0,0 +1,778 @@
+{
+  "best_metric": 0.11032938957214355,
+  "best_model_checkpoint": "./xlam_lora_new_2560_1_delete_over_size_3epoch_multi/checkpoint-2079",
+  "epoch": 0.9993390614672836,
+  "eval_steps": 693,
+  "global_step": 2079,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.009613651384966653,
+      "grad_norm": 1.1487505435943604,
+      "learning_rate": 1.0638297872340427e-06,
+      "loss": 0.7127,
+      "step": 20
+    },
+    {
+      "epoch": 0.019227302769933306,
+      "grad_norm": 1.1384060382843018,
+      "learning_rate": 2.1276595744680853e-06,
+      "loss": 0.682,
+      "step": 40
+    },
+    {
+      "epoch": 0.02884095415489996,
+      "grad_norm": 0.6661604046821594,
+      "learning_rate": 3.191489361702128e-06,
+      "loss": 0.7262,
+      "step": 60
+    },
+    {
+      "epoch": 0.03845460553986661,
+      "grad_norm": 0.4899628460407257,
+      "learning_rate": 4.255319148936171e-06,
+      "loss": 0.6003,
+      "step": 80
+    },
+    {
+      "epoch": 0.04806825692483326,
+      "grad_norm": 1.2069792747497559,
+      "learning_rate": 5.319148936170213e-06,
+      "loss": 0.5991,
+      "step": 100
+    },
+    {
+      "epoch": 0.05768190830979992,
+      "grad_norm": 1.0019866228103638,
+      "learning_rate": 6.382978723404256e-06,
+      "loss": 0.5028,
+      "step": 120
+    },
+    {
+      "epoch": 0.06729555969476657,
+      "grad_norm": 1.056580662727356,
+      "learning_rate": 7.446808510638298e-06,
+      "loss": 0.433,
+      "step": 140
+    },
+    {
+      "epoch": 0.07690921107973323,
+      "grad_norm": 0.6433275938034058,
+      "learning_rate": 8.510638297872341e-06,
+      "loss": 0.336,
+      "step": 160
+    },
+    {
+      "epoch": 0.08652286246469987,
+      "grad_norm": 0.33471840620040894,
+      "learning_rate": 9.574468085106385e-06,
+      "loss": 0.248,
+      "step": 180
+    },
+    {
+      "epoch": 0.09613651384966652,
+      "grad_norm": 0.5343720316886902,
+      "learning_rate": 9.999902993015826e-06,
+      "loss": 0.2601,
+      "step": 200
+    },
+    {
+      "epoch": 0.10575016523463318,
+      "grad_norm": 0.9687429070472717,
+      "learning_rate": 9.999310186188382e-06,
+      "loss": 0.2559,
+      "step": 220
+    },
+    {
+      "epoch": 0.11536381661959984,
+      "grad_norm": 0.4758259356021881,
+      "learning_rate": 9.998178529120666e-06,
+      "loss": 0.2103,
+      "step": 240
+    },
+    {
+      "epoch": 0.12497746800456648,
+      "grad_norm": 0.782993495464325,
+      "learning_rate": 9.996508143788239e-06,
+      "loss": 0.2466,
+      "step": 260
+    },
+    {
+      "epoch": 0.13459111938953314,
+      "grad_norm": 0.621163010597229,
+      "learning_rate": 9.994299210233451e-06,
+      "loss": 0.2037,
+      "step": 280
+    },
+    {
+      "epoch": 0.14420477077449978,
+      "grad_norm": 0.41969236731529236,
+      "learning_rate": 9.991551966546023e-06,
+      "loss": 0.1842,
+      "step": 300
+    },
+    {
+      "epoch": 0.15381842215946645,
+      "grad_norm": 0.493594765663147,
+      "learning_rate": 9.988266708837393e-06,
+      "loss": 0.1475,
+      "step": 320
+    },
+    {
+      "epoch": 0.1634320735444331,
+      "grad_norm": 1.0131076574325562,
+      "learning_rate": 9.984443791208795e-06,
+      "loss": 0.2276,
+      "step": 340
+    },
+    {
+      "epoch": 0.17304572492939974,
+      "grad_norm": 0.9529913663864136,
+      "learning_rate": 9.98008362571309e-06,
+      "loss": 0.1857,
+      "step": 360
+    },
+    {
+      "epoch": 0.1826593763143664,
+      "grad_norm": 0.7426656484603882,
+      "learning_rate": 9.975186682310364e-06,
+      "loss": 0.1898,
+      "step": 380
+    },
+    {
+      "epoch": 0.19227302769933305,
+      "grad_norm": 0.867158830165863,
+      "learning_rate": 9.969753488817263e-06,
+      "loss": 0.1766,
+      "step": 400
+    },
+    {
+      "epoch": 0.20188667908429972,
+      "grad_norm": 0.7929105162620544,
+      "learning_rate": 9.963784630850099e-06,
+      "loss": 0.2066,
+      "step": 420
+    },
+    {
+      "epoch": 0.21150033046926636,
+      "grad_norm": 0.6801859736442566,
+      "learning_rate": 9.957280751761743e-06,
+      "loss": 0.1975,
+      "step": 440
+    },
+    {
+      "epoch": 0.221113981854233,
+      "grad_norm": 0.9697579145431519,
+      "learning_rate": 9.950242552572272e-06,
+      "loss": 0.1587,
+      "step": 460
+    },
+    {
+      "epoch": 0.23072763323919968,
+      "grad_norm": 0.9611632227897644,
+      "learning_rate": 9.942670791893412e-06,
+      "loss": 0.1772,
+      "step": 480
+    },
+    {
+      "epoch": 0.24034128462416632,
+      "grad_norm": 0.9096260070800781,
+      "learning_rate": 9.934566285846767e-06,
+      "loss": 0.1352,
+      "step": 500
+    },
+    {
+      "epoch": 0.24995493600913296,
+      "grad_norm": 0.8181589245796204,
+      "learning_rate": 9.925929907975864e-06,
+      "loss": 0.1402,
+      "step": 520
+    },
+    {
+      "epoch": 0.2595685873940996,
+      "grad_norm": 0.8003833889961243,
+      "learning_rate": 9.91676258915198e-06,
+      "loss": 0.1882,
+      "step": 540
+    },
+    {
+      "epoch": 0.2691822387790663,
+      "grad_norm": 0.5620806217193604,
+      "learning_rate": 9.907065317473832e-06,
+      "loss": 0.1663,
+      "step": 560
+    },
+    {
+      "epoch": 0.27879589016403294,
+      "grad_norm": 1.0216277837753296,
+      "learning_rate": 9.896839138161053e-06,
+      "loss": 0.1341,
+      "step": 580
+    },
+    {
+      "epoch": 0.28840954154899956,
+      "grad_norm": 0.6821846961975098,
+      "learning_rate": 9.886085153441549e-06,
+      "loss": 0.1953,
+      "step": 600
+    },
+    {
+      "epoch": 0.29802319293396623,
+      "grad_norm": 1.1055521965026855,
+      "learning_rate": 9.87480452243268e-06,
+      "loss": 0.1493,
+      "step": 620
+    },
+    {
+      "epoch": 0.3076368443189329,
+      "grad_norm": 0.7052333950996399,
+      "learning_rate": 9.86299846101634e-06,
+      "loss": 0.2156,
+      "step": 640
+    },
+    {
+      "epoch": 0.3172504957038995,
+      "grad_norm": 0.7262466549873352,
+      "learning_rate": 9.850668241707897e-06,
+      "loss": 0.122,
+      "step": 660
+    },
+    {
+      "epoch": 0.3268641470888662,
+      "grad_norm": 1.1921933889389038,
+      "learning_rate": 9.837815193519031e-06,
+      "loss": 0.1534,
+      "step": 680
+    },
+    {
+      "epoch": 0.3331130204890945,
+      "eval_loss": 0.13637545704841614,
+      "eval_runtime": 206.7827,
+      "eval_samples_per_second": 5.184,
+      "eval_steps_per_second": 5.184,
+      "step": 693
+    },
+    {
+      "epoch": 0.33647779847383286,
+      "grad_norm": 0.8965498208999634,
+      "learning_rate": 9.824440701814491e-06,
+      "loss": 0.206,
+      "step": 700
+    },
+    {
+      "epoch": 0.34609144985879947,
+      "grad_norm": 1.2437098026275635,
+      "learning_rate": 9.810546208162776e-06,
+      "loss": 0.1498,
+      "step": 720
+    },
+    {
+      "epoch": 0.35570510124376614,
+      "grad_norm": 0.823193371295929,
+      "learning_rate": 9.796133210180748e-06,
+      "loss": 0.142,
+      "step": 740
+    },
+    {
+      "epoch": 0.3653187526287328,
+      "grad_norm": 1.1031694412231445,
+      "learning_rate": 9.781203261372216e-06,
+      "loss": 0.1411,
+      "step": 760
+    },
+    {
+      "epoch": 0.3749324040136994,
+      "grad_norm": 0.788958728313446,
+      "learning_rate": 9.765757970960496e-06,
+      "loss": 0.1354,
+      "step": 780
+    },
+    {
+      "epoch": 0.3845460553986661,
+      "grad_norm": 0.3245677351951599,
+      "learning_rate": 9.749799003714954e-06,
+      "loss": 0.1547,
+      "step": 800
+    },
+    {
+      "epoch": 0.39415970678363277,
+      "grad_norm": 0.6731569766998291,
+      "learning_rate": 9.733328079771573e-06,
+      "loss": 0.1158,
+      "step": 820
+    },
+    {
+      "epoch": 0.40377335816859944,
+      "grad_norm": 1.334695816040039,
+      "learning_rate": 9.71634697444754e-06,
+      "loss": 0.1606,
+      "step": 840
+    },
+    {
+      "epoch": 0.41338700955356605,
+      "grad_norm": 1.1176824569702148,
+      "learning_rate": 9.698857518049913e-06,
+      "loss": 0.1659,
+      "step": 860
+    },
+    {
+      "epoch": 0.4230006609385327,
+      "grad_norm": 1.074682593345642,
+      "learning_rate": 9.68086159567832e-06,
+      "loss": 0.1382,
+      "step": 880
+    },
+    {
+      "epoch": 0.4326143123234994,
+      "grad_norm": 0.8198431730270386,
+      "learning_rate": 9.66236114702178e-06,
+      "loss": 0.1431,
+      "step": 900
+    },
+    {
+      "epoch": 0.442227963708466,
+      "grad_norm": 0.26566508412361145,
+      "learning_rate": 9.643358166149646e-06,
+      "loss": 0.1671,
+      "step": 920
+    },
+    {
+      "epoch": 0.4518416150934327,
+      "grad_norm": 1.1486155986785889,
+      "learning_rate": 9.623854701296658e-06,
+      "loss": 0.1302,
+      "step": 940
+    },
+    {
+      "epoch": 0.46145526647839935,
+      "grad_norm": 0.9278470873832703,
+      "learning_rate": 9.603852854642183e-06,
+      "loss": 0.1884,
+      "step": 960
+    },
+    {
+      "epoch": 0.47106891786336597,
+      "grad_norm": 2.031203508377075,
+      "learning_rate": 9.58335478208363e-06,
+      "loss": 0.1672,
+      "step": 980
+    },
+    {
+      "epoch": 0.48068256924833264,
+      "grad_norm": 0.7224697470664978,
+      "learning_rate": 9.562362693004076e-06,
+      "loss": 0.1947,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4902962206332993,
+      "grad_norm": 1.2762720584869385,
+      "learning_rate": 9.54087885003413e-06,
+      "loss": 0.125,
+      "step": 1020
+    },
+    {
+      "epoch": 0.4999098720182659,
+      "grad_norm": 1.0751471519470215,
+      "learning_rate": 9.51890556880805e-06,
+      "loss": 0.1866,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5095235234032326,
+      "grad_norm": 1.022700309753418,
+      "learning_rate": 9.49644521771416e-06,
+      "loss": 0.1377,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5191371747881992,
+      "grad_norm": 1.0384761095046997,
+      "learning_rate": 9.473500217639571e-06,
+      "loss": 0.1275,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5287508261731659,
+      "grad_norm": 1.288801908493042,
+      "learning_rate": 9.450073041709243e-06,
+      "loss": 0.1563,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5383644775581325,
+      "grad_norm": 1.2543052434921265,
+      "learning_rate": 9.426166215019417e-06,
+      "loss": 0.1878,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5479781289430992,
+      "grad_norm": 1.3297594785690308,
+      "learning_rate": 9.401782314365458e-06,
+      "loss": 0.1533,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5575917803280659,
+      "grad_norm": 0.599759042263031,
+      "learning_rate": 9.376923967964108e-06,
+      "loss": 0.1473,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5672054317130325,
+      "grad_norm": 0.18831883370876312,
+      "learning_rate": 9.351593855170204e-06,
+      "loss": 0.1076,
+      "step": 1180
+    },
+    {
+      "epoch": 0.5768190830979991,
+      "grad_norm": 1.0437901020050049,
+      "learning_rate": 9.325794706187885e-06,
+      "loss": 0.1644,
+      "step": 1200
+    },
+    {
+      "epoch": 0.5864327344829658,
+      "grad_norm": 0.46901631355285645,
+      "learning_rate": 9.29952930177632e-06,
+      "loss": 0.0936,
+      "step": 1220
+    },
+    {
+      "epoch": 0.5960463858679325,
+      "grad_norm": 1.486289620399475,
+      "learning_rate": 9.272800472949976e-06,
+      "loss": 0.1931,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6056600372528991,
+      "grad_norm": 0.6040721535682678,
+      "learning_rate": 9.245611100673488e-06,
+      "loss": 0.1397,
+      "step": 1260
+    },
+    {
+      "epoch": 0.6152736886378658,
+      "grad_norm": 0.18753597140312195,
+      "learning_rate": 9.217964115551127e-06,
+      "loss": 0.1169,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6248873400228324,
+      "grad_norm": 0.9620321989059448,
+      "learning_rate": 9.189862497510927e-06,
+      "loss": 0.1573,
+      "step": 1300
+    },
+    {
+      "epoch": 0.634500991407799,
+      "grad_norm": 1.2345646619796753,
+      "learning_rate": 9.161309275483491e-06,
+      "loss": 0.1587,
+      "step": 1320
+    },
+    {
+      "epoch": 0.6441146427927658,
+      "grad_norm": 1.7545924186706543,
+      "learning_rate": 9.132307527075531e-06,
+      "loss": 0.1384,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6537282941777324,
+      "grad_norm": 1.2889124155044556,
+      "learning_rate": 9.102860378238127e-06,
+      "loss": 0.1425,
+      "step": 1360
+    },
+    {
+      "epoch": 0.663341945562699,
+      "grad_norm": 0.955100953578949,
+      "learning_rate": 9.072971002929812e-06,
+      "loss": 0.1375,
+      "step": 1380
+    },
+    {
+      "epoch": 0.666226040978189,
+      "eval_loss": 0.11805504560470581,
+      "eval_runtime": 207.1597,
+      "eval_samples_per_second": 5.175,
+      "eval_steps_per_second": 5.175,
+      "step": 1386
+    },
+    {
+      "epoch": 0.6729555969476657,
+      "grad_norm": 0.9628621339797974,
+      "learning_rate": 9.042642622774465e-06,
+      "loss": 0.122,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6825692483326323,
+      "grad_norm": 0.8323156833648682,
+      "learning_rate": 9.011878506714065e-06,
+      "loss": 0.142,
+      "step": 1420
+    },
+    {
+      "epoch": 0.6921828997175989,
+      "grad_norm": 1.0403231382369995,
+      "learning_rate": 8.980681970656349e-06,
+      "loss": 0.1435,
+      "step": 1440
+    },
+    {
+      "epoch": 0.7017965511025657,
+      "grad_norm": 1.2075049877166748,
+      "learning_rate": 8.949056377117408e-06,
+      "loss": 0.1236,
+      "step": 1460
+    },
+    {
+      "epoch": 0.7114102024875323,
+      "grad_norm": 1.3533861637115479,
+      "learning_rate": 8.917005134859263e-06,
+      "loss": 0.1378,
+      "step": 1480
+    },
+    {
+      "epoch": 0.7210238538724989,
+      "grad_norm": 1.1251810789108276,
+      "learning_rate": 8.884531698522443e-06,
+      "loss": 0.1413,
+      "step": 1500
+    },
+    {
+      "epoch": 0.7306375052574656,
+      "grad_norm": 1.074367880821228,
+      "learning_rate": 8.85163956825363e-06,
+      "loss": 0.1685,
+      "step": 1520
+    },
+    {
+      "epoch": 0.7402511566424322,
+      "grad_norm": 1.4021315574645996,
+      "learning_rate": 8.818332289328402e-06,
+      "loss": 0.1299,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7498648080273989,
+      "grad_norm": 1.6010384559631348,
+      "learning_rate": 8.784613451769098e-06,
+      "loss": 0.1602,
+      "step": 1560
+    },
+    {
+      "epoch": 0.7594784594123656,
+      "grad_norm": 1.1963365077972412,
+      "learning_rate": 8.750486689957874e-06,
+      "loss": 0.1677,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7690921107973322,
+      "grad_norm": 1.3554548025131226,
+      "learning_rate": 8.71595568224496e-06,
+      "loss": 0.1462,
+      "step": 1600
+    },
+    {
+      "epoch": 0.7787057621822988,
+      "grad_norm": 0.5640055537223816,
+      "learning_rate": 8.681024150552208e-06,
+      "loss": 0.1339,
+      "step": 1620
+    },
+    {
+      "epoch": 0.7883194135672655,
+      "grad_norm": 1.2408941984176636,
+      "learning_rate": 8.64569585997191e-06,
+      "loss": 0.1313,
+      "step": 1640
+    },
+    {
+      "epoch": 0.7979330649522322,
+      "grad_norm": 1.0671769380569458,
+      "learning_rate": 8.609974618360978e-06,
+      "loss": 0.1757,
+      "step": 1660
+    },
+    {
+      "epoch": 0.8075467163371989,
+      "grad_norm": 1.298189401626587,
+      "learning_rate": 8.573864275930526e-06,
+      "loss": 0.1833,
+      "step": 1680
+    },
+    {
+      "epoch": 0.8171603677221655,
+      "grad_norm": 0.5296375155448914,
+      "learning_rate": 8.537368724830865e-06,
+      "loss": 0.1517,
+      "step": 1700
+    },
+    {
+      "epoch": 0.8267740191071321,
+      "grad_norm": 1.1902680397033691,
+      "learning_rate": 8.50049189873199e-06,
+      "loss": 0.1231,
+      "step": 1720
+    },
+    {
+      "epoch": 0.8363876704920988,
+      "grad_norm": 0.0974334254860878,
+      "learning_rate": 8.463237772399589e-06,
+      "loss": 0.0947,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8460013218770654,
+      "grad_norm": 1.0790835618972778,
+      "learning_rate": 8.42561036126663e-06,
+      "loss": 0.1616,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8556149732620321,
+      "grad_norm": 0.5958455204963684,
+      "learning_rate": 8.38761372100055e-06,
+      "loss": 0.1409,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8652286246469988,
+      "grad_norm": 1.6728020906448364,
+      "learning_rate": 8.349251947066118e-06,
+      "loss": 0.137,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8748422760319654,
+      "grad_norm": 1.0818266868591309,
+      "learning_rate": 8.310529174284004e-06,
+      "loss": 0.1462,
+      "step": 1820
+    },
+    {
+      "epoch": 0.884455927416932,
+      "grad_norm": 0.9389984607696533,
+      "learning_rate": 8.271449576385112e-06,
+      "loss": 0.1283,
+      "step": 1840
+    },
+    {
+      "epoch": 0.8940695788018987,
+      "grad_norm": 1.3795585632324219,
+      "learning_rate": 8.23201736556071e-06,
+      "loss": 0.1292,
+      "step": 1860
+    },
+    {
+      "epoch": 0.9036832301868654,
+      "grad_norm": 1.248744249343872,
+      "learning_rate": 8.192236792008426e-06,
+      "loss": 0.1447,
+      "step": 1880
+    },
+    {
+      "epoch": 0.913296881571832,
+      "grad_norm": 1.3501381874084473,
+      "learning_rate": 8.15211214347413e-06,
+      "loss": 0.1285,
+      "step": 1900
+    },
+    {
+      "epoch": 0.9229105329567987,
+      "grad_norm": 1.1559216976165771,
+      "learning_rate": 8.111647744789788e-06,
+      "loss": 0.1033,
+      "step": 1920
+    },
+    {
+      "epoch": 0.9325241843417653,
+      "grad_norm": 1.5532070398330688,
+      "learning_rate": 8.07084795740731e-06,
+      "loss": 0.1457,
+      "step": 1940
+    },
+    {
+      "epoch": 0.9421378357267319,
+      "grad_norm": 0.901100218296051,
+      "learning_rate": 8.029717178928446e-06,
+      "loss": 0.1368,
+      "step": 1960
+    },
+    {
+      "epoch": 0.9517514871116987,
+      "grad_norm": 1.8944685459136963,
+      "learning_rate": 7.9882598426308e-06,
+      "loss": 0.1213,
+      "step": 1980
+    },
+    {
+      "epoch": 0.9613651384966653,
+      "grad_norm": 1.8067469596862793,
+      "learning_rate": 7.946480416989978e-06,
+      "loss": 0.1269,
+      "step": 2000
+    },
+    {
+      "epoch": 0.9709787898816319,
+      "grad_norm": 1.817403793334961,
+      "learning_rate": 7.904383405197962e-06,
+      "loss": 0.1724,
+      "step": 2020
+    },
+    {
+      "epoch": 0.9805924412665986,
+      "grad_norm": 1.2046632766723633,
+      "learning_rate": 7.86197334467773e-06,
+      "loss": 0.1406,
+      "step": 2040
+    },
+    {
+      "epoch": 0.9902060926515652,
+      "grad_norm": 1.3356693983078003,
+      "learning_rate": 7.819254806594194e-06,
+      "loss": 0.1488,
+      "step": 2060
+    },
+    {
+      "epoch": 0.9993390614672836,
+      "eval_loss": 0.11032938957214355,
+      "eval_runtime": 207.1464,
+      "eval_samples_per_second": 5.175,
+      "eval_steps_per_second": 5.175,
+      "step": 2079
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 6240,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 693,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.5386516115456e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2079/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:228958bc198813ec3c7334720d792de981c3477375cb092239e1e5c068b39593
+size 5624

checkpoint-2772/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Salesforce/Llama-xLAM-2-8b-fc-r
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Salesforce/Llama-xLAM-2-8b-fc-r
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-2772/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Salesforce/Llama-xLAM-2-8b-fc-r",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-2772/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:757310f0fb9c41f8fcaaf9836f924263219e2ec87f044ebe3b69b80a98ad63d5
+size 54560368

checkpoint-2772/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eeb110b962cf3431c68056f57c9691c61c026e70eaf6e333cfc0e36510b45568
+size 109267450

checkpoint-2772/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:169ad2930fbb28aad514c156889ecbdb4c952335bf00f9a9bf957706b2afeaef
+size 14244

checkpoint-2772/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd237bbb82cbb328db320b57c9181b7b1032160b9bf647b34d9634039f01a812
+size 1064

checkpoint-2772/special_tokens_map.json

@@ -0,0 +1,27 @@
+{
+  "additional_special_tokens": [
+    "<|eot_id|>",
+    "<|eom_id|>"
+  ],
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-2772/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

checkpoint-2772/tokenizer_config.json

@@ -0,0 +1,2070 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|eot_id|>",
+    "<|eom_id|>"
+  ],
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- Extract system message #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content'] | trim %}\n    {%- set messages = messages[1:] %}\n    {{- system_message + \"\\n\" }}\n{%- else %}\n    {%- set system_message = \"You are a helpful assistant that can use tools. You are developed by Salesforce xLAM team.\" %}\n    {% set format_instruction %}You have access to a set of tools. When using tools, make calls in a single JSON array: \n\n[{\"name\": \"tool_call_name\", \"arguments\": {\"arg1\": \"value1\", \"arg2\": \"value2\"}}, ... (additional parallel tool calls as needed)]\n\nIf no tool is suitable, state that explicitly. If the user's input lacks required parameters, ask for clarification. Do not interpret or respond until tool results are returned. Once they are available, process them or make additional calls if needed. For tasks that don't require tools, such as casual conversation or general advice, respond directly in plain text. The available tools are:{% endset %}\n    {{- system_message + \"\\n\" }}\n    {%- if tools is not none %}\n        {{- format_instruction + \"\\n\\n\" }}\n    {%- endif %}\n{%- endif %}\n\n\n{%- if tools is not none %}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- \"<|eot_id|>\" }}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n        {%- if message['tool_calls'] %}\n            {{- \"[\" }}\n            {%- for tool_call_function in message.tool_calls %}\n                {%- set tool_call = tool_call_function.function %}\n                {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n                {{- '\"arguments\": ' }}\n                {{- tool_call.arguments | tojson }}\n                {{- \"}\" }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n            {%- endfor %}\n            {{- \"]\" }}\n            {{- \"<|eot_id|>\" }}\n        {%- elif message['content'] %}\n            {{- message['content'] | trim + '<|eot_id|>' }}\n        {%- else %}\n            {{- \"[]\\n\" + '<|eot_id|>' }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>\" + \"ipython\" + \"<|end_header_id|>\\n\\n\" }}\n        {%- set content = message[\"content\"] %}\n        {%- if content is mapping or (content is iterable and content is not string) %}\n            {{- content | tojson }}\n        {%- else %}\n            {{- content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 16384,
+  "pad_token": "<|eot_id|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

checkpoint-2772/trainer_state.json

@@ -0,0 +1,1031 @@
+{
+  "best_metric": 0.10718663781881332,
+  "best_model_checkpoint": "./xlam_lora_new_2560_1_delete_over_size_3epoch_multi/checkpoint-2772",
+  "epoch": 1.332452081956378,
+  "eval_steps": 693,
+  "global_step": 2772,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.009613651384966653,
+      "grad_norm": 1.1487505435943604,
+      "learning_rate": 1.0638297872340427e-06,
+      "loss": 0.7127,
+      "step": 20
+    },
+    {
+      "epoch": 0.019227302769933306,
+      "grad_norm": 1.1384060382843018,
+      "learning_rate": 2.1276595744680853e-06,
+      "loss": 0.682,
+      "step": 40
+    },
+    {
+      "epoch": 0.02884095415489996,
+      "grad_norm": 0.6661604046821594,
+      "learning_rate": 3.191489361702128e-06,
+      "loss": 0.7262,
+      "step": 60
+    },
+    {
+      "epoch": 0.03845460553986661,
+      "grad_norm": 0.4899628460407257,
+      "learning_rate": 4.255319148936171e-06,
+      "loss": 0.6003,
+      "step": 80
+    },
+    {
+      "epoch": 0.04806825692483326,
+      "grad_norm": 1.2069792747497559,
+      "learning_rate": 5.319148936170213e-06,
+      "loss": 0.5991,
+      "step": 100
+    },
+    {
+      "epoch": 0.05768190830979992,
+      "grad_norm": 1.0019866228103638,
+      "learning_rate": 6.382978723404256e-06,
+      "loss": 0.5028,
+      "step": 120
+    },
+    {
+      "epoch": 0.06729555969476657,
+      "grad_norm": 1.056580662727356,
+      "learning_rate": 7.446808510638298e-06,
+      "loss": 0.433,
+      "step": 140
+    },
+    {
+      "epoch": 0.07690921107973323,
+      "grad_norm": 0.6433275938034058,
+      "learning_rate": 8.510638297872341e-06,
+      "loss": 0.336,
+      "step": 160
+    },
+    {
+      "epoch": 0.08652286246469987,
+      "grad_norm": 0.33471840620040894,
+      "learning_rate": 9.574468085106385e-06,
+      "loss": 0.248,
+      "step": 180
+    },
+    {
+      "epoch": 0.09613651384966652,
+      "grad_norm": 0.5343720316886902,
+      "learning_rate": 9.999902993015826e-06,
+      "loss": 0.2601,
+      "step": 200
+    },
+    {
+      "epoch": 0.10575016523463318,
+      "grad_norm": 0.9687429070472717,
+      "learning_rate": 9.999310186188382e-06,
+      "loss": 0.2559,
+      "step": 220
+    },
+    {
+      "epoch": 0.11536381661959984,
+      "grad_norm": 0.4758259356021881,
+      "learning_rate": 9.998178529120666e-06,
+      "loss": 0.2103,
+      "step": 240
+    },
+    {
+      "epoch": 0.12497746800456648,
+      "grad_norm": 0.782993495464325,
+      "learning_rate": 9.996508143788239e-06,
+      "loss": 0.2466,
+      "step": 260
+    },
+    {
+      "epoch": 0.13459111938953314,
+      "grad_norm": 0.621163010597229,
+      "learning_rate": 9.994299210233451e-06,
+      "loss": 0.2037,
+      "step": 280
+    },
+    {
+      "epoch": 0.14420477077449978,
+      "grad_norm": 0.41969236731529236,
+      "learning_rate": 9.991551966546023e-06,
+      "loss": 0.1842,
+      "step": 300
+    },
+    {
+      "epoch": 0.15381842215946645,
+      "grad_norm": 0.493594765663147,
+      "learning_rate": 9.988266708837393e-06,
+      "loss": 0.1475,
+      "step": 320
+    },
+    {
+      "epoch": 0.1634320735444331,
+      "grad_norm": 1.0131076574325562,
+      "learning_rate": 9.984443791208795e-06,
+      "loss": 0.2276,
+      "step": 340
+    },
+    {
+      "epoch": 0.17304572492939974,
+      "grad_norm": 0.9529913663864136,
+      "learning_rate": 9.98008362571309e-06,
+      "loss": 0.1857,
+      "step": 360
+    },
+    {
+      "epoch": 0.1826593763143664,
+      "grad_norm": 0.7426656484603882,
+      "learning_rate": 9.975186682310364e-06,
+      "loss": 0.1898,
+      "step": 380
+    },
+    {
+      "epoch": 0.19227302769933305,
+      "grad_norm": 0.867158830165863,
+      "learning_rate": 9.969753488817263e-06,
+      "loss": 0.1766,
+      "step": 400
+    },
+    {
+      "epoch": 0.20188667908429972,
+      "grad_norm": 0.7929105162620544,
+      "learning_rate": 9.963784630850099e-06,
+      "loss": 0.2066,
+      "step": 420
+    },
+    {
+      "epoch": 0.21150033046926636,
+      "grad_norm": 0.6801859736442566,
+      "learning_rate": 9.957280751761743e-06,
+      "loss": 0.1975,
+      "step": 440
+    },
+    {
+      "epoch": 0.221113981854233,
+      "grad_norm": 0.9697579145431519,
+      "learning_rate": 9.950242552572272e-06,
+      "loss": 0.1587,
+      "step": 460
+    },
+    {
+      "epoch": 0.23072763323919968,
+      "grad_norm": 0.9611632227897644,
+      "learning_rate": 9.942670791893412e-06,
+      "loss": 0.1772,
+      "step": 480
+    },
+    {
+      "epoch": 0.24034128462416632,
+      "grad_norm": 0.9096260070800781,
+      "learning_rate": 9.934566285846767e-06,
+      "loss": 0.1352,
+      "step": 500
+    },
+    {
+      "epoch": 0.24995493600913296,
+      "grad_norm": 0.8181589245796204,
+      "learning_rate": 9.925929907975864e-06,
+      "loss": 0.1402,
+      "step": 520
+    },
+    {
+      "epoch": 0.2595685873940996,
+      "grad_norm": 0.8003833889961243,
+      "learning_rate": 9.91676258915198e-06,
+      "loss": 0.1882,
+      "step": 540
+    },
+    {
+      "epoch": 0.2691822387790663,
+      "grad_norm": 0.5620806217193604,
+      "learning_rate": 9.907065317473832e-06,
+      "loss": 0.1663,
+      "step": 560
+    },
+    {
+      "epoch": 0.27879589016403294,
+      "grad_norm": 1.0216277837753296,
+      "learning_rate": 9.896839138161053e-06,
+      "loss": 0.1341,
+      "step": 580
+    },
+    {
+      "epoch": 0.28840954154899956,
+      "grad_norm": 0.6821846961975098,
+      "learning_rate": 9.886085153441549e-06,
+      "loss": 0.1953,
+      "step": 600
+    },
+    {
+      "epoch": 0.29802319293396623,
+      "grad_norm": 1.1055521965026855,
+      "learning_rate": 9.87480452243268e-06,
+      "loss": 0.1493,
+      "step": 620
+    },
+    {
+      "epoch": 0.3076368443189329,
+      "grad_norm": 0.7052333950996399,
+      "learning_rate": 9.86299846101634e-06,
+      "loss": 0.2156,
+      "step": 640
+    },
+    {
+      "epoch": 0.3172504957038995,
+      "grad_norm": 0.7262466549873352,
+      "learning_rate": 9.850668241707897e-06,
+      "loss": 0.122,
+      "step": 660
+    },
+    {
+      "epoch": 0.3268641470888662,
+      "grad_norm": 1.1921933889389038,
+      "learning_rate": 9.837815193519031e-06,
+      "loss": 0.1534,
+      "step": 680
+    },
+    {
+      "epoch": 0.3331130204890945,
+      "eval_loss": 0.13637545704841614,
+      "eval_runtime": 206.7827,
+      "eval_samples_per_second": 5.184,
+      "eval_steps_per_second": 5.184,
+      "step": 693
+    },
+    {
+      "epoch": 0.33647779847383286,
+      "grad_norm": 0.8965498208999634,
+      "learning_rate": 9.824440701814491e-06,
+      "loss": 0.206,
+      "step": 700
+    },
+    {
+      "epoch": 0.34609144985879947,
+      "grad_norm": 1.2437098026275635,
+      "learning_rate": 9.810546208162776e-06,
+      "loss": 0.1498,
+      "step": 720
+    },
+    {
+      "epoch": 0.35570510124376614,
+      "grad_norm": 0.823193371295929,
+      "learning_rate": 9.796133210180748e-06,
+      "loss": 0.142,
+      "step": 740
+    },
+    {
+      "epoch": 0.3653187526287328,
+      "grad_norm": 1.1031694412231445,
+      "learning_rate": 9.781203261372216e-06,
+      "loss": 0.1411,
+      "step": 760
+    },
+    {
+      "epoch": 0.3749324040136994,
+      "grad_norm": 0.788958728313446,
+      "learning_rate": 9.765757970960496e-06,
+      "loss": 0.1354,
+      "step": 780
+    },
+    {
+      "epoch": 0.3845460553986661,
+      "grad_norm": 0.3245677351951599,
+      "learning_rate": 9.749799003714954e-06,
+      "loss": 0.1547,
+      "step": 800
+    },
+    {
+      "epoch": 0.39415970678363277,
+      "grad_norm": 0.6731569766998291,
+      "learning_rate": 9.733328079771573e-06,
+      "loss": 0.1158,
+      "step": 820
+    },
+    {
+      "epoch": 0.40377335816859944,
+      "grad_norm": 1.334695816040039,
+      "learning_rate": 9.71634697444754e-06,
+      "loss": 0.1606,
+      "step": 840
+    },
+    {
+      "epoch": 0.41338700955356605,
+      "grad_norm": 1.1176824569702148,
+      "learning_rate": 9.698857518049913e-06,
+      "loss": 0.1659,
+      "step": 860
+    },
+    {
+      "epoch": 0.4230006609385327,
+      "grad_norm": 1.074682593345642,
+      "learning_rate": 9.68086159567832e-06,
+      "loss": 0.1382,
+      "step": 880
+    },
+    {
+      "epoch": 0.4326143123234994,
+      "grad_norm": 0.8198431730270386,
+      "learning_rate": 9.66236114702178e-06,
+      "loss": 0.1431,
+      "step": 900
+    },
+    {
+      "epoch": 0.442227963708466,
+      "grad_norm": 0.26566508412361145,
+      "learning_rate": 9.643358166149646e-06,
+      "loss": 0.1671,
+      "step": 920
+    },
+    {
+      "epoch": 0.4518416150934327,
+      "grad_norm": 1.1486155986785889,
+      "learning_rate": 9.623854701296658e-06,
+      "loss": 0.1302,
+      "step": 940
+    },
+    {
+      "epoch": 0.46145526647839935,
+      "grad_norm": 0.9278470873832703,
+      "learning_rate": 9.603852854642183e-06,
+      "loss": 0.1884,
+      "step": 960
+    },
+    {
+      "epoch": 0.47106891786336597,
+      "grad_norm": 2.031203508377075,
+      "learning_rate": 9.58335478208363e-06,
+      "loss": 0.1672,
+      "step": 980
+    },
+    {
+      "epoch": 0.48068256924833264,
+      "grad_norm": 0.7224697470664978,
+      "learning_rate": 9.562362693004076e-06,
+      "loss": 0.1947,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4902962206332993,
+      "grad_norm": 1.2762720584869385,
+      "learning_rate": 9.54087885003413e-06,
+      "loss": 0.125,
+      "step": 1020
+    },
+    {
+      "epoch": 0.4999098720182659,
+      "grad_norm": 1.0751471519470215,
+      "learning_rate": 9.51890556880805e-06,
+      "loss": 0.1866,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5095235234032326,
+      "grad_norm": 1.022700309753418,
+      "learning_rate": 9.49644521771416e-06,
+      "loss": 0.1377,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5191371747881992,
+      "grad_norm": 1.0384761095046997,
+      "learning_rate": 9.473500217639571e-06,
+      "loss": 0.1275,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5287508261731659,
+      "grad_norm": 1.288801908493042,
+      "learning_rate": 9.450073041709243e-06,
+      "loss": 0.1563,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5383644775581325,
+      "grad_norm": 1.2543052434921265,
+      "learning_rate": 9.426166215019417e-06,
+      "loss": 0.1878,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5479781289430992,
+      "grad_norm": 1.3297594785690308,
+      "learning_rate": 9.401782314365458e-06,
+      "loss": 0.1533,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5575917803280659,
+      "grad_norm": 0.599759042263031,
+      "learning_rate": 9.376923967964108e-06,
+      "loss": 0.1473,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5672054317130325,
+      "grad_norm": 0.18831883370876312,
+      "learning_rate": 9.351593855170204e-06,
+      "loss": 0.1076,
+      "step": 1180
+    },
+    {
+      "epoch": 0.5768190830979991,
+      "grad_norm": 1.0437901020050049,
+      "learning_rate": 9.325794706187885e-06,
+      "loss": 0.1644,
+      "step": 1200
+    },
+    {
+      "epoch": 0.5864327344829658,
+      "grad_norm": 0.46901631355285645,
+      "learning_rate": 9.29952930177632e-06,
+      "loss": 0.0936,
+      "step": 1220
+    },
+    {
+      "epoch": 0.5960463858679325,
+      "grad_norm": 1.486289620399475,
+      "learning_rate": 9.272800472949976e-06,
+      "loss": 0.1931,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6056600372528991,
+      "grad_norm": 0.6040721535682678,
+      "learning_rate": 9.245611100673488e-06,
+      "loss": 0.1397,
+      "step": 1260
+    },
+    {
+      "epoch": 0.6152736886378658,
+      "grad_norm": 0.18753597140312195,
+      "learning_rate": 9.217964115551127e-06,
+      "loss": 0.1169,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6248873400228324,
+      "grad_norm": 0.9620321989059448,
+      "learning_rate": 9.189862497510927e-06,
+      "loss": 0.1573,
+      "step": 1300
+    },
+    {
+      "epoch": 0.634500991407799,
+      "grad_norm": 1.2345646619796753,
+      "learning_rate": 9.161309275483491e-06,
+      "loss": 0.1587,
+      "step": 1320
+    },
+    {
+      "epoch": 0.6441146427927658,
+      "grad_norm": 1.7545924186706543,
+      "learning_rate": 9.132307527075531e-06,
+      "loss": 0.1384,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6537282941777324,
+      "grad_norm": 1.2889124155044556,
+      "learning_rate": 9.102860378238127e-06,
+      "loss": 0.1425,
+      "step": 1360
+    },
+    {
+      "epoch": 0.663341945562699,
+      "grad_norm": 0.955100953578949,
+      "learning_rate": 9.072971002929812e-06,
+      "loss": 0.1375,
+      "step": 1380
+    },
+    {
+      "epoch": 0.666226040978189,
+      "eval_loss": 0.11805504560470581,
+      "eval_runtime": 207.1597,
+      "eval_samples_per_second": 5.175,
+      "eval_steps_per_second": 5.175,
+      "step": 1386
+    },
+    {
+      "epoch": 0.6729555969476657,
+      "grad_norm": 0.9628621339797974,
+      "learning_rate": 9.042642622774465e-06,
+      "loss": 0.122,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6825692483326323,
+      "grad_norm": 0.8323156833648682,
+      "learning_rate": 9.011878506714065e-06,
+      "loss": 0.142,
+      "step": 1420
+    },
+    {
+      "epoch": 0.6921828997175989,
+      "grad_norm": 1.0403231382369995,
+      "learning_rate": 8.980681970656349e-06,
+      "loss": 0.1435,
+      "step": 1440
+    },
+    {
+      "epoch": 0.7017965511025657,
+      "grad_norm": 1.2075049877166748,
+      "learning_rate": 8.949056377117408e-06,
+      "loss": 0.1236,
+      "step": 1460
+    },
+    {
+      "epoch": 0.7114102024875323,
+      "grad_norm": 1.3533861637115479,
+      "learning_rate": 8.917005134859263e-06,
+      "loss": 0.1378,
+      "step": 1480
+    },
+    {
+      "epoch": 0.7210238538724989,
+      "grad_norm": 1.1251810789108276,
+      "learning_rate": 8.884531698522443e-06,
+      "loss": 0.1413,
+      "step": 1500
+    },
+    {
+      "epoch": 0.7306375052574656,
+      "grad_norm": 1.074367880821228,
+      "learning_rate": 8.85163956825363e-06,
+      "loss": 0.1685,
+      "step": 1520
+    },
+    {
+      "epoch": 0.7402511566424322,
+      "grad_norm": 1.4021315574645996,
+      "learning_rate": 8.818332289328402e-06,
+      "loss": 0.1299,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7498648080273989,
+      "grad_norm": 1.6010384559631348,
+      "learning_rate": 8.784613451769098e-06,
+      "loss": 0.1602,
+      "step": 1560
+    },
+    {
+      "epoch": 0.7594784594123656,
+      "grad_norm": 1.1963365077972412,
+      "learning_rate": 8.750486689957874e-06,
+      "loss": 0.1677,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7690921107973322,
+      "grad_norm": 1.3554548025131226,
+      "learning_rate": 8.71595568224496e-06,
+      "loss": 0.1462,
+      "step": 1600
+    },
+    {
+      "epoch": 0.7787057621822988,
+      "grad_norm": 0.5640055537223816,
+      "learning_rate": 8.681024150552208e-06,
+      "loss": 0.1339,
+      "step": 1620
+    },
+    {
+      "epoch": 0.7883194135672655,
+      "grad_norm": 1.2408941984176636,
+      "learning_rate": 8.64569585997191e-06,
+      "loss": 0.1313,
+      "step": 1640
+    },
+    {
+      "epoch": 0.7979330649522322,
+      "grad_norm": 1.0671769380569458,
+      "learning_rate": 8.609974618360978e-06,
+      "loss": 0.1757,
+      "step": 1660
+    },
+    {
+      "epoch": 0.8075467163371989,
+      "grad_norm": 1.298189401626587,
+      "learning_rate": 8.573864275930526e-06,
+      "loss": 0.1833,
+      "step": 1680
+    },
+    {
+      "epoch": 0.8171603677221655,
+      "grad_norm": 0.5296375155448914,
+      "learning_rate": 8.537368724830865e-06,
+      "loss": 0.1517,
+      "step": 1700
+    },
+    {
+      "epoch": 0.8267740191071321,
+      "grad_norm": 1.1902680397033691,
+      "learning_rate": 8.50049189873199e-06,
+      "loss": 0.1231,
+      "step": 1720
+    },
+    {
+      "epoch": 0.8363876704920988,
+      "grad_norm": 0.0974334254860878,
+      "learning_rate": 8.463237772399589e-06,
+      "loss": 0.0947,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8460013218770654,
+      "grad_norm": 1.0790835618972778,
+      "learning_rate": 8.42561036126663e-06,
+      "loss": 0.1616,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8556149732620321,
+      "grad_norm": 0.5958455204963684,
+      "learning_rate": 8.38761372100055e-06,
+      "loss": 0.1409,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8652286246469988,
+      "grad_norm": 1.6728020906448364,
+      "learning_rate": 8.349251947066118e-06,
+      "loss": 0.137,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8748422760319654,
+      "grad_norm": 1.0818266868591309,
+      "learning_rate": 8.310529174284004e-06,
+      "loss": 0.1462,
+      "step": 1820
+    },
+    {
+      "epoch": 0.884455927416932,
+      "grad_norm": 0.9389984607696533,
+      "learning_rate": 8.271449576385112e-06,
+      "loss": 0.1283,
+      "step": 1840
+    },
+    {
+      "epoch": 0.8940695788018987,
+      "grad_norm": 1.3795585632324219,
+      "learning_rate": 8.23201736556071e-06,
+      "loss": 0.1292,
+      "step": 1860
+    },
+    {
+      "epoch": 0.9036832301868654,
+      "grad_norm": 1.248744249343872,
+      "learning_rate": 8.192236792008426e-06,
+      "loss": 0.1447,
+      "step": 1880
+    },
+    {
+      "epoch": 0.913296881571832,
+      "grad_norm": 1.3501381874084473,
+      "learning_rate": 8.15211214347413e-06,
+      "loss": 0.1285,
+      "step": 1900
+    },
+    {
+      "epoch": 0.9229105329567987,
+      "grad_norm": 1.1559216976165771,
+      "learning_rate": 8.111647744789788e-06,
+      "loss": 0.1033,
+      "step": 1920
+    },
+    {
+      "epoch": 0.9325241843417653,
+      "grad_norm": 1.5532070398330688,
+      "learning_rate": 8.07084795740731e-06,
+      "loss": 0.1457,
+      "step": 1940
+    },
+    {
+      "epoch": 0.9421378357267319,
+      "grad_norm": 0.901100218296051,
+      "learning_rate": 8.029717178928446e-06,
+      "loss": 0.1368,
+      "step": 1960
+    },
+    {
+      "epoch": 0.9517514871116987,
+      "grad_norm": 1.8944685459136963,
+      "learning_rate": 7.9882598426308e-06,
+      "loss": 0.1213,
+      "step": 1980
+    },
+    {
+      "epoch": 0.9613651384966653,
+      "grad_norm": 1.8067469596862793,
+      "learning_rate": 7.946480416989978e-06,
+      "loss": 0.1269,
+      "step": 2000
+    },
+    {
+      "epoch": 0.9709787898816319,
+      "grad_norm": 1.817403793334961,
+      "learning_rate": 7.904383405197962e-06,
+      "loss": 0.1724,
+      "step": 2020
+    },
+    {
+      "epoch": 0.9805924412665986,
+      "grad_norm": 1.2046632766723633,
+      "learning_rate": 7.86197334467773e-06,
+      "loss": 0.1406,
+      "step": 2040
+    },
+    {
+      "epoch": 0.9902060926515652,
+      "grad_norm": 1.3356693983078003,
+      "learning_rate": 7.819254806594194e-06,
+      "loss": 0.1488,
+      "step": 2060
+    },
+    {
+      "epoch": 0.9993390614672836,
+      "eval_loss": 0.11032938957214355,
+      "eval_runtime": 207.1464,
+      "eval_samples_per_second": 5.175,
+      "eval_steps_per_second": 5.175,
+      "step": 2079
+    },
+    {
+      "epoch": 0.9998197440365318,
+      "grad_norm": 1.34231698513031,
+      "learning_rate": 7.776232395361496e-06,
+      "loss": 0.1295,
+      "step": 2080
+    },
+    {
+      "epoch": 1.0094333954214985,
+      "grad_norm": 0.8685508370399475,
+      "learning_rate": 7.732910748146715e-06,
+      "loss": 0.103,
+      "step": 2100
+    },
+    {
+      "epoch": 1.0190470468064652,
+      "grad_norm": 1.2184851169586182,
+      "learning_rate": 7.68929453437006e-06,
+      "loss": 0.0962,
+      "step": 2120
+    },
+    {
+      "epoch": 1.028660698191432,
+      "grad_norm": 0.4934062957763672,
+      "learning_rate": 7.645388455201572e-06,
+      "loss": 0.1523,
+      "step": 2140
+    },
+    {
+      "epoch": 1.0382743495763984,
+      "grad_norm": 1.7481762170791626,
+      "learning_rate": 7.601197243054411e-06,
+      "loss": 0.1286,
+      "step": 2160
+    },
+    {
+      "epoch": 1.0478880009613651,
+      "grad_norm": 0.16584353148937225,
+      "learning_rate": 7.556725661074771e-06,
+      "loss": 0.1158,
+      "step": 2180
+    },
+    {
+      "epoch": 1.0575016523463319,
+      "grad_norm": 1.316250205039978,
+      "learning_rate": 7.511978502628482e-06,
+      "loss": 0.1113,
+      "step": 2200
+    },
+    {
+      "epoch": 1.0671153037312984,
+      "grad_norm": 1.3466953039169312,
+      "learning_rate": 7.4669605907843654e-06,
+      "loss": 0.1116,
+      "step": 2220
+    },
+    {
+      "epoch": 1.076728955116265,
+      "grad_norm": 2.4155526161193848,
+      "learning_rate": 7.421676777794376e-06,
+      "loss": 0.1118,
+      "step": 2240
+    },
+    {
+      "epoch": 1.0863426065012318,
+      "grad_norm": 1.2634962797164917,
+      "learning_rate": 7.376131944570597e-06,
+      "loss": 0.1508,
+      "step": 2260
+    },
+    {
+      "epoch": 1.0959562578861983,
+      "grad_norm": 0.6129007339477539,
+      "learning_rate": 7.330331000159161e-06,
+      "loss": 0.105,
+      "step": 2280
+    },
+    {
+      "epoch": 1.105569909271165,
+      "grad_norm": 1.5106947422027588,
+      "learning_rate": 7.284278881211121e-06,
+      "loss": 0.1343,
+      "step": 2300
+    },
+    {
+      "epoch": 1.1151835606561318,
+      "grad_norm": 0.7808032035827637,
+      "learning_rate": 7.237980551450358e-06,
+      "loss": 0.121,
+      "step": 2320
+    },
+    {
+      "epoch": 1.1247972120410983,
+      "grad_norm": 1.4901185035705566,
+      "learning_rate": 7.191441001138567e-06,
+      "loss": 0.0929,
+      "step": 2340
+    },
+    {
+      "epoch": 1.134410863426065,
+      "grad_norm": 1.4136017560958862,
+      "learning_rate": 7.144665246537379e-06,
+      "loss": 0.1312,
+      "step": 2360
+    },
+    {
+      "epoch": 1.1440245148110317,
+      "grad_norm": 1.4810831546783447,
+      "learning_rate": 7.097658329367682e-06,
+      "loss": 0.106,
+      "step": 2380
+    },
+    {
+      "epoch": 1.1536381661959982,
+      "grad_norm": 1.8399866819381714,
+      "learning_rate": 7.050425316266209e-06,
+      "loss": 0.1121,
+      "step": 2400
+    },
+    {
+      "epoch": 1.163251817580965,
+      "grad_norm": 0.4116855263710022,
+      "learning_rate": 7.002971298239419e-06,
+      "loss": 0.1302,
+      "step": 2420
+    },
+    {
+      "epoch": 1.1728654689659317,
+      "grad_norm": 1.8218494653701782,
+      "learning_rate": 6.955301390114771e-06,
+      "loss": 0.1258,
+      "step": 2440
+    },
+    {
+      "epoch": 1.1824791203508982,
+      "grad_norm": 2.300889730453491,
+      "learning_rate": 6.90742072998942e-06,
+      "loss": 0.1311,
+      "step": 2460
+    },
+    {
+      "epoch": 1.192092771735865,
+      "grad_norm": 1.5062252283096313,
+      "learning_rate": 6.859334478676411e-06,
+      "loss": 0.1084,
+      "step": 2480
+    },
+    {
+      "epoch": 1.2017064231208316,
+      "grad_norm": 0.3353615403175354,
+      "learning_rate": 6.811047819148413e-06,
+      "loss": 0.1159,
+      "step": 2500
+    },
+    {
+      "epoch": 1.2113200745057981,
+      "grad_norm": 1.5180888175964355,
+      "learning_rate": 6.762565955979083e-06,
+      "loss": 0.1036,
+      "step": 2520
+    },
+    {
+      "epoch": 1.2209337258907649,
+      "grad_norm": 1.7501280307769775,
+      "learning_rate": 6.713894114782085e-06,
+      "loss": 0.1125,
+      "step": 2540
+    },
+    {
+      "epoch": 1.2305473772757316,
+      "grad_norm": 0.561276376247406,
+      "learning_rate": 6.665037541647845e-06,
+      "loss": 0.1213,
+      "step": 2560
+    },
+    {
+      "epoch": 1.240161028660698,
+      "grad_norm": 1.9739066362380981,
+      "learning_rate": 6.616001502578116e-06,
+      "loss": 0.1098,
+      "step": 2580
+    },
+    {
+      "epoch": 1.2497746800456648,
+      "grad_norm": 1.1374402046203613,
+      "learning_rate": 6.5667912829183615e-06,
+      "loss": 0.1582,
+      "step": 2600
+    },
+    {
+      "epoch": 1.2593883314306316,
+      "grad_norm": 2.0057950019836426,
+      "learning_rate": 6.517412186788089e-06,
+      "loss": 0.1154,
+      "step": 2620
+    },
+    {
+      "epoch": 1.269001982815598,
+      "grad_norm": 1.2539591789245605,
+      "learning_rate": 6.4678695365091455e-06,
+      "loss": 0.1164,
+      "step": 2640
+    },
+    {
+      "epoch": 1.2786156342005648,
+      "grad_norm": 0.8485791087150574,
+      "learning_rate": 6.418168672032048e-06,
+      "loss": 0.0873,
+      "step": 2660
+    },
+    {
+      "epoch": 1.2882292855855315,
+      "grad_norm": 0.7628265023231506,
+      "learning_rate": 6.368314950360416e-06,
+      "loss": 0.1313,
+      "step": 2680
+    },
+    {
+      "epoch": 1.2978429369704982,
+      "grad_norm": 1.0029041767120361,
+      "learning_rate": 6.318313744973566e-06,
+      "loss": 0.1372,
+      "step": 2700
+    },
+    {
+      "epoch": 1.3074565883554647,
+      "grad_norm": 1.3355886936187744,
+      "learning_rate": 6.268170445247344e-06,
+      "loss": 0.1408,
+      "step": 2720
+    },
+    {
+      "epoch": 1.3170702397404315,
+      "grad_norm": 1.5066125392913818,
+      "learning_rate": 6.217890455873213e-06,
+      "loss": 0.1698,
+      "step": 2740
+    },
+    {
+      "epoch": 1.326683891125398,
+      "grad_norm": 2.3855671882629395,
+      "learning_rate": 6.1674791962757255e-06,
+      "loss": 0.1224,
+      "step": 2760
+    },
+    {
+      "epoch": 1.332452081956378,
+      "eval_loss": 0.10718663781881332,
+      "eval_runtime": 206.8134,
+      "eval_samples_per_second": 5.183,
+      "eval_steps_per_second": 5.183,
+      "step": 2772
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 6240,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 693,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.053432564151091e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-2772/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:228958bc198813ec3c7334720d792de981c3477375cb092239e1e5c068b39593
+size 5624

checkpoint-3465/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Salesforce/Llama-xLAM-2-8b-fc-r
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Salesforce/Llama-xLAM-2-8b-fc-r
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-3465/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Salesforce/Llama-xLAM-2-8b-fc-r",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-3465/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32ca54370e4e2f9d78092b4c18f0c02607e5cdd59f0269cde6c233ab1ab72b2e
+size 54560368

checkpoint-3465/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ffd20b63123acb0f281567190be46182080ccbf3c35f37756737d81c6dde9c96
+size 109267450

checkpoint-3465/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8b6145c061ea9f61fb522fdcf4c2658bb16e97e53120684e59bf0c9a7b0280f
+size 14244

checkpoint-3465/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3930ea6995c4ebe61be4cd0283eac7a552578e35f2f048b65153fde77468e152
+size 1064

checkpoint-3465/special_tokens_map.json

@@ -0,0 +1,27 @@
+{
+  "additional_special_tokens": [
+    "<|eot_id|>",
+    "<|eom_id|>"
+  ],
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-3465/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

checkpoint-3465/tokenizer_config.json

@@ -0,0 +1,2070 @@
+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|eot_id|>",
+    "<|eom_id|>"
+  ],
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- Extract system message #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content'] | trim %}\n    {%- set messages = messages[1:] %}\n    {{- system_message + \"\\n\" }}\n{%- else %}\n    {%- set system_message = \"You are a helpful assistant that can use tools. You are developed by Salesforce xLAM team.\" %}\n    {% set format_instruction %}You have access to a set of tools. When using tools, make calls in a single JSON array: \n\n[{\"name\": \"tool_call_name\", \"arguments\": {\"arg1\": \"value1\", \"arg2\": \"value2\"}}, ... (additional parallel tool calls as needed)]\n\nIf no tool is suitable, state that explicitly. If the user's input lacks required parameters, ask for clarification. Do not interpret or respond until tool results are returned. Once they are available, process them or make additional calls if needed. For tasks that don't require tools, such as casual conversation or general advice, respond directly in plain text. The available tools are:{% endset %}\n    {{- system_message + \"\\n\" }}\n    {%- if tools is not none %}\n        {{- format_instruction + \"\\n\\n\" }}\n    {%- endif %}\n{%- endif %}\n\n\n{%- if tools is not none %}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- \"<|eot_id|>\" }}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n        {%- if message['tool_calls'] %}\n            {{- \"[\" }}\n            {%- for tool_call_function in message.tool_calls %}\n                {%- set tool_call = tool_call_function.function %}\n                {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n                {{- '\"arguments\": ' }}\n                {{- tool_call.arguments | tojson }}\n                {{- \"}\" }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n            {%- endfor %}\n            {{- \"]\" }}\n            {{- \"<|eot_id|>\" }}\n        {%- elif message['content'] %}\n            {{- message['content'] | trim + '<|eot_id|>' }}\n        {%- else %}\n            {{- \"[]\\n\" + '<|eot_id|>' }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>\" + \"ipython\" + \"<|end_header_id|>\\n\\n\" }}\n        {%- set content = message[\"content\"] %}\n        {%- if content is mapping or (content is iterable and content is not string) %}\n            {{- content | tojson }}\n        {%- else %}\n            {{- content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "extra_special_tokens": {},
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 16384,
+  "pad_token": "<|eot_id|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

checkpoint-3465/trainer_state.json

@@ -0,0 +1,1284 @@
+{
+  "best_metric": 0.10441361367702484,
+  "best_model_checkpoint": "./xlam_lora_new_2560_1_delete_over_size_3epoch_multi/checkpoint-3465",
+  "epoch": 1.6655651024454725,
+  "eval_steps": 693,
+  "global_step": 3465,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.009613651384966653,
+      "grad_norm": 1.1487505435943604,
+      "learning_rate": 1.0638297872340427e-06,
+      "loss": 0.7127,
+      "step": 20
+    },
+    {
+      "epoch": 0.019227302769933306,
+      "grad_norm": 1.1384060382843018,
+      "learning_rate": 2.1276595744680853e-06,
+      "loss": 0.682,
+      "step": 40
+    },
+    {
+      "epoch": 0.02884095415489996,
+      "grad_norm": 0.6661604046821594,
+      "learning_rate": 3.191489361702128e-06,
+      "loss": 0.7262,
+      "step": 60
+    },
+    {
+      "epoch": 0.03845460553986661,
+      "grad_norm": 0.4899628460407257,
+      "learning_rate": 4.255319148936171e-06,
+      "loss": 0.6003,
+      "step": 80
+    },
+    {
+      "epoch": 0.04806825692483326,
+      "grad_norm": 1.2069792747497559,
+      "learning_rate": 5.319148936170213e-06,
+      "loss": 0.5991,
+      "step": 100
+    },
+    {
+      "epoch": 0.05768190830979992,
+      "grad_norm": 1.0019866228103638,
+      "learning_rate": 6.382978723404256e-06,
+      "loss": 0.5028,
+      "step": 120
+    },
+    {
+      "epoch": 0.06729555969476657,
+      "grad_norm": 1.056580662727356,
+      "learning_rate": 7.446808510638298e-06,
+      "loss": 0.433,
+      "step": 140
+    },
+    {
+      "epoch": 0.07690921107973323,
+      "grad_norm": 0.6433275938034058,
+      "learning_rate": 8.510638297872341e-06,
+      "loss": 0.336,
+      "step": 160
+    },
+    {
+      "epoch": 0.08652286246469987,
+      "grad_norm": 0.33471840620040894,
+      "learning_rate": 9.574468085106385e-06,
+      "loss": 0.248,
+      "step": 180
+    },
+    {
+      "epoch": 0.09613651384966652,
+      "grad_norm": 0.5343720316886902,
+      "learning_rate": 9.999902993015826e-06,
+      "loss": 0.2601,
+      "step": 200
+    },
+    {
+      "epoch": 0.10575016523463318,
+      "grad_norm": 0.9687429070472717,
+      "learning_rate": 9.999310186188382e-06,
+      "loss": 0.2559,
+      "step": 220
+    },
+    {
+      "epoch": 0.11536381661959984,
+      "grad_norm": 0.4758259356021881,
+      "learning_rate": 9.998178529120666e-06,
+      "loss": 0.2103,
+      "step": 240
+    },
+    {
+      "epoch": 0.12497746800456648,
+      "grad_norm": 0.782993495464325,
+      "learning_rate": 9.996508143788239e-06,
+      "loss": 0.2466,
+      "step": 260
+    },
+    {
+      "epoch": 0.13459111938953314,
+      "grad_norm": 0.621163010597229,
+      "learning_rate": 9.994299210233451e-06,
+      "loss": 0.2037,
+      "step": 280
+    },
+    {
+      "epoch": 0.14420477077449978,
+      "grad_norm": 0.41969236731529236,
+      "learning_rate": 9.991551966546023e-06,
+      "loss": 0.1842,
+      "step": 300
+    },
+    {
+      "epoch": 0.15381842215946645,
+      "grad_norm": 0.493594765663147,
+      "learning_rate": 9.988266708837393e-06,
+      "loss": 0.1475,
+      "step": 320
+    },
+    {
+      "epoch": 0.1634320735444331,
+      "grad_norm": 1.0131076574325562,
+      "learning_rate": 9.984443791208795e-06,
+      "loss": 0.2276,
+      "step": 340
+    },
+    {
+      "epoch": 0.17304572492939974,
+      "grad_norm": 0.9529913663864136,
+      "learning_rate": 9.98008362571309e-06,
+      "loss": 0.1857,
+      "step": 360
+    },
+    {
+      "epoch": 0.1826593763143664,
+      "grad_norm": 0.7426656484603882,
+      "learning_rate": 9.975186682310364e-06,
+      "loss": 0.1898,
+      "step": 380
+    },
+    {
+      "epoch": 0.19227302769933305,
+      "grad_norm": 0.867158830165863,
+      "learning_rate": 9.969753488817263e-06,
+      "loss": 0.1766,
+      "step": 400
+    },
+    {
+      "epoch": 0.20188667908429972,
+      "grad_norm": 0.7929105162620544,
+      "learning_rate": 9.963784630850099e-06,
+      "loss": 0.2066,
+      "step": 420
+    },
+    {
+      "epoch": 0.21150033046926636,
+      "grad_norm": 0.6801859736442566,
+      "learning_rate": 9.957280751761743e-06,
+      "loss": 0.1975,
+      "step": 440
+    },
+    {
+      "epoch": 0.221113981854233,
+      "grad_norm": 0.9697579145431519,
+      "learning_rate": 9.950242552572272e-06,
+      "loss": 0.1587,
+      "step": 460
+    },
+    {
+      "epoch": 0.23072763323919968,
+      "grad_norm": 0.9611632227897644,
+      "learning_rate": 9.942670791893412e-06,
+      "loss": 0.1772,
+      "step": 480
+    },
+    {
+      "epoch": 0.24034128462416632,
+      "grad_norm": 0.9096260070800781,
+      "learning_rate": 9.934566285846767e-06,
+      "loss": 0.1352,
+      "step": 500
+    },
+    {
+      "epoch": 0.24995493600913296,
+      "grad_norm": 0.8181589245796204,
+      "learning_rate": 9.925929907975864e-06,
+      "loss": 0.1402,
+      "step": 520
+    },
+    {
+      "epoch": 0.2595685873940996,
+      "grad_norm": 0.8003833889961243,
+      "learning_rate": 9.91676258915198e-06,
+      "loss": 0.1882,
+      "step": 540
+    },
+    {
+      "epoch": 0.2691822387790663,
+      "grad_norm": 0.5620806217193604,
+      "learning_rate": 9.907065317473832e-06,
+      "loss": 0.1663,
+      "step": 560
+    },
+    {
+      "epoch": 0.27879589016403294,
+      "grad_norm": 1.0216277837753296,
+      "learning_rate": 9.896839138161053e-06,
+      "loss": 0.1341,
+      "step": 580
+    },
+    {
+      "epoch": 0.28840954154899956,
+      "grad_norm": 0.6821846961975098,
+      "learning_rate": 9.886085153441549e-06,
+      "loss": 0.1953,
+      "step": 600
+    },
+    {
+      "epoch": 0.29802319293396623,
+      "grad_norm": 1.1055521965026855,
+      "learning_rate": 9.87480452243268e-06,
+      "loss": 0.1493,
+      "step": 620
+    },
+    {
+      "epoch": 0.3076368443189329,
+      "grad_norm": 0.7052333950996399,
+      "learning_rate": 9.86299846101634e-06,
+      "loss": 0.2156,
+      "step": 640
+    },
+    {
+      "epoch": 0.3172504957038995,
+      "grad_norm": 0.7262466549873352,
+      "learning_rate": 9.850668241707897e-06,
+      "loss": 0.122,
+      "step": 660
+    },
+    {
+      "epoch": 0.3268641470888662,
+      "grad_norm": 1.1921933889389038,
+      "learning_rate": 9.837815193519031e-06,
+      "loss": 0.1534,
+      "step": 680
+    },
+    {
+      "epoch": 0.3331130204890945,
+      "eval_loss": 0.13637545704841614,
+      "eval_runtime": 206.7827,
+      "eval_samples_per_second": 5.184,
+      "eval_steps_per_second": 5.184,
+      "step": 693
+    },
+    {
+      "epoch": 0.33647779847383286,
+      "grad_norm": 0.8965498208999634,
+      "learning_rate": 9.824440701814491e-06,
+      "loss": 0.206,
+      "step": 700
+    },
+    {
+      "epoch": 0.34609144985879947,
+      "grad_norm": 1.2437098026275635,
+      "learning_rate": 9.810546208162776e-06,
+      "loss": 0.1498,
+      "step": 720
+    },
+    {
+      "epoch": 0.35570510124376614,
+      "grad_norm": 0.823193371295929,
+      "learning_rate": 9.796133210180748e-06,
+      "loss": 0.142,
+      "step": 740
+    },
+    {
+      "epoch": 0.3653187526287328,
+      "grad_norm": 1.1031694412231445,
+      "learning_rate": 9.781203261372216e-06,
+      "loss": 0.1411,
+      "step": 760
+    },
+    {
+      "epoch": 0.3749324040136994,
+      "grad_norm": 0.788958728313446,
+      "learning_rate": 9.765757970960496e-06,
+      "loss": 0.1354,
+      "step": 780
+    },
+    {
+      "epoch": 0.3845460553986661,
+      "grad_norm": 0.3245677351951599,
+      "learning_rate": 9.749799003714954e-06,
+      "loss": 0.1547,
+      "step": 800
+    },
+    {
+      "epoch": 0.39415970678363277,
+      "grad_norm": 0.6731569766998291,
+      "learning_rate": 9.733328079771573e-06,
+      "loss": 0.1158,
+      "step": 820
+    },
+    {
+      "epoch": 0.40377335816859944,
+      "grad_norm": 1.334695816040039,
+      "learning_rate": 9.71634697444754e-06,
+      "loss": 0.1606,
+      "step": 840
+    },
+    {
+      "epoch": 0.41338700955356605,
+      "grad_norm": 1.1176824569702148,
+      "learning_rate": 9.698857518049913e-06,
+      "loss": 0.1659,
+      "step": 860
+    },
+    {
+      "epoch": 0.4230006609385327,
+      "grad_norm": 1.074682593345642,
+      "learning_rate": 9.68086159567832e-06,
+      "loss": 0.1382,
+      "step": 880
+    },
+    {
+      "epoch": 0.4326143123234994,
+      "grad_norm": 0.8198431730270386,
+      "learning_rate": 9.66236114702178e-06,
+      "loss": 0.1431,
+      "step": 900
+    },
+    {
+      "epoch": 0.442227963708466,
+      "grad_norm": 0.26566508412361145,
+      "learning_rate": 9.643358166149646e-06,
+      "loss": 0.1671,
+      "step": 920
+    },
+    {
+      "epoch": 0.4518416150934327,
+      "grad_norm": 1.1486155986785889,
+      "learning_rate": 9.623854701296658e-06,
+      "loss": 0.1302,
+      "step": 940
+    },
+    {
+      "epoch": 0.46145526647839935,
+      "grad_norm": 0.9278470873832703,
+      "learning_rate": 9.603852854642183e-06,
+      "loss": 0.1884,
+      "step": 960
+    },
+    {
+      "epoch": 0.47106891786336597,
+      "grad_norm": 2.031203508377075,
+      "learning_rate": 9.58335478208363e-06,
+      "loss": 0.1672,
+      "step": 980
+    },
+    {
+      "epoch": 0.48068256924833264,
+      "grad_norm": 0.7224697470664978,
+      "learning_rate": 9.562362693004076e-06,
+      "loss": 0.1947,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4902962206332993,
+      "grad_norm": 1.2762720584869385,
+      "learning_rate": 9.54087885003413e-06,
+      "loss": 0.125,
+      "step": 1020
+    },
+    {
+      "epoch": 0.4999098720182659,
+      "grad_norm": 1.0751471519470215,
+      "learning_rate": 9.51890556880805e-06,
+      "loss": 0.1866,
+      "step": 1040
+    },
+    {
+      "epoch": 0.5095235234032326,
+      "grad_norm": 1.022700309753418,
+      "learning_rate": 9.49644521771416e-06,
+      "loss": 0.1377,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5191371747881992,
+      "grad_norm": 1.0384761095046997,
+      "learning_rate": 9.473500217639571e-06,
+      "loss": 0.1275,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5287508261731659,
+      "grad_norm": 1.288801908493042,
+      "learning_rate": 9.450073041709243e-06,
+      "loss": 0.1563,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5383644775581325,
+      "grad_norm": 1.2543052434921265,
+      "learning_rate": 9.426166215019417e-06,
+      "loss": 0.1878,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5479781289430992,
+      "grad_norm": 1.3297594785690308,
+      "learning_rate": 9.401782314365458e-06,
+      "loss": 0.1533,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5575917803280659,
+      "grad_norm": 0.599759042263031,
+      "learning_rate": 9.376923967964108e-06,
+      "loss": 0.1473,
+      "step": 1160
+    },
+    {
+      "epoch": 0.5672054317130325,
+      "grad_norm": 0.18831883370876312,
+      "learning_rate": 9.351593855170204e-06,
+      "loss": 0.1076,
+      "step": 1180
+    },
+    {
+      "epoch": 0.5768190830979991,
+      "grad_norm": 1.0437901020050049,
+      "learning_rate": 9.325794706187885e-06,
+      "loss": 0.1644,
+      "step": 1200
+    },
+    {
+      "epoch": 0.5864327344829658,
+      "grad_norm": 0.46901631355285645,
+      "learning_rate": 9.29952930177632e-06,
+      "loss": 0.0936,
+      "step": 1220
+    },
+    {
+      "epoch": 0.5960463858679325,
+      "grad_norm": 1.486289620399475,
+      "learning_rate": 9.272800472949976e-06,
+      "loss": 0.1931,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6056600372528991,
+      "grad_norm": 0.6040721535682678,
+      "learning_rate": 9.245611100673488e-06,
+      "loss": 0.1397,
+      "step": 1260
+    },
+    {
+      "epoch": 0.6152736886378658,
+      "grad_norm": 0.18753597140312195,
+      "learning_rate": 9.217964115551127e-06,
+      "loss": 0.1169,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6248873400228324,
+      "grad_norm": 0.9620321989059448,
+      "learning_rate": 9.189862497510927e-06,
+      "loss": 0.1573,
+      "step": 1300
+    },
+    {
+      "epoch": 0.634500991407799,
+      "grad_norm": 1.2345646619796753,
+      "learning_rate": 9.161309275483491e-06,
+      "loss": 0.1587,
+      "step": 1320
+    },
+    {
+      "epoch": 0.6441146427927658,
+      "grad_norm": 1.7545924186706543,
+      "learning_rate": 9.132307527075531e-06,
+      "loss": 0.1384,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6537282941777324,
+      "grad_norm": 1.2889124155044556,
+      "learning_rate": 9.102860378238127e-06,
+      "loss": 0.1425,
+      "step": 1360
+    },
+    {
+      "epoch": 0.663341945562699,
+      "grad_norm": 0.955100953578949,
+      "learning_rate": 9.072971002929812e-06,
+      "loss": 0.1375,
+      "step": 1380
+    },
+    {
+      "epoch": 0.666226040978189,
+      "eval_loss": 0.11805504560470581,
+      "eval_runtime": 207.1597,
+      "eval_samples_per_second": 5.175,
+      "eval_steps_per_second": 5.175,
+      "step": 1386
+    },
+    {
+      "epoch": 0.6729555969476657,
+      "grad_norm": 0.9628621339797974,
+      "learning_rate": 9.042642622774465e-06,
+      "loss": 0.122,
+      "step": 1400
+    },
+    {
+      "epoch": 0.6825692483326323,
+      "grad_norm": 0.8323156833648682,
+      "learning_rate": 9.011878506714065e-06,
+      "loss": 0.142,
+      "step": 1420
+    },
+    {
+      "epoch": 0.6921828997175989,
+      "grad_norm": 1.0403231382369995,
+      "learning_rate": 8.980681970656349e-06,
+      "loss": 0.1435,
+      "step": 1440
+    },
+    {
+      "epoch": 0.7017965511025657,
+      "grad_norm": 1.2075049877166748,
+      "learning_rate": 8.949056377117408e-06,
+      "loss": 0.1236,
+      "step": 1460
+    },
+    {
+      "epoch": 0.7114102024875323,
+      "grad_norm": 1.3533861637115479,
+      "learning_rate": 8.917005134859263e-06,
+      "loss": 0.1378,
+      "step": 1480
+    },
+    {
+      "epoch": 0.7210238538724989,
+      "grad_norm": 1.1251810789108276,
+      "learning_rate": 8.884531698522443e-06,
+      "loss": 0.1413,
+      "step": 1500
+    },
+    {
+      "epoch": 0.7306375052574656,
+      "grad_norm": 1.074367880821228,
+      "learning_rate": 8.85163956825363e-06,
+      "loss": 0.1685,
+      "step": 1520
+    },
+    {
+      "epoch": 0.7402511566424322,
+      "grad_norm": 1.4021315574645996,
+      "learning_rate": 8.818332289328402e-06,
+      "loss": 0.1299,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7498648080273989,
+      "grad_norm": 1.6010384559631348,
+      "learning_rate": 8.784613451769098e-06,
+      "loss": 0.1602,
+      "step": 1560
+    },
+    {
+      "epoch": 0.7594784594123656,
+      "grad_norm": 1.1963365077972412,
+      "learning_rate": 8.750486689957874e-06,
+      "loss": 0.1677,
+      "step": 1580
+    },
+    {
+      "epoch": 0.7690921107973322,
+      "grad_norm": 1.3554548025131226,
+      "learning_rate": 8.71595568224496e-06,
+      "loss": 0.1462,
+      "step": 1600
+    },
+    {
+      "epoch": 0.7787057621822988,
+      "grad_norm": 0.5640055537223816,
+      "learning_rate": 8.681024150552208e-06,
+      "loss": 0.1339,
+      "step": 1620
+    },
+    {
+      "epoch": 0.7883194135672655,
+      "grad_norm": 1.2408941984176636,
+      "learning_rate": 8.64569585997191e-06,
+      "loss": 0.1313,
+      "step": 1640
+    },
+    {
+      "epoch": 0.7979330649522322,
+      "grad_norm": 1.0671769380569458,
+      "learning_rate": 8.609974618360978e-06,
+      "loss": 0.1757,
+      "step": 1660
+    },
+    {
+      "epoch": 0.8075467163371989,
+      "grad_norm": 1.298189401626587,
+      "learning_rate": 8.573864275930526e-06,
+      "loss": 0.1833,
+      "step": 1680
+    },
+    {
+      "epoch": 0.8171603677221655,
+      "grad_norm": 0.5296375155448914,
+      "learning_rate": 8.537368724830865e-06,
+      "loss": 0.1517,
+      "step": 1700
+    },
+    {
+      "epoch": 0.8267740191071321,
+      "grad_norm": 1.1902680397033691,
+      "learning_rate": 8.50049189873199e-06,
+      "loss": 0.1231,
+      "step": 1720
+    },
+    {
+      "epoch": 0.8363876704920988,
+      "grad_norm": 0.0974334254860878,
+      "learning_rate": 8.463237772399589e-06,
+      "loss": 0.0947,
+      "step": 1740
+    },
+    {
+      "epoch": 0.8460013218770654,
+      "grad_norm": 1.0790835618972778,
+      "learning_rate": 8.42561036126663e-06,
+      "loss": 0.1616,
+      "step": 1760
+    },
+    {
+      "epoch": 0.8556149732620321,
+      "grad_norm": 0.5958455204963684,
+      "learning_rate": 8.38761372100055e-06,
+      "loss": 0.1409,
+      "step": 1780
+    },
+    {
+      "epoch": 0.8652286246469988,
+      "grad_norm": 1.6728020906448364,
+      "learning_rate": 8.349251947066118e-06,
+      "loss": 0.137,
+      "step": 1800
+    },
+    {
+      "epoch": 0.8748422760319654,
+      "grad_norm": 1.0818266868591309,
+      "learning_rate": 8.310529174284004e-06,
+      "loss": 0.1462,
+      "step": 1820
+    },
+    {
+      "epoch": 0.884455927416932,
+      "grad_norm": 0.9389984607696533,
+      "learning_rate": 8.271449576385112e-06,
+      "loss": 0.1283,
+      "step": 1840
+    },
+    {
+      "epoch": 0.8940695788018987,
+      "grad_norm": 1.3795585632324219,
+      "learning_rate": 8.23201736556071e-06,
+      "loss": 0.1292,
+      "step": 1860
+    },
+    {
+      "epoch": 0.9036832301868654,
+      "grad_norm": 1.248744249343872,
+      "learning_rate": 8.192236792008426e-06,
+      "loss": 0.1447,
+      "step": 1880
+    },
+    {
+      "epoch": 0.913296881571832,
+      "grad_norm": 1.3501381874084473,
+      "learning_rate": 8.15211214347413e-06,
+      "loss": 0.1285,
+      "step": 1900
+    },
+    {
+      "epoch": 0.9229105329567987,
+      "grad_norm": 1.1559216976165771,
+      "learning_rate": 8.111647744789788e-06,
+      "loss": 0.1033,
+      "step": 1920
+    },
+    {
+      "epoch": 0.9325241843417653,
+      "grad_norm": 1.5532070398330688,
+      "learning_rate": 8.07084795740731e-06,
+      "loss": 0.1457,
+      "step": 1940
+    },
+    {
+      "epoch": 0.9421378357267319,
+      "grad_norm": 0.901100218296051,
+      "learning_rate": 8.029717178928446e-06,
+      "loss": 0.1368,
+      "step": 1960
+    },
+    {
+      "epoch": 0.9517514871116987,
+      "grad_norm": 1.8944685459136963,
+      "learning_rate": 7.9882598426308e-06,
+      "loss": 0.1213,
+      "step": 1980
+    },
+    {
+      "epoch": 0.9613651384966653,
+      "grad_norm": 1.8067469596862793,
+      "learning_rate": 7.946480416989978e-06,
+      "loss": 0.1269,
+      "step": 2000
+    },
+    {
+      "epoch": 0.9709787898816319,
+      "grad_norm": 1.817403793334961,
+      "learning_rate": 7.904383405197962e-06,
+      "loss": 0.1724,
+      "step": 2020
+    },
+    {
+      "epoch": 0.9805924412665986,
+      "grad_norm": 1.2046632766723633,
+      "learning_rate": 7.86197334467773e-06,
+      "loss": 0.1406,
+      "step": 2040
+    },
+    {
+      "epoch": 0.9902060926515652,
+      "grad_norm": 1.3356693983078003,
+      "learning_rate": 7.819254806594194e-06,
+      "loss": 0.1488,
+      "step": 2060
+    },
+    {
+      "epoch": 0.9993390614672836,
+      "eval_loss": 0.11032938957214355,
+      "eval_runtime": 207.1464,
+      "eval_samples_per_second": 5.175,
+      "eval_steps_per_second": 5.175,
+      "step": 2079
+    },
+    {
+      "epoch": 0.9998197440365318,
+      "grad_norm": 1.34231698513031,
+      "learning_rate": 7.776232395361496e-06,
+      "loss": 0.1295,
+      "step": 2080
+    },
+    {
+      "epoch": 1.0094333954214985,
+      "grad_norm": 0.8685508370399475,
+      "learning_rate": 7.732910748146715e-06,
+      "loss": 0.103,
+      "step": 2100
+    },
+    {
+      "epoch": 1.0190470468064652,
+      "grad_norm": 1.2184851169586182,
+      "learning_rate": 7.68929453437006e-06,
+      "loss": 0.0962,
+      "step": 2120
+    },
+    {
+      "epoch": 1.028660698191432,
+      "grad_norm": 0.4934062957763672,
+      "learning_rate": 7.645388455201572e-06,
+      "loss": 0.1523,
+      "step": 2140
+    },
+    {
+      "epoch": 1.0382743495763984,
+      "grad_norm": 1.7481762170791626,
+      "learning_rate": 7.601197243054411e-06,
+      "loss": 0.1286,
+      "step": 2160
+    },
+    {
+      "epoch": 1.0478880009613651,
+      "grad_norm": 0.16584353148937225,
+      "learning_rate": 7.556725661074771e-06,
+      "loss": 0.1158,
+      "step": 2180
+    },
+    {
+      "epoch": 1.0575016523463319,
+      "grad_norm": 1.316250205039978,
+      "learning_rate": 7.511978502628482e-06,
+      "loss": 0.1113,
+      "step": 2200
+    },
+    {
+      "epoch": 1.0671153037312984,
+      "grad_norm": 1.3466953039169312,
+      "learning_rate": 7.4669605907843654e-06,
+      "loss": 0.1116,
+      "step": 2220
+    },
+    {
+      "epoch": 1.076728955116265,
+      "grad_norm": 2.4155526161193848,
+      "learning_rate": 7.421676777794376e-06,
+      "loss": 0.1118,
+      "step": 2240
+    },
+    {
+      "epoch": 1.0863426065012318,
+      "grad_norm": 1.2634962797164917,
+      "learning_rate": 7.376131944570597e-06,
+      "loss": 0.1508,
+      "step": 2260
+    },
+    {
+      "epoch": 1.0959562578861983,
+      "grad_norm": 0.6129007339477539,
+      "learning_rate": 7.330331000159161e-06,
+      "loss": 0.105,
+      "step": 2280
+    },
+    {
+      "epoch": 1.105569909271165,
+      "grad_norm": 1.5106947422027588,
+      "learning_rate": 7.284278881211121e-06,
+      "loss": 0.1343,
+      "step": 2300
+    },
+    {
+      "epoch": 1.1151835606561318,
+      "grad_norm": 0.7808032035827637,
+      "learning_rate": 7.237980551450358e-06,
+      "loss": 0.121,
+      "step": 2320
+    },
+    {
+      "epoch": 1.1247972120410983,
+      "grad_norm": 1.4901185035705566,
+      "learning_rate": 7.191441001138567e-06,
+      "loss": 0.0929,
+      "step": 2340
+    },
+    {
+      "epoch": 1.134410863426065,
+      "grad_norm": 1.4136017560958862,
+      "learning_rate": 7.144665246537379e-06,
+      "loss": 0.1312,
+      "step": 2360
+    },
+    {
+      "epoch": 1.1440245148110317,
+      "grad_norm": 1.4810831546783447,
+      "learning_rate": 7.097658329367682e-06,
+      "loss": 0.106,
+      "step": 2380
+    },
+    {
+      "epoch": 1.1536381661959982,
+      "grad_norm": 1.8399866819381714,
+      "learning_rate": 7.050425316266209e-06,
+      "loss": 0.1121,
+      "step": 2400
+    },
+    {
+      "epoch": 1.163251817580965,
+      "grad_norm": 0.4116855263710022,
+      "learning_rate": 7.002971298239419e-06,
+      "loss": 0.1302,
+      "step": 2420
+    },
+    {
+      "epoch": 1.1728654689659317,
+      "grad_norm": 1.8218494653701782,
+      "learning_rate": 6.955301390114771e-06,
+      "loss": 0.1258,
+      "step": 2440
+    },
+    {
+      "epoch": 1.1824791203508982,
+      "grad_norm": 2.300889730453491,
+      "learning_rate": 6.90742072998942e-06,
+      "loss": 0.1311,
+      "step": 2460
+    },
+    {
+      "epoch": 1.192092771735865,
+      "grad_norm": 1.5062252283096313,
+      "learning_rate": 6.859334478676411e-06,
+      "loss": 0.1084,
+      "step": 2480
+    },
+    {
+      "epoch": 1.2017064231208316,
+      "grad_norm": 0.3353615403175354,
+      "learning_rate": 6.811047819148413e-06,
+      "loss": 0.1159,
+      "step": 2500
+    },
+    {
+      "epoch": 1.2113200745057981,
+      "grad_norm": 1.5180888175964355,
+      "learning_rate": 6.762565955979083e-06,
+      "loss": 0.1036,
+      "step": 2520
+    },
+    {
+      "epoch": 1.2209337258907649,
+      "grad_norm": 1.7501280307769775,
+      "learning_rate": 6.713894114782085e-06,
+      "loss": 0.1125,
+      "step": 2540
+    },
+    {
+      "epoch": 1.2305473772757316,
+      "grad_norm": 0.561276376247406,
+      "learning_rate": 6.665037541647845e-06,
+      "loss": 0.1213,
+      "step": 2560
+    },
+    {
+      "epoch": 1.240161028660698,
+      "grad_norm": 1.9739066362380981,
+      "learning_rate": 6.616001502578116e-06,
+      "loss": 0.1098,
+      "step": 2580
+    },
+    {
+      "epoch": 1.2497746800456648,
+      "grad_norm": 1.1374402046203613,
+      "learning_rate": 6.5667912829183615e-06,
+      "loss": 0.1582,
+      "step": 2600
+    },
+    {
+      "epoch": 1.2593883314306316,
+      "grad_norm": 2.0057950019836426,
+      "learning_rate": 6.517412186788089e-06,
+      "loss": 0.1154,
+      "step": 2620
+    },
+    {
+      "epoch": 1.269001982815598,
+      "grad_norm": 1.2539591789245605,
+      "learning_rate": 6.4678695365091455e-06,
+      "loss": 0.1164,
+      "step": 2640
+    },
+    {
+      "epoch": 1.2786156342005648,
+      "grad_norm": 0.8485791087150574,
+      "learning_rate": 6.418168672032048e-06,
+      "loss": 0.0873,
+      "step": 2660
+    },
+    {
+      "epoch": 1.2882292855855315,
+      "grad_norm": 0.7628265023231506,
+      "learning_rate": 6.368314950360416e-06,
+      "loss": 0.1313,
+      "step": 2680
+    },
+    {
+      "epoch": 1.2978429369704982,
+      "grad_norm": 1.0029041767120361,
+      "learning_rate": 6.318313744973566e-06,
+      "loss": 0.1372,
+      "step": 2700
+    },
+    {
+      "epoch": 1.3074565883554647,
+      "grad_norm": 1.3355886936187744,
+      "learning_rate": 6.268170445247344e-06,
+      "loss": 0.1408,
+      "step": 2720
+    },
+    {
+      "epoch": 1.3170702397404315,
+      "grad_norm": 1.5066125392913818,
+      "learning_rate": 6.217890455873213e-06,
+      "loss": 0.1698,
+      "step": 2740
+    },
+    {
+      "epoch": 1.326683891125398,
+      "grad_norm": 2.3855671882629395,
+      "learning_rate": 6.1674791962757255e-06,
+      "loss": 0.1224,
+      "step": 2760
+    },
+    {
+      "epoch": 1.332452081956378,
+      "eval_loss": 0.10718663781881332,
+      "eval_runtime": 206.8134,
+      "eval_samples_per_second": 5.183,
+      "eval_steps_per_second": 5.183,
+      "step": 2772
+    },
+    {
+      "epoch": 1.3362975425103647,
+      "grad_norm": 1.6030802726745605,
+      "learning_rate": 6.11694210002838e-06,
+      "loss": 0.1298,
+      "step": 2780
+    },
+    {
+      "epoch": 1.3459111938953314,
+      "grad_norm": 1.2093404531478882,
+      "learning_rate": 6.066284614267972e-06,
+      "loss": 0.0846,
+      "step": 2800
+    },
+    {
+      "epoch": 1.3555248452802982,
+      "grad_norm": 2.3291215896606445,
+      "learning_rate": 6.015512199107465e-06,
+      "loss": 0.1324,
+      "step": 2820
+    },
+    {
+      "epoch": 1.3651384966652647,
+      "grad_norm": 1.0915559530258179,
+      "learning_rate": 5.964630327047485e-06,
+      "loss": 0.1079,
+      "step": 2840
+    },
+    {
+      "epoch": 1.3747521480502314,
+      "grad_norm": 0.6072287559509277,
+      "learning_rate": 5.913644482386456e-06,
+      "loss": 0.1406,
+      "step": 2860
+    },
+    {
+      "epoch": 1.3843657994351979,
+      "grad_norm": 2.3056912422180176,
+      "learning_rate": 5.8625601606294834e-06,
+      "loss": 0.1312,
+      "step": 2880
+    },
+    {
+      "epoch": 1.3939794508201646,
+      "grad_norm": 1.0984218120574951,
+      "learning_rate": 5.8113828678960175e-06,
+      "loss": 0.1127,
+      "step": 2900
+    },
+    {
+      "epoch": 1.4035931022051313,
+      "grad_norm": 1.8017250299453735,
+      "learning_rate": 5.760118120326377e-06,
+      "loss": 0.1414,
+      "step": 2920
+    },
+    {
+      "epoch": 1.413206753590098,
+      "grad_norm": 2.120990753173828,
+      "learning_rate": 5.708771443487191e-06,
+      "loss": 0.1038,
+      "step": 2940
+    },
+    {
+      "epoch": 1.4228204049750646,
+      "grad_norm": 1.3273861408233643,
+      "learning_rate": 5.657348371775836e-06,
+      "loss": 0.1485,
+      "step": 2960
+    },
+    {
+      "epoch": 1.4324340563600313,
+      "grad_norm": 1.7992357015609741,
+      "learning_rate": 5.605854447823894e-06,
+      "loss": 0.1017,
+      "step": 2980
+    },
+    {
+      "epoch": 1.4420477077449978,
+      "grad_norm": 2.0718681812286377,
+      "learning_rate": 5.554295221899755e-06,
+      "loss": 0.1112,
+      "step": 3000
+    },
+    {
+      "epoch": 1.4516613591299645,
+      "grad_norm": 0.7796396613121033,
+      "learning_rate": 5.502676251310376e-06,
+      "loss": 0.1161,
+      "step": 3020
+    },
+    {
+      "epoch": 1.4612750105149312,
+      "grad_norm": 0.9383320212364197,
+      "learning_rate": 5.451003099802289e-06,
+      "loss": 0.0965,
+      "step": 3040
+    },
+    {
+      "epoch": 1.470888661899898,
+      "grad_norm": 1.632103681564331,
+      "learning_rate": 5.39928133696191e-06,
+      "loss": 0.1079,
+      "step": 3060
+    },
+    {
+      "epoch": 1.4805023132848645,
+      "grad_norm": 0.36383581161499023,
+      "learning_rate": 5.347516537615224e-06,
+      "loss": 0.1199,
+      "step": 3080
+    },
+    {
+      "epoch": 1.4901159646698312,
+      "grad_norm": 3.008059501647949,
+      "learning_rate": 5.295714281226903e-06,
+      "loss": 0.1389,
+      "step": 3100
+    },
+    {
+      "epoch": 1.4997296160547977,
+      "grad_norm": 1.3714076280593872,
+      "learning_rate": 5.243880151298919e-06,
+      "loss": 0.1184,
+      "step": 3120
+    },
+    {
+      "epoch": 1.5093432674397644,
+      "grad_norm": 3.220608711242676,
+      "learning_rate": 5.192019734768739e-06,
+      "loss": 0.1219,
+      "step": 3140
+    },
+    {
+      "epoch": 1.5189569188247312,
+      "grad_norm": 1.9042980670928955,
+      "learning_rate": 5.140138621407127e-06,
+      "loss": 0.1015,
+      "step": 3160
+    },
+    {
+      "epoch": 1.5285705702096979,
+      "grad_norm": 0.3965034484863281,
+      "learning_rate": 5.088242403215644e-06,
+      "loss": 0.1109,
+      "step": 3180
+    },
+    {
+      "epoch": 1.5381842215946644,
+      "grad_norm": 1.1455339193344116,
+      "learning_rate": 5.036336673823935e-06,
+      "loss": 0.1098,
+      "step": 3200
+    },
+    {
+      "epoch": 1.5477978729796311,
+      "grad_norm": 3.0579867362976074,
+      "learning_rate": 4.984427027886805e-06,
+      "loss": 0.1305,
+      "step": 3220
+    },
+    {
+      "epoch": 1.5574115243645976,
+      "grad_norm": 1.3159704208374023,
+      "learning_rate": 4.9325190604812035e-06,
+      "loss": 0.1225,
+      "step": 3240
+    },
+    {
+      "epoch": 1.5670251757495643,
+      "grad_norm": 2.9321792125701904,
+      "learning_rate": 4.8806183665031565e-06,
+      "loss": 0.1398,
+      "step": 3260
+    },
+    {
+      "epoch": 1.576638827134531,
+      "grad_norm": 1.0290374755859375,
+      "learning_rate": 4.828730540064733e-06,
+      "loss": 0.127,
+      "step": 3280
+    },
+    {
+      "epoch": 1.5862524785194978,
+      "grad_norm": 1.1592423915863037,
+      "learning_rate": 4.776861173891062e-06,
+      "loss": 0.1177,
+      "step": 3300
+    },
+    {
+      "epoch": 1.5958661299044643,
+      "grad_norm": 1.7778676748275757,
+      "learning_rate": 4.725015858717549e-06,
+      "loss": 0.0957,
+      "step": 3320
+    },
+    {
+      "epoch": 1.605479781289431,
+      "grad_norm": 1.5151275396347046,
+      "learning_rate": 4.673200182687255e-06,
+      "loss": 0.0973,
+      "step": 3340
+    },
+    {
+      "epoch": 1.6150934326743975,
+      "grad_norm": 1.167035698890686,
+      "learning_rate": 4.621419730748588e-06,
+      "loss": 0.152,
+      "step": 3360
+    },
+    {
+      "epoch": 1.6247070840593643,
+      "grad_norm": 0.4204403758049011,
+      "learning_rate": 4.569680084053336e-06,
+      "loss": 0.135,
+      "step": 3380
+    },
+    {
+      "epoch": 1.634320735444331,
+      "grad_norm": 1.054061770439148,
+      "learning_rate": 4.517986819355092e-06,
+      "loss": 0.1193,
+      "step": 3400
+    },
+    {
+      "epoch": 1.6439343868292977,
+      "grad_norm": 0.7610969543457031,
+      "learning_rate": 4.466345508408173e-06,
+      "loss": 0.0934,
+      "step": 3420
+    },
+    {
+      "epoch": 1.6535480382142642,
+      "grad_norm": 2.2499160766601562,
+      "learning_rate": 4.414761717367063e-06,
+      "loss": 0.111,
+      "step": 3440
+    },
+    {
+      "epoch": 1.663161689599231,
+      "grad_norm": 2.020918607711792,
+      "learning_rate": 4.363241006186466e-06,
+      "loss": 0.0835,
+      "step": 3460
+    },
+    {
+      "epoch": 1.6655651024454725,
+      "eval_loss": 0.10441361367702484,
+      "eval_runtime": 206.479,
+      "eval_samples_per_second": 5.192,
+      "eval_steps_per_second": 5.192,
+      "step": 3465
+    }
+  ],
+  "logging_steps": 20,
+  "max_steps": 6240,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 693,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 7.572306017245594e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-3465/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:228958bc198813ec3c7334720d792de981c3477375cb092239e1e5c068b39593
+size 5624

checkpoint-4158/README.md

@@ -0,0 +1,209 @@
+---
+base_model: Salesforce/Llama-xLAM-2-8b-fc-r
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Salesforce/Llama-xLAM-2-8b-fc-r
+- lora
+- sft
+- transformers
+- trl
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-4158/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Salesforce/Llama-xLAM-2-8b-fc-r",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "q_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}