Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,130 @@
+---
+library_name: transformers
+tags:
+- generated_from_trainer
+model-index:
+- name: lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
+<details><summary>See axolotl config</summary>
+
+axolotl version: `0.12.2`
+```yaml
+base_model: /lustre/fswork/projects/rech/qwv/udv55np/Gemma/base/gemma-3-12b
+
+datasets:
+- path: /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking
+  ds_type: json
+  type: chat_template
+  field_messages: conversations
+  data_files:
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0007.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0009.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0005.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0006.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0014.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0010.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0012.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0008.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0001.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0002.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0013.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0015.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0004.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0011.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0000.jsonl
+  - /lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0003.jsonl
+
+dataset_prepared_path: /lustre/fswork/projects/rech/dgo/udv55np/dataset_gemma/Nemotron-Super-49B-v1_5/split_0
+tokenizer_config: "/lustre/fswork/projects/rech/qwv/udv55np/Gemma/base/gemma-3-27b"
+chat_template: gemma3
+eot_tokens:
+  - "<end_of_turn>"
+
+shuffle_merged_datasets: true
+output_dir: /lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0
+
+sequence_len: 16384
+sample_packing: true
+
+gradient_accumulation_steps: 1
+micro_batch_size: 1
+num_epochs: 0.6
+auto_resume_from_checkpoints: true
+
+optimizer: adamw_torch_fused
+lr_scheduler: warmup_stable_decay
+learning_rate: 2e-6
+lr_scheduler_kwargs:
+  num_decay_steps: 200
+  min_lr_ratio: 0.1
+warmup_steps: 100
+
+bf16: true
+tf32: false
+
+gradient_checkpointing: true
+logging_steps: 10
+flash_attention: true
+
+evals_per_epoch: 0
+saves_per_epoch: 1
+save_total_limit: 20
+save_only_model: true
+
+use_tensorboard: true
+deepspeed: /lustre/fswork/projects/rech/qwv/udv55np/axolotl/zero3.json
+
+```
+
+</details><br>
+
+# lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0
+
+This model was trained from scratch on the None dataset.
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-06
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 16
+- total_train_batch_size: 16
+- total_eval_batch_size: 16
+- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: warmup_stable_decay
+- lr_scheduler_warmup_steps: 100
+- training_steps: 711
+
+### Training results
+
+
+
+### Framework versions
+
+- Transformers 4.55.2
+- Pytorch 2.6.0+cu124
+- Datasets 4.0.0
+- Tokenizers 0.21.1

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<image_soft_token>": 262144
+}

chat_template.jinja

@@ -0,0 +1,47 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

config.json

@@ -0,0 +1,107 @@
+{
+  "architectures": [
+    "Gemma3ForConditionalGeneration"
+  ],
+  "boi_token_index": 255999,
+  "eoi_token_index": 256000,
+  "image_token_index": 262144,
+  "initializer_range": 0.02,
+  "mm_tokens_per_image": 256,
+  "model_type": "gemma3",
+  "text_config": {
+    "_sliding_window_pattern": 6,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "final_logit_softcapping": null,
+    "head_dim": 256,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 3840,
+    "initializer_range": 0.02,
+    "intermediate_size": 15360,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "max_position_embeddings": 131072,
+    "model_type": "gemma3_text",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 48,
+    "num_key_value_heads": 8,
+    "query_pre_attn_scalar": 256,
+    "rms_norm_eps": 1e-06,
+    "rope_local_base_freq": 10000.0,
+    "rope_scaling": {
+      "factor": 8.0,
+      "rope_type": "linear"
+    },
+    "rope_theta": 1000000.0,
+    "sliding_window": 1024,
+    "torch_dtype": "bfloat16",
+    "use_cache": false,
+    "vocab_size": 262208
+  },
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.55.2",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "image_size": 896,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "patch_size": 14,
+    "torch_dtype": "bfloat16",
+    "vision_use_head": false
+  }
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "4.55.2"
+}

model-00001-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48713f75a388bf872e79b3cdf174382aa5b2969c2681393afca5e2d6d8a14763
+size 4979902192

model-00002-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a420cf54d4f15c12aec81b16c493493710751552f296f122a04765066848758
+size 4931296592

model-00003-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac5e315f04c8f2e2fc82758ab450578958518e2b2fb4ec1743775b5f4d6a6683
+size 4931296656

model-00004-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a851bc5d8dfee5bd6a242188d006af820dca1c9775340bb3e543fd5b466041f
+size 4931296656

model-00005-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d2e99d57cd426e6d1e5004830b646ddaf5f52c50fdb117706ffa11356ccb63a
+size 4601000928

preprocessor_config.json

@@ -0,0 +1,29 @@
+{
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_pan_and_scan": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Gemma3ImageProcessor",
+  "image_seq_length": 256,
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "pan_and_scan_max_num_crops": null,
+  "pan_and_scan_min_crop_size": null,
+  "pan_and_scan_min_ratio_to_activate": null,
+  "processor_class": "Gemma3Processor",
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 896,
+    "width": 896
+  }
+}

processor_config.json

@@ -0,0 +1,4 @@
+{
+  "image_seq_length": 256,
+  "processor_class": "Gemma3Processor"
+}

runs/Nov24_00-10-21_jzxh071/events.out.tfevents.1763939522.jzxh071.3081979.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f37098112ad379ad01786305414c07051d2d994842774ecf72b33167987104ed
+size 42188

slurm.out

@@ -0,0 +1,387 @@
+1: W1124 00:08:17.923000 737761 torch/distributed/run.py:792] 
+1: W1124 00:08:17.923000 737761 torch/distributed/run.py:792] *****************************************
+1: W1124 00:08:17.923000 737761 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+1: W1124 00:08:17.923000 737761 torch/distributed/run.py:792] *****************************************
+0: W1124 00:08:17.924000 3081902 torch/distributed/run.py:792] 
+0: W1124 00:08:17.924000 3081902 torch/distributed/run.py:792] *****************************************
+0: W1124 00:08:17.924000 3081902 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+0: W1124 00:08:17.924000 3081902 torch/distributed/run.py:792] *****************************************
+2: W1124 00:08:17.928000 1779991 torch/distributed/run.py:792] 
+2: W1124 00:08:17.928000 1779991 torch/distributed/run.py:792] *****************************************
+2: W1124 00:08:17.928000 1779991 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+2: W1124 00:08:17.928000 1779991 torch/distributed/run.py:792] *****************************************
+3: W1124 00:08:17.934000 3626745 torch/distributed/run.py:792] 
+3: W1124 00:08:17.934000 3626745 torch/distributed/run.py:792] *****************************************
+3: W1124 00:08:17.934000 3626745 torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
+3: W1124 00:08:17.934000 3626745 torch/distributed/run.py:792] *****************************************
+2: [2025-11-24 00:08:36,323] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:119] [PID:1780066] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
+0: [2025-11-24 00:08:36,323] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:119] [PID:3081979] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
+2: [2025-11-24 00:08:36,323] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:218] [PID:1780066] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
+0: [2025-11-24 00:08:36,323] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:218] [PID:3081979] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
+3: [2025-11-24 00:08:36,434] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:119] [PID:3626820] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
+3: [2025-11-24 00:08:36,434] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:218] [PID:3626820] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
+1: [2025-11-24 00:08:36,535] [INFO] [axolotl.utils.schemas.validation.check_eval_packing:119] [PID:737836] [RANK:0] explicitly setting `eval_sample_packing` to match `sample_packing`
+1: [2025-11-24 00:08:36,535] [INFO] [axolotl.utils.schemas.validation.hint_sample_packing_padding:218] [PID:737836] [RANK:0] Setting `pad_to_sequence_len: true` to prevent memory leaks when sample_packing
+0: [2025-11-24 00:08:40,005] [WARNING] [axolotl.utils.config.normalize_config:139] [PID:3081979] [RANK:0] Invalid value for save_steps (1.6666666666666667) from saves_per_epoch and/or num_epochs. Saving at training end only.
+0: [2025-11-24 00:08:40,025] [INFO] [axolotl.cli.config.load_cfg:245] [PID:3081979] [RANK:0] config:
+0: {
+0:   "activation_offloading": false,
+0:   "auto_resume_from_checkpoints": true,
+0:   "axolotl_config_path": "/lustre/fswork/projects/rech/dgo/udv55np/train/tmp/1763939290349239182.yaml",
+0:   "base_model": "/lustre/fswork/projects/rech/qwv/udv55np/Gemma/base/gemma-3-12b",
+0:   "base_model_config": "/lustre/fswork/projects/rech/qwv/udv55np/Gemma/base/gemma-3-12b",
+0:   "batch_size": 16,
+0:   "bf16": true,
+0:   "capabilities": {
+0:     "bf16": true,
+0:     "compute_capability": "sm_90",
+0:     "fp8": false,
+0:     "n_gpu": 16,
+0:     "n_node": 1
+0:   },
+0:   "chat_template": "gemma3",
+0:   "context_parallel_size": 1,
+0:   "dataloader_num_workers": 16,
+0:   "dataloader_pin_memory": true,
+0:   "dataloader_prefetch_factor": 256,
+0:   "dataset_prepared_path": "/lustre/fswork/projects/rech/dgo/udv55np/dataset_gemma/Nemotron-Super-49B-v1_5/split_0",
+0:   "dataset_processes": 192,
+0:   "datasets": [
+0:     {
+0:       "chat_template": "tokenizer_default",
+0:       "data_files": [
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0007.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0009.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0005.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0006.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0014.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0010.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0012.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0008.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0001.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0002.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0013.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0015.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0004.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0011.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0000.jsonl",
+0:         "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking/0003.jsonl"
+0:       ],
+0:       "ds_type": "json",
+0:       "field_messages": "conversations",
+0:       "message_property_mappings": {
+0:         "content": "content",
+0:         "role": "role"
+0:       },
+0:       "path": "/lustre/fswork/projects/rech/qwv/udv55np/dataset/ift/Nemotron-Super-49B-v1_5/no_thinking",
+0:       "trust_remote_code": false,
+0:       "type": "chat_template"
+0:     }
+0:   ],
+0:   "ddp": true,
+0:   "deepspeed": {
+0:     "bf16": {
+0:       "enabled": true
+0:     },
+0:     "gradient_accumulation_steps": "auto",
+0:     "gradient_clipping": "auto",
+0:     "train_batch_size": "auto",
+0:     "train_micro_batch_size_per_gpu": "auto",
+0:     "wall_clock_breakdown": false,
+0:     "zero_optimization": {
+0:       "contiguous_gradients": true,
+0:       "overlap_comm": true,
+0:       "reduce_bucket_size": "auto",
+0:       "stage": 3,
+0:       "stage3_gather_16bit_weights_on_model_save": true,
+0:       "stage3_param_persistence_threshold": "auto",
+0:       "stage3_prefetch_bucket_size": "auto",
+0:       "sub_group_size": 0
+0:     }
+0:   },
+0:   "device": "cuda:0",
+0:   "device_map": {
+0:     "": 0
+0:   },
+0:   "dion_rank_fraction": 1.0,
+0:   "dion_rank_multiple_of": 1,
+0:   "env_capabilities": {
+0:     "torch_version": "2.6.0"
+0:   },
+0:   "eot_tokens": [
+0:     "<end_of_turn>"
+0:   ],
+0:   "eval_batch_size": 1,
+0:   "eval_causal_lm_metrics": [
+0:     "sacrebleu",
+0:     "comet",
+0:     "ter",
+0:     "chrf"
+0:   ],
+0:   "eval_max_new_tokens": 128,
+0:   "eval_sample_packing": true,
+0:   "eval_table_size": 0,
+0:   "evals_per_epoch": 0,
+0:   "flash_attention": true,
+0:   "fp16": false,
+0:   "gradient_accumulation_steps": 1,
+0:   "gradient_checkpointing": true,
+0:   "gradient_checkpointing_kwargs": {
+0:     "use_reentrant": true
+0:   },
+0:   "is_multimodal": true,
+0:   "learning_rate": 2e-06,
+0:   "lisa_layers_attribute": "model.layers",
+0:   "load_best_model_at_end": false,
+0:   "load_in_4bit": false,
+0:   "load_in_8bit": false,
+0:   "local_rank": 0,
+0:   "logging_steps": 10,
+0:   "lora_dropout": 0.0,
+0:   "loraplus_lr_embedding": 1e-06,
+0:   "lr_scheduler": "warmup_stable_decay",
+0:   "lr_scheduler_kwargs": {
+0:     "min_lr_ratio": 0.1,
+0:     "num_decay_steps": 200
+0:   },
+0:   "max_prompt_len": 512,
+0:   "mean_resizing_embeddings": false,
+0:   "micro_batch_size": 1,
+0:   "model_config_type": "gemma3",
+0:   "num_epochs": 0.6,
+0:   "optimizer": "adamw_torch_fused",
+0:   "output_dir": "/lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0",
+0:   "pad_to_sequence_len": true,
+0:   "pretrain_multipack_attn": true,
+0:   "pretrain_multipack_buffer_size": 10000,
+0:   "processor_config": "/lustre/fswork/projects/rech/qwv/udv55np/Gemma/base/gemma-3-12b",
+0:   "profiler_steps_start": 0,
+0:   "qlora_sharded_model_loading": false,
+0:   "ray_num_workers": 1,
+0:   "resources_per_worker": {
+0:     "GPU": 1
+0:   },
+0:   "sample_packing": true,
+0:   "sample_packing_bin_size": 200,
+0:   "sample_packing_group_size": 100000,
+0:   "save_only_model": true,
+0:   "save_safetensors": true,
+0:   "save_total_limit": 20,
+0:   "saves_per_epoch": 1,
+0:   "sequence_len": 16384,
+0:   "shuffle_before_merging_datasets": false,
+0:   "shuffle_merged_datasets": true,
+0:   "skip_prepare_dataset": false,
+0:   "strict": false,
+0:   "tensor_parallel_size": 1,
+0:   "tf32": false,
+0:   "tiled_mlp_use_original_mlp": true,
+0:   "tokenizer_config": "/lustre/fswork/projects/rech/qwv/udv55np/Gemma/base/gemma-3-27b",
+0:   "torch_dtype": "torch.bfloat16",
+0:   "train_on_inputs": false,
+0:   "trl": {
+0:     "log_completions": false,
+0:     "mask_truncated_completions": false,
+0:     "ref_model_mixup_alpha": 0.9,
+0:     "ref_model_sync_steps": 64,
+0:     "scale_rewards": true,
+0:     "sync_ref_model": false,
+0:     "use_vllm": false,
+0:     "vllm_server_host": "0.0.0.0",
+0:     "vllm_server_port": 8000
+0:   },
+0:   "use_ray": false,
+0:   "use_tensorboard": true,
+0:   "val_set_size": 0.0,
+0:   "vllm": {
+0:     "device": "auto",
+0:     "dtype": "auto",
+0:     "gpu_memory_utilization": 0.9,
+0:     "host": "0.0.0.0",
+0:     "port": 8000
+0:   },
+0:   "warmup_steps": 100,
+0:   "weight_decay": 0.0,
+0:   "world_size": 16
+0: }
+0: [2025-11-24 00:08:40,026] [INFO] [axolotl.cli.checks.check_user_token:35] [PID:3081979] [RANK:0] Skipping HuggingFace token verification because HF_HUB_OFFLINE is set to True. Only local files will be used.
+0: [2025-11-24 00:08:41,217] [INFO] [axolotl.utils.data.shared.load_preprocessed_dataset:472] [PID:3081979] [RANK:0] Loading prepared dataset from disk at /lustre/fswork/projects/rech/dgo/udv55np/dataset_gemma/Nemotron-Super-49B-v1_5/split_0/06698e902d3dba325ca34849b1dea5ea...
+0: [2025-11-24 00:09:14,927] [INFO] [axolotl.utils.samplers.multipack.calc_min_len:436] [PID:3081979] [RANK:0] gather_len_batches: [18976, 18976, 18976, 18975, 18977, 18976, 18975, 18976, 18976, 18975, 18976, 18976, 18976, 18976, 18976, 18976]
+0: [2025-11-24 00:09:14,950] [INFO] [axolotl.utils.trainer.calc_sample_packing_eff_est:495] [PID:3081979] [RANK:0] sample_packing_eff_est across ranks: [0.9989354014396667, 0.9988301396369934, 0.9989880323410034, 0.9988827705383301, 0.9988827705383301, 0.9988827705383301, 0.9989354014396667, 0.9989354014396667, 0.9989354014396667, 0.9988827705383301, 0.9989354014396667, 0.9988827705383301, 0.9988827705383301, 0.9988827705383301, 0.9988827705383301, 0.9989354014396667]
+0: [2025-11-24 00:09:14,959] [INFO] [axolotl.utils.data.sft._prepare_standard_dataset:127] [PID:3081979] [RANK:0] Maximum number of steps set at 711
+3: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+1: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+3: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+1: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+2: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+2: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+2: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+2: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+1: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+1: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+0: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+3: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+3: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+0: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+0: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+0: Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
+0: [2025-11-24 00:09:22,718] [INFO] [axolotl.monkeypatch.transformers.trainer_loss_calc.patch_evaluation_loop:110] [PID:3081979] [RANK:0] Patched Trainer.evaluation_loop with nanmean loss calculation
+0: [2025-11-24 00:09:22,719] [INFO] [axolotl.monkeypatch.transformers.trainer_loss_calc.patch_maybe_log_save_evaluate:164] [PID:3081979] [RANK:0] Patched Trainer._maybe_log_save_evaluate with nanmean loss calculation
+3: 
+1: 
+2: 
+0: 
+0: �█    | 3/5 [00:31<00:20, 10.27s/it]
+3: �█    | 3/5 [00:31<00:20, 10.27s/it]
+1: �█    | 3/5 [00:31<00:20, 10.27s/it]
+1: s/it]
+1: 
+3: 
+0: 
+1: 
+3: 
+0: 
+2: �█    | 3/5 [00:31<00:20, 10.27s/it]
+3: 
+2: 
+2: 
+2: 
+1: 
+0: 
+0: [2025-11-24 00:10:19,017] [INFO] [axolotl.loaders.model._configure_embedding_dtypes:345] [PID:3081979] [RANK:0] Converting modules to torch.bfloat16
+0: [2025-11-24 00:10:22,748] [INFO] [axolotl.train.save_initial_configs:416] [PID:3081979] [RANK:0] Pre-saving tokenizer to /lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0...
+0: [2025-11-24 00:10:23,317] [INFO] [axolotl.train.save_initial_configs:419] [PID:3081979] [RANK:0] Pre-saving model config to /lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0...
+0: [2025-11-24 00:10:23,327] [INFO] [axolotl.train.save_initial_configs:423] [PID:3081979] [RANK:0] Pre-saving processor to /lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0...
+0: [2025-11-24 00:10:26,392] [INFO] [axolotl.train.execute_training:203] [PID:3081979] [RANK:0] Starting trainer...
+0: [2025-11-24 00:11:58,358] [INFO] [axolotl.utils.samplers.multipack.calc_min_len:436] [PID:3081979] [RANK:0] gather_len_batches: [18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976, 18976]
+0: Parameter Offload - Persistent parameters statistics: param_count = 563, numel = 1166448
+0: {'loss': 0.6182, 'grad_norm': 2.8189747023390073, 'learning_rate': 3.62e-07, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.01}
+0: 
  0%|          | 0/711 [00:00<?, ?it/s]
  0%|          | 1/711 [03:14<38:20:16, 194.39s/it]
  0%|          | 2/711 [03:19<16:22:13, 83.12s/it] 
  0%|          | 3/711 [03:24<9:20:24, 47.49s/it] 
  1%|          | 4/711 [03:29<6:02:21, 30.75s/it]
  1%|          | 5/711 [03:35<4:14:24, 21.62s/it]
  1%|          | 6/711 [03:40<3:09:35, 16.14s/it]
  1%|          | 7/711 [03:45<2:27:08, 12.54s/it]
  1%|          | 8/711 [03:50<1:59:06, 10.17s/it]
  1%|▏         | 9/711 [03:56<1:40:22,  8.58s/it]
  1%|▏         | 10/711 [04:01<1:27:41,  7.51s/it]
                                                  
  1%|▏         | 10/711 [04:01<1:27:41,  7.51s/it]
  2%|▏         | 11/711 [04:06<1:18:52,  6.76s/it]
  2%|▏         | 12/711 [04:11<1:12:59,  6.27s/it]
  2%|▏         | 13/711 [04:16<1:08:45,  5.91s/it]
  2%|▏         | 14/711 [04:21<1:06:01,  5.68s/it]
  2%|▏         | 15/711 [04:26<1:03:51,  5.51s/it]
  2%|▏         | 16/711 [04:31<1:02:15,  5.38s/it]
  2%|▏         | 17/711 [04:37<1:01:45,  5.34s/i
+0: {'loss': 0.5822, 'grad_norm': 1.7276350224873818, 'learning_rate': 5.420000000000001e-07, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.02}
+0: {'loss': 0.5571, 'grad_norm': 2.161001413543057, 'learning_rate': 7.219999999999999e-07, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.03}
+0: t]
  3%|▎         | 18/711 [04:42<1:01:12,  5.30s/it]
  3%|▎         | 19/711 [04:47<1:00:37,  5.26s/it]
  3%|▎         | 20/711 [04:52<1:00:10,  5.23s/it]
                                                  
  3%|▎         | 20/711 [04:52<1:00:10,  5.23s/it]
  3%|▎         | 21/711 [04:57<59:44,  5.19s/it]  
  3%|▎         | 22/711 [05:02<59:22,  5.17s/it]
  3%|▎         | 23/711 [05:07<59:00,  5.15s/it]
  3%|▎         | 24/711 [05:13<58:57,  5.15s/it]
  4%|▎         | 25/711 [05:18<59:09,  5.17s/it]
  4%|▎         | 26/711 [05:23<58:51,  5.16s/it]
  4%|▍         | 27/711 [05:28<58:27,  5.13s/it]
  4%|▍         | 28/711 [05:33<58:17,  5.12s/it]
  4%|▍         | 29/711 [05:38<58:07,  5.11s/it]
  4%|▍         | 30/711 [05:44<59:10,  5.21s/it]
                                                
  4%|▍         | 30/711 [05:44<59:10,  5.21s/it]
  4%|▍         | 31/711 [05:49<58:50,  5.19s/it]
  5%|▍         | 32/711 [05:54<58:36,  5.18s/it]
  5%|▍         | 33/711 [05:59<59:31,  5
+0: {'loss': 0.5218, 'grad_norm': 1.0906530849609661, 'learning_rate': 9.020000000000001e-07, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.03}
+0: .27s/it]
  5%|▍         | 34/711 [06:04<58:50,  5.22s/it]
  5%|▍         | 35/711 [06:10<58:21,  5.18s/it]
  5%|▌         | 36/711 [06:15<57:59,  5.16s/it]
  5%|▌         | 37/711 [06:20<57:35,  5.13s/it]
  5%|▌         | 38/711 [06:25<57:22,  5.12s/it]
  5%|▌         | 39/711 [06:30<57:16,  5.11s/it]
  6%|▌         | 40/711 [06:35<57:20,  5.13s/it]
                                                
  6%|▌         | 40/711 [06:35<57:20,  5.13s/it]
  6%|▌         | 41/711 [06:40<57:23,  5.14s/it]
  6%|▌         | 42/711 [06:45<57:09,  5.13s/it]
  6%|▌         | 43/711 [06:51<57:31,  5.17s/it]
  6%|▌         | 44/711 [06:56<57:25,  5.17s/it]
  6%|▋         | 45/711 [07:01<57:08,  5.15s/it]
  6%|▋         | 46/711 [07:06<57:01,  5.15s/it]
  7%|▋         | 47/711 [07:11<56:53,  5.14s/it]
  7%|▋         | 48/711 [07:16<57:06,  5.17s/it]
  7%|▋         | 49/711 [07:22<57:03,  5.17s/it]
  7%|▋         | 50/711 [07:27<57:26,  5.21s/it]
                                               
+0: {'loss': 0.4959, 'grad_norm': 0.8804401875885586, 'learning_rate': 1.082e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.04}
+0: {'loss': 0.4729, 'grad_norm': 1.7572079203155466, 'learning_rate': 1.262e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.05}
+0:  
  7%|▋         | 50/711 [07:27<57:26,  5.21s/it]
  7%|▋         | 51/711 [07:32<57:06,  5.19s/it]
  7%|▋         | 52/711 [07:37<57:02,  5.19s/it]
  7%|▋         | 53/711 [07:43<57:29,  5.24s/it]
  8%|▊         | 54/711 [07:48<58:04,  5.30s/it]
  8%|▊         | 55/711 [07:53<57:20,  5.25s/it]
  8%|▊         | 56/711 [07:59<57:54,  5.30s/it]
  8%|▊         | 57/711 [08:04<57:10,  5.25s/it]
  8%|▊         | 58/711 [08:09<57:55,  5.32s/it]
  8%|▊         | 59/711 [08:14<57:11,  5.26s/it]
  8%|▊         | 60/711 [08:19<56:30,  5.21s/it]
                                                
  8%|▊         | 60/711 [08:19<56:30,  5.21s/it]
  9%|▊         | 61/711 [08:25<56:35,  5.22s/it]
  9%|▊         | 62/711 [08:30<56:18,  5.21s/it]
  9%|▉         | 63/711 [08:35<57:08,  5.29s/it]
  9%|▉         | 64/711 [08:41<56:59,  5.28s/it]
  9%|▉         | 65/711 [08:46<56:22,  5.24s/it]
  9%|▉         | 66/711 [08:51<56:36,  5.27s/it]
  9%|▉         | 67/711 [08:56<56:06,  5.23s/it]
 1
+0: {'loss': 0.4737, 'grad_norm': 1.0832691281380182, 'learning_rate': 1.442e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.06}
+0: {'loss': 0.4648, 'grad_norm': 0.9351167776948649, 'learning_rate': 1.622e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.07}
+0: 0%|▉         | 68/711 [09:01<55:32,  5.18s/it]
 10%|▉         | 69/711 [09:06<55:12,  5.16s/it]
 10%|▉         | 70/711 [09:12<55:17,  5.18s/it]
                                                
 10%|▉         | 70/711 [09:12<55:17,  5.18s/it]
 10%|▉         | 71/711 [09:17<54:55,  5.15s/it]
 10%|█         | 72/711 [09:22<54:39,  5.13s/it]
 10%|█         | 73/711 [09:27<54:25,  5.12s/it]
 10%|█         | 74/711 [09:32<54:18,  5.12s/it]
 11%|█         | 75/711 [09:37<54:05,  5.10s/it]
 11%|█         | 76/711 [09:42<54:01,  5.11s/it]
 11%|█         | 77/711 [09:47<53:56,  5.11s/it]
 11%|█         | 78/711 [09:52<54:06,  5.13s/it]
 11%|█         | 79/711 [09:57<53:56,  5.12s/it]
 11%|█▏        | 80/711 [10:03<54:04,  5.14s/it]
                                                
 11%|█▏        | 80/711 [10:03<54:04,  5.14s/it]
 11%|█▏        | 81/711 [10:08<55:07,  5.25s/it]
 12%|█▏        | 82/711 [10:13<54:50,  5.23s/it]
 12%|█▏        | 83/711 [10:19<55:29,  5.30s/it
+0: {'loss': 0.4437, 'grad_norm': 1.0944333242533355, 'learning_rate': 1.802e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.08}
+0: ]
 12%|█▏        | 84/711 [10:24<54:57,  5.26s/it]
 12%|█▏        | 85/711 [10:29<54:21,  5.21s/it]
 12%|█▏        | 86/711 [10:35<55:05,  5.29s/it]
 12%|█▏        | 87/711 [10:40<54:33,  5.25s/it]
 12%|█▏        | 88/711 [10:45<54:02,  5.20s/it]
 13%|█▎        | 89/711 [10:50<53:35,  5.17s/it]
 13%|█▎        | 90/711 [10:55<53:14,  5.14s/it]
                                                
 13%|█▎        | 90/711 [10:55<53:14,  5.14s/it]
 13%|█▎        | 91/711 [11:00<52:59,  5.13s/it]
 13%|█▎        | 92/711 [11:05<53:42,  5.21s/it]
 13%|█▎        | 93/711 [11:11<54:32,  5.30s/it]
 13%|█▎        | 94/711 [11:16<54:50,  5.33s/it]
 13%|█▎        | 95/711 [11:22<55:36,  5.42s/it]
 14%|█▎        | 96/711 [11:27<55:11,  5.38s/it]
 14%|█▎        | 97/711 [11:32<54:13,  5.30s/it]
 14%|█▍        | 98/711 [11:38<54:39,  5.35s/it]
 14%|█▍        | 99/711 [11:43<53:58,  5.29s/it]
 14%|█▍        | 100/711 [11:48<53:15,  5.23s/it]
                 
+0: {'loss': 0.4312, 'grad_norm': 0.821415164120209, 'learning_rate': 1.982e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.08}
+0: {'loss': 0.4519, 'grad_norm': 1.098049116364939, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.09}
+0:                                 
 14%|█▍        | 100/711 [11:48<53:15,  5.23s/it]
 14%|█▍        | 101/711 [11:53<52:46,  5.19s/it]
 14%|█▍        | 102/711 [11:58<52:21,  5.16s/it]
 14%|█▍        | 103/711 [12:03<52:00,  5.13s/it]
 15%|█▍        | 104/711 [12:08<51:44,  5.11s/it]
 15%|█▍        | 105/711 [12:14<51:39,  5.11s/it]
 15%|█▍        | 106/711 [12:19<51:43,  5.13s/it]
 15%|█▌        | 107/711 [12:24<51:34,  5.12s/it]
 15%|█▌        | 108/711 [12:29<51:22,  5.11s/it]
 15%|█▌        | 109/711 [12:34<51:11,  5.10s/it]
 15%|█▌        | 110/711 [12:39<51:14,  5.12s/it]
                                                 
 15%|█▌        | 110/711 [12:39<51:14,  5.12s/it]
 16%|█▌        | 111/711 [12:44<51:43,  5.17s/it]
 16%|█▌        | 112/711 [12:50<51:33,  5.16s/it]
 16%|█▌        | 113/711 [12:55<52:08,  5.23s/it]
 16%|█▌        | 114/711 [13:00<51:48,  5.21s/it]
 16%|█▌        | 115/711 [13:05<51:23,  5.17s/it]
 16%|█▋        | 
+0: {'loss': 0.4418, 'grad_norm': 0.8654847799165983, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.1}
+0: {'loss': 0.4272, 'grad_norm': 0.8743836149172823, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.11}
+0: 116/711 [13:10<51:17,  5.17s/it]
 16%|█▋        | 117/711 [13:16<51:03,  5.16s/it]
 17%|█▋        | 118/711 [13:21<50:59,  5.16s/it]
 17%|█▋        | 119/711 [13:26<51:20,  5.20s/it]
 17%|█▋        | 120/711 [13:31<50:54,  5.17s/it]
                                                 
 17%|█▋        | 120/711 [13:31<50:54,  5.17s/it]
 17%|█▋        | 121/711 [13:36<50:42,  5.16s/it]
 17%|█▋        | 122/711 [13:41<50:37,  5.16s/it]
 17%|█▋        | 123/711 [13:47<50:33,  5.16s/it]
 17%|█▋        | 124/711 [13:52<50:44,  5.19s/it]
 18%|█▊        | 125/711 [13:57<51:31,  5.28s/it]
 18%|█▊        | 126/711 [14:03<52:11,  5.35s/it]
 18%|█▊        | 127/711 [14:08<52:12,  5.36s/it]
 18%|█▊        | 128/711 [14:14<52:10,  5.37s/it]
 18%|█▊        | 129/711 [14:19<51:21,  5.29s/it]
 18%|█▊        | 130/711 [14:24<50:50,  5.25s/it]
                                                 
 18%|█▊        | 130/711 [14:24<50:50,  5.25s/it]
 18%|█▊        | 131/
+0: {'loss': 0.4317, 'grad_norm': 0.886837889977122, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.12}
+0: 711 [14:29<50:24,  5.22s/it]
 19%|█▊        | 132/711 [14:34<50:04,  5.19s/it]
 19%|█▊        | 133/711 [14:39<50:21,  5.23s/it]
 19%|█▉        | 134/711 [14:45<49:58,  5.20s/it]
 19%|█▉        | 135/711 [14:50<49:37,  5.17s/it]
 19%|█▉        | 136/711 [14:55<49:54,  5.21s/it]
 19%|█▉        | 137/711 [15:00<49:29,  5.17s/it]
 19%|█▉        | 138/711 [15:05<49:14,  5.16s/it]
 20%|█▉        | 139/711 [15:10<49:05,  5.15s/it]
 20%|█▉        | 140/711 [15:16<49:19,  5.18s/it]
                                                 
 20%|█▉        | 140/711 [15:16<49:19,  5.18s/it]
 20%|█▉        | 141/711 [15:21<49:04,  5.17s/it]
 20%|█▉        | 142/711 [15:26<48:43,  5.14s/it]
 20%|██        | 143/711 [15:31<48:31,  5.13s/it]
 20%|██        | 144/711 [15:36<48:59,  5.19s/it]
 20%|██        | 145/711 [15:41<48:54,  5.19s/it]
 21%|██        | 146/711 [15:46<48:39,  5.17s/it]
 21%|██        | 147/711 [15:52<48:22,  5.15s/it]
 21%|██        | 148/7
+0: {'loss': 0.4309, 'grad_norm': 1.0717708423744885, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.13}
+0: {'loss': 0.4316, 'grad_norm': 0.8573484702226136, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.13}
+0: 11 [15:57<48:11,  5.14s/it]
 21%|██        | 149/711 [16:02<48:06,  5.14s/it]
 21%|██        | 150/711 [16:07<48:06,  5.14s/it]
                                                 
 21%|██        | 150/711 [16:07<48:06,  5.14s/it]
 21%|██        | 151/711 [16:12<48:00,  5.14s/it]
 21%|██▏       | 152/711 [16:17<48:35,  5.22s/it]
 22%|██▏       | 153/711 [16:23<48:21,  5.20s/it]
 22%|██▏       | 154/711 [16:28<48:00,  5.17s/it]
 22%|██▏       | 155/711 [16:33<47:54,  5.17s/it]
 22%|██▏       | 156/711 [16:38<47:42,  5.16s/it]
 22%|██▏       | 157/711 [16:43<47:36,  5.16s/it]
 22%|██▏       | 158/711 [16:48<47:27,  5.15s/it]
 22%|██▏       | 159/711 [16:54<48:09,  5.23s/it]
 23%|██▎       | 160/711 [16:59<47:40,  5.19s/it]
                                                 
 23%|██▎       | 160/711 [16:59<47:40,  5.19s/it]
 23%|██▎       | 161/711 [17:04<47:17,  5.16s/it]
 23%|██▎       | 162/711 [17:09<47:17,  5.17s/it]
 23%|�
+0: {'loss': 0.4239, 'grad_norm': 0.8728825320101697, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.14}
+0: ��█▎       | 163/711 [17:14<46:56,  5.14s/it]
 23%|██▎       | 164/711 [17:20<47:49,  5.25s/it]
 23%|██▎       | 165/711 [17:25<47:18,  5.20s/it]
 23%|██▎       | 166/711 [17:30<47:54,  5.27s/it]
 23%|██▎       | 167/711 [17:35<47:21,  5.22s/it]
 24%|██▎       | 168/711 [17:41<47:10,  5.21s/it]
 24%|██▍       | 169/711 [17:46<46:57,  5.20s/it]
 24%|██▍       | 170/711 [17:51<46:38,  5.17s/it]
                                                 
 24%|██▍       | 170/711 [17:51<46:38,  5.17s/it]
 24%|██▍       | 171/711 [17:56<46:21,  5.15s/it]
 24%|██▍       | 172/711 [18:01<46:17,  5.15s/it]
 24%|██▍       | 173/711 [18:06<46:18,  5.16s/it]
 24%|██▍       | 174/711 [18:11<46:01,  5.14s/it]
 25%|██▍       | 175/711 [18:16<45:50,  5.13s/it]
 25%|██▍       | 176/711 [18:22<45:49,  5.14s/it]
 25%|██▍       | 177/711 [18:27<45:49,  5.15s/it]
 25%|██▌       | 178/711 [18:32<46:45,  5.26s/it]
 25%|██▌       | 179/
+0: {'loss': 0.4173, 'grad_norm': 2.470513995230686, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.15}
+0: {'loss': 0.4151, 'grad_norm': 0.9038938137872402, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.16}
+0: 711 [18:37<46:12,  5.21s/it]
 25%|██▌       | 180/711 [18:43<45:49,  5.18s/it]
                                                 
 25%|██▌       | 180/711 [18:43<45:49,  5.18s/it]
 25%|██▌       | 181/711 [18:48<45:33,  5.16s/it]
 26%|██▌       | 182/711 [18:53<45:22,  5.15s/it]
 26%|██▌       | 183/711 [18:58<45:10,  5.13s/it]
 26%|██▌       | 184/711 [19:03<45:33,  5.19s/it]
 26%|██▌       | 185/711 [19:08<45:10,  5.15s/it]
 26%|██▌       | 186/711 [19:13<44:56,  5.14s/it]
 26%|██▋       | 187/711 [19:18<44:44,  5.12s/it]
 26%|██▋       | 188/711 [19:24<45:35,  5.23s/it]
 27%|██▋       | 189/711 [19:29<45:10,  5.19s/it]
 27%|██▋       | 190/711 [19:34<44:59,  5.18s/it]
                                                 
 27%|██▋       | 190/711 [19:34<44:59,  5.18s/it]
 27%|██▋       | 191/711 [19:39<44:41,  5.16s/it]
 27%|██▋       | 192/711 [19:44<44:29,  5.14s/it]
 27%|██▋       | 193/711 [19:50<44:25,  5.15s/i
+0: {'loss': 0.4194, 'grad_norm': 2.3527260378633015, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.17}
+0: t]
 27%|██▋       | 194/711 [19:55<44:10,  5.13s/it]
 27%|██▋       | 195/711 [20:00<44:25,  5.17s/it]
 28%|██▊       | 196/711 [20:05<44:10,  5.15s/it]
 28%|██▊       | 197/711 [20:10<44:27,  5.19s/it]
 28%|██▊       | 198/711 [20:16<45:08,  5.28s/it]
 28%|██▊       | 199/711 [20:21<44:34,  5.22s/it]
 28%|██▊       | 200/711 [20:26<44:17,  5.20s/it]
                                                 
 28%|██▊       | 200/711 [20:26<44:17,  5.20s/it]
 28%|██▊       | 201/711 [20:31<43:55,  5.17s/it]
 28%|██▊       | 202/711 [20:36<43:35,  5.14s/it]
 29%|██▊       | 203/711 [20:41<43:37,  5.15s/it]
 29%|██▊       | 204/711 [20:47<44:14,  5.24s/it]
 29%|██▉       | 205/711 [20:52<44:01,  5.22s/it]
 29%|██▉       | 206/711 [20:57<43:43,  5.20s/it]
 29%|██▉       | 207/711 [21:02<43:30,  5.18s/it]
 29%|██▉       | 208/711 [21:07<43:14,  5.16s/it]
 29%|██▉       | 209/711 [21:12<43:02,  5.15s/it]
 30%|██▉    
+0: {'loss': 0.413, 'grad_norm': 0.893185793908122, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.18}
+0: {'loss': 0.4217, 'grad_norm': 1.160958862723743, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.19}
+0:    | 210/711 [21:18<42:50,  5.13s/it]
                                                 
 30%|██▉       | 210/711 [21:18<42:50,  5.13s/it]
 30%|██▉       | 211/711 [21:23<42:40,  5.12s/it]
 30%|██▉       | 212/711 [21:28<42:38,  5.13s/it]
 30%|██▉       | 213/711 [21:33<42:38,  5.14s/it]
 30%|███       | 214/711 [21:38<42:36,  5.14s/it]
 30%|███       | 215/711 [21:43<42:24,  5.13s/it]
 30%|███       | 216/711 [21:48<42:15,  5.12s/it]
 31%|███       | 217/711 [21:53<42:11,  5.12s/it]
 31%|███       | 218/711 [21:59<42:01,  5.12s/it]
 31%|███       | 219/711 [22:04<42:01,  5.12s/it]
 31%|███       | 220/711 [22:09<41:50,  5.11s/it]
                                                 
 31%|███       | 220/711 [22:09<41:50,  5.11s/it]
 31%|███       | 221/711 [22:14<42:03,  5.15s/it]
 31%|███       | 222/711 [22:19<42:36,  5.23s/it]
 31%|███▏      | 223/711 [22:25<42:39,  5.25s/it]
 32%|███▏      | 224/711 [22:30<42
+0: {'loss': 0.4102, 'grad_norm': 0.8461972280700218, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.19}
+0: :22,  5.22s/it]
 32%|███▏      | 225/711 [22:35<42:13,  5.21s/it]
 32%|███▏      | 226/711 [22:40<42:40,  5.28s/it]
 32%|███▏      | 227/711 [22:46<42:11,  5.23s/it]
 32%|███▏      | 228/711 [22:51<41:45,  5.19s/it]
 32%|███▏      | 229/711 [22:56<41:33,  5.17s/it]
 32%|███▏      | 230/711 [23:01<41:16,  5.15s/it]
                                                 
 32%|███▏      | 230/711 [23:01<41:16,  5.15s/it]
 32%|███▏      | 231/711 [23:06<41:02,  5.13s/it]
 33%|███▎      | 232/711 [23:11<40:52,  5.12s/it]
 33%|███▎      | 233/711 [23:16<40:57,  5.14s/it]
 33%|███▎      | 234/711 [23:21<40:58,  5.15s/it]
 33%|███▎      | 235/711 [23:27<40:52,  5.15s/it]
 33%|███▎      | 236/711 [23:32<40:41,  5.14s/it]
 33%|███▎      | 237/711 [23:37<40:32,  5.13s/it]
 33%|███▎      | 238/711 [23:42<40:21,  5.12s/it]
 34%|███▎      | 239/711 [23:47<40:22,  5.13s/it]
 34%|███▍      | 240/
+0: {'loss': 0.4155, 'grad_norm': 0.8314605620161184, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 67.99, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.2}
+0: {'loss': 0.4045, 'grad_norm': 0.8328744939735258, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.21}
+0: 711 [23:52<40:14,  5.13s/it]
                                                 
 34%|███▍      | 240/711 [23:52<40:14,  5.13s/it]
 34%|███▍      | 241/711 [23:57<40:04,  5.12s/it]
 34%|███▍      | 242/711 [24:03<40:30,  5.18s/it]
 34%|███▍      | 243/711 [24:08<40:55,  5.25s/it]
 34%|███▍      | 244/711 [24:13<40:38,  5.22s/it]
 34%|███▍      | 245/711 [24:18<40:16,  5.18s/it]
 35%|███▍      | 246/711 [24:23<40:06,  5.18s/it]
 35%|███▍      | 247/711 [24:29<40:20,  5.22s/it]
 35%|███▍      | 248/711 [24:34<39:58,  5.18s/it]
 35%|███▌      | 249/711 [24:39<39:40,  5.15s/it]
 35%|███▌      | 250/711 [24:44<39:34,  5.15s/it]
                                                 
 35%|███▌      | 250/711 [24:44<39:34,  5.15s/it]
 35%|███▌      | 251/711 [24:49<39:23,  5.14s/it]
 35%|███▌      | 252/711 [24:54<39:08,  5.12s/it]
 36%|███▌      | 253/711 [24:59<39:13,  5.14s/it]
 36%|███▌      
+0: {'loss': 0.4005, 'grad_norm': 0.8810433727017853, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.22}
+0: | 254/711 [25:05<39:10,  5.14s/it]
 36%|███▌      | 255/711 [25:10<39:07,  5.15s/it]
 36%|███▌      | 256/711 [25:15<39:17,  5.18s/it]
 36%|███▌      | 257/711 [25:20<38:57,  5.15s/it]
 36%|███▋      | 258/711 [25:26<39:48,  5.27s/it]
 36%|███▋      | 259/711 [25:31<39:17,  5.22s/it]
 37%|███▋      | 260/711 [25:36<38:58,  5.19s/it]
                                                 
 37%|███▋      | 260/711 [25:36<38:58,  5.19s/it]
 37%|███▋      | 261/711 [25:41<38:40,  5.16s/it]
 37%|███▋      | 262/711 [25:46<38:26,  5.14s/it]
 37%|███▋      | 263/711 [25:51<38:26,  5.15s/it]
 37%|███▋      | 264/711 [25:56<38:28,  5.16s/it]
 37%|███▋      | 265/711 [26:02<38:16,  5.15s/it]
 37%|███▋      | 266/711 [26:07<38:12,  5.15s/it]
 38%|███▊      | 267/711 [26:12<38:50,  5.25s/it]
 38%|███▊      | 268/711 [26:18<39:17,  5.32s/it]
 38%|███▊      | 269/711 [26:23<38:44,  5.26s/it]
 38%|█�
+0: {'loss': 0.4021, 'grad_norm': 1.0060252029086465, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.23}
+0: [2025-11-24 00:38:52,072] [WARNING] [stage3.py:2150:step] 2 pytorch allocator cache flushes since last step. this happens when there is high memory pressure and is detrimental to performance. if this is happening frequently consider adjusting settings to reduce memory consumption. If you are unable to make the cache flushes go away consider adding get_accelerator().empty_cache() calls in your training loop to ensure that all ranks flush their caches at the same time
+0: {'loss': 0.4124, 'grad_norm': 0.9014415482740915, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.24}
+0: �█▊      | 270/711 [26:28<38:17,  5.21s/it]
                                                 
 38%|███▊      | 270/711 [26:28<38:17,  5.21s/it]
 38%|███▊      | 271/711 [26:33<37:58,  5.18s/it]
 38%|███▊      | 272/711 [26:38<37:42,  5.15s/it]
 38%|███▊      | 273/711 [26:43<37:32,  5.14s/it]
 39%|███▊      | 274/711 [26:49<39:58,  5.49s/it]
 39%|███▊      | 275/711 [26:55<39:08,  5.39s/it]
 39%|███▉      | 276/711 [27:00<38:28,  5.31s/it]
 39%|███▉      | 277/711 [27:05<38:44,  5.36s/it]
 39%|███▉      | 278/711 [27:10<38:16,  5.30s/it]
 39%|███▉      | 279/711 [27:16<37:45,  5.24s/it]
 39%|███▉      | 280/711 [27:21<37:26,  5.21s/it]
                                                 
 39%|███▉      | 280/711 [27:21<37:26,  5.21s/it]
 40%|███▉      | 281/711 [27:26<38:16,  5.34s/it]
 40%|███▉      | 282/711 [27:31<37:40,  5.27s/it]
 40%|███▉      | 283/711 [27:37<37:37,  5.28s/it]
 40%
+0: {'loss': 0.3928, 'grad_norm': 1.1303634088009527, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.24}
+0: |███▉      | 284/711 [27:42<37:57,  5.33s/it]
 40%|████      | 285/711 [27:47<37:34,  5.29s/it]
 40%|████      | 286/711 [27:53<37:14,  5.26s/it]
 40%|████      | 287/711 [27:58<37:31,  5.31s/it]
 41%|████      | 288/711 [28:03<37:04,  5.26s/it]
 41%|████      | 289/711 [28:08<36:40,  5.21s/it]
 41%|████      | 290/711 [28:13<36:19,  5.18s/it]
                                                 
 41%|████      | 290/711 [28:13<36:19,  5.18s/it]
 41%|████      | 291/711 [28:19<36:32,  5.22s/it]
 41%|████      | 292/711 [28:24<36:57,  5.29s/it]
 41%|████      | 293/711 [28:29<36:57,  5.31s/it]
 41%|████▏     | 294/711 [28:35<36:36,  5.27s/it]
 41%|████▏     | 295/711 [28:40<36:18,  5.24s/it]
 42%|████▏     | 296/711 [28:45<35:59,  5.20s/it]
 42%|████▏     | 297/711 [28:50<36:22,  5.27s/it]
 42%|████▏     | 298/711 [28:55<36:02,  5.24s/it]
 42%|████▏     | 299/711 [29:
+0: {'loss': 0.4025, 'grad_norm': 0.8789278025527175, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.25}
+0: {'loss': 0.4015, 'grad_norm': 0.7615557087401322, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.26}
+0: 01<35:58,  5.24s/it]
 42%|████▏     | 300/711 [29:06<36:01,  5.26s/it]
                                                 
 42%|████▏     | 300/711 [29:06<36:01,  5.26s/it]
 42%|████▏     | 301/711 [29:11<35:42,  5.23s/it]
 42%|████▏     | 302/711 [29:16<35:28,  5.20s/it]
 43%|████▎     | 303/711 [29:21<35:11,  5.17s/it]
 43%|████▎     | 304/711 [29:27<35:03,  5.17s/it]
 43%|████▎     | 305/711 [29:32<35:16,  5.21s/it]
 43%|████▎     | 306/711 [29:37<34:57,  5.18s/it]
 43%|████▎     | 307/711 [29:42<34:42,  5.15s/it]
 43%|████▎     | 308/711 [29:47<34:33,  5.15s/it]
 43%|████▎     | 309/711 [29:53<35:23,  5.28s/it]
 44%|████▎     | 310/711 [29:59<36:23,  5.45s/it]
                                                 
 44%|████▎     | 310/711 [29:59<36:23,  5.45s/it]
 44%|████▎     | 311/711 [30:04<35:40,  5.35s/it]
 44%|████▍     | 312/711 [30:09<35:47,  5.38s/it]
 
+0: {'loss': 0.4047, 'grad_norm': 1.0096950251075136, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.27}
+0: 44%|████▍     | 313/711 [30:14<35:07,  5.30s/it]
 44%|████▍     | 314/711 [30:20<35:18,  5.34s/it]
 44%|████▍     | 315/711 [30:25<35:09,  5.33s/it]
 44%|████▍     | 316/711 [30:30<34:42,  5.27s/it]
 45%|████▍     | 317/711 [30:35<34:17,  5.22s/it]
 45%|████▍     | 318/711 [30:40<34:02,  5.20s/it]
 45%|████▍     | 319/711 [30:46<33:48,  5.18s/it]
 45%|████▌     | 320/711 [30:51<33:36,  5.16s/it]
                                                 
 45%|████▌     | 320/711 [30:51<33:36,  5.16s/it]
 45%|████▌     | 321/711 [30:56<33:30,  5.16s/it]
 45%|████▌     | 322/711 [31:01<33:19,  5.14s/it]
 45%|████▌     | 323/711 [31:06<33:37,  5.20s/it]
 46%|████▌     | 324/711 [31:11<33:25,  5.18s/it]
 46%|████▌     | 325/711 [31:17<33:17,  5.17s/it]
 46%|████▌     | 326/711 [31:22<33:09,  5.17s/it]
 46%|████▌     | 327/711 [31:27<33:08,  5.18s/it]
 46%|███
+0: {'loss': 0.4025, 'grad_norm': 0.9227721040091849, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.28}
+0: {'loss': 0.3987, 'grad_norm': 1.8038936518267323, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.29}
+0: █▌     | 328/711 [31:32<32:55,  5.16s/it]
 46%|████▋     | 329/711 [31:37<32:44,  5.14s/it]
 46%|████▋     | 330/711 [31:42<32:58,  5.19s/it]
                                                 
 46%|████▋     | 330/711 [31:42<32:58,  5.19s/it]
 47%|████▋     | 331/711 [31:48<32:46,  5.18s/it]
 47%|████▋     | 332/711 [31:53<32:35,  5.16s/it]
 47%|████▋     | 333/711 [31:58<33:05,  5.25s/it]
 47%|████▋     | 334/711 [32:03<32:47,  5.22s/it]
 47%|████▋     | 335/711 [32:09<33:10,  5.29s/it]
 47%|████▋     | 336/711 [32:14<32:41,  5.23s/it]
 47%|████▋     | 337/711 [32:19<32:21,  5.19s/it]
 48%|████▊     | 338/711 [32:24<32:19,  5.20s/it]
 48%|████▊     | 339/711 [32:29<32:05,  5.18s/it]
 48%|████▊     | 340/711 [32:34<31:51,  5.15s/it]
                                                 
 48%|████▊     | 340/711 [32:34<31:51,  5.15s/it]
 48%|████▊     | 341/711 [
+0: {'loss': 0.4004, 'grad_norm': 0.8530478906547682, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.3}
+0: 32:40<31:54,  5.17s/it]
 48%|████▊     | 342/711 [32:45<31:45,  5.16s/it]
 48%|████▊     | 343/711 [32:50<31:37,  5.16s/it]
 48%|████▊     | 344/711 [32:55<31:44,  5.19s/it]
 49%|████▊     | 345/711 [33:00<31:29,  5.16s/it]
 49%|████▊     | 346/711 [33:05<31:18,  5.15s/it]
 49%|████▉     | 347/711 [33:10<31:07,  5.13s/it]
 49%|████▉     | 348/711 [33:16<30:59,  5.12s/it]
 49%|████▉     | 349/711 [33:21<30:57,  5.13s/it]
 49%|████▉     | 350/711 [33:26<30:52,  5.13s/it]
                                                 
 49%|████▉     | 350/711 [33:26<30:52,  5.13s/it]
 49%|████▉     | 351/711 [33:31<30:45,  5.13s/it]
 50%|████▉     | 352/711 [33:36<30:36,  5.12s/it]
 50%|████▉     | 353/711 [33:41<30:32,  5.12s/it]
 50%|████▉     | 354/711 [33:46<30:26,  5.12s/it]
 50%|████▉     | 355/711 [33:51<30:20,  5.11s/it]
 50%|█████     | 356/711 [33:57<30:13, 
+0: {'loss': 0.4055, 'grad_norm': 0.8072887895552343, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.3}
+0:  5.11s/it]
 50%|█████     | 357/711 [34:02<30:10,  5.11s/it]
 50%|█████     | 358/711 [34:07<30:07,  5.12s/it]
 50%|█████     | 359/711 [34:12<30:03,  5.12s/it]
 51%|█████     | 360/711 [34:17<29:57,  5.12s/it]
                                                 
 51%|█████     | 360/711 [34:17<29:57,  5.12s/it]
 51%|█████     | 361/711 [34:22<29:53,  5.12s/it]
 51%|█████     | 362/711 [34:27<29:46,  5.12s/it]
 51%|█████     | 363/711 [34:32<29:41,  5.12s/it]
 51%|█████     | 364/711 [34:38<29:39,  5.13s/it]
 51%|█████▏    | 365/711 [34:43<29:59,  5.20s/it]
 51%|█████▏    | 366/711 [34:48<29:44,  5.17s/it]
 52%|█████▏    | 367/711 [34:53<29:30,  5.15s/it]
 52%|█████▏    | 368/711 [34:58<29:27,  5.15s/it]
 52%|█████▏    | 369/711 [35:03<29:14,  5.13s/it]
 52%|█████▏    | 370/711 [35:09<29:11,  5.14s/it]
                                                 
+0: {'loss': 0.4024, 'grad_norm': 0.8486839849343547, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.31}
+0: {'loss': 0.4021, 'grad_norm': 0.8529581759108179, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.32}
+0: 
 52%|█████▏    | 370/711 [35:09<29:11,  5.14s/it]
 52%|█████▏    | 371/711 [35:14<29:01,  5.12s/it]
 52%|█████▏    | 372/711 [35:19<28:54,  5.12s/it]
 52%|█████▏    | 373/711 [35:24<29:01,  5.15s/it]
 53%|█████▎    | 374/711 [35:29<28:53,  5.14s/it]
 53%|█████▎    | 375/711 [35:34<28:43,  5.13s/it]
 53%|█████▎    | 376/711 [35:39<28:33,  5.11s/it]
 53%|█████▎    | 377/711 [35:44<28:25,  5.11s/it]
 53%|█████▎    | 378/711 [35:49<28:21,  5.11s/it]
 53%|█████▎    | 379/711 [35:55<28:52,  5.22s/it]
 53%|█████▎    | 380/711 [36:00<28:33,  5.18s/it]
                                                 
 53%|█████▎    | 380/711 [36:00<28:33,  5.18s/it]
 54%|█████▎    | 381/711 [36:05<28:53,  5.25s/it]
 54%|█████▎    | 382/711 [36:11<28:32,  5.21s/it]
 54%|█████▍    | 383/711 [36:16<28:38,  5.24s/it]
 54%|█████▍    | 384/711 [36:2
+0: {'loss': 0.3972, 'grad_norm': 0.8357610935717936, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.33}
+0: 1<28:17,  5.19s/it]
 54%|█████▍    | 385/711 [36:26<28:02,  5.16s/it]
 54%|█████▍    | 386/711 [36:31<27:52,  5.15s/it]
 54%|█████▍    | 387/711 [36:36<27:46,  5.14s/it]
 55%|█████▍    | 388/711 [36:41<27:35,  5.12s/it]
 55%|█████▍    | 389/711 [36:47<27:35,  5.14s/it]
 55%|█████▍    | 390/711 [36:52<27:38,  5.17s/it]
                                                 
 55%|█████▍    | 390/711 [36:52<27:38,  5.17s/it]
 55%|█████▍    | 391/711 [36:57<27:26,  5.14s/it]
 55%|█████▌    | 392/711 [37:02<27:35,  5.19s/it]
 55%|█████▌    | 393/711 [37:07<27:27,  5.18s/it]
 55%|█████▌    | 394/711 [37:12<27:12,  5.15s/it]
 56%|█████▌    | 395/711 [37:18<27:39,  5.25s/it]
 56%|█████▌    | 396/711 [37:23<27:19,  5.20s/it]
 56%|█████▌    | 397/711 [37:28<27:03,  5.17s/it]
 56%|█████▌    | 398/711 [37:33<26:52,  5.15s/it]
 56%|█████▌
+0: {'loss': 0.3859, 'grad_norm': 0.8058568338786659, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.34}
+0: {'loss': 0.3898, 'grad_norm': 0.7954384150397931, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.35}
+0:     | 399/711 [37:38<26:46,  5.15s/it]
 56%|█████▋    | 400/711 [37:43<26:35,  5.13s/it]
                                                 
 56%|█████▋    | 400/711 [37:43<26:35,  5.13s/it]
 56%|█████▋    | 401/711 [37:48<26:27,  5.12s/it]
 57%|█████▋    | 402/711 [37:54<26:18,  5.11s/it]
 57%|█████▋    | 403/711 [37:59<26:11,  5.10s/it]
 57%|█████▋    | 404/711 [38:04<26:10,  5.11s/it]
 57%|█████▋    | 405/711 [38:09<26:34,  5.21s/it]
 57%|█████▋    | 406/711 [38:14<26:25,  5.20s/it]
 57%|█████▋    | 407/711 [38:19<26:11,  5.17s/it]
 57%|█████▋    | 408/711 [38:25<26:03,  5.16s/it]
 58%|█████▊    | 409/711 [38:30<26:30,  5.27s/it]
 58%|█████▊    | 410/711 [38:35<26:14,  5.23s/it]
                                                 
 58%|█████▊    | 410/711 [38:35<26:14,  5.23s/it]
 58%|█████▊    | 411/711 [38:41<26:30,  5.30s/it]
 58%|███�
+0: {'loss': 0.3925, 'grad_norm': 0.8145567494437453, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.35}
+0: ��█▊    | 412/711 [38:46<26:10,  5.25s/it]
 58%|█████▊    | 413/711 [38:51<25:55,  5.22s/it]
 58%|█████▊    | 414/711 [38:56<25:40,  5.19s/it]
 58%|█████▊    | 415/711 [39:01<25:28,  5.16s/it]
 59%|█████▊    | 416/711 [39:07<25:49,  5.25s/it]
 59%|█████▊    | 417/711 [39:12<25:31,  5.21s/it]
 59%|█████▉    | 418/711 [39:17<25:17,  5.18s/it]
 59%|█████▉    | 419/711 [39:22<25:06,  5.16s/it]
 59%|█████▉    | 420/711 [39:27<24:59,  5.15s/it]
                                                 
 59%|█████▉    | 420/711 [39:27<24:59,  5.15s/it]
 59%|█████▉    | 421/711 [39:32<24:49,  5.14s/it]
 59%|█████▉    | 422/711 [39:37<24:44,  5.14s/it]
 59%|█████▉    | 423/711 [39:43<24:35,  5.12s/it]
 60%|█████▉    | 424/711 [39:48<24:42,  5.17s/it]
 60%|█████▉    | 425/711 [39:53<24:34,  5.16s/it]
 60%|█████▉    | 426/711 [39:58<24:30,  5.16s/
+0: {'loss': 0.3927, 'grad_norm': 0.8237856091804933, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.36}
+0: it]
 60%|██████    | 427/711 [40:03<24:40,  5.21s/it]
 60%|██████    | 428/711 [40:09<24:55,  5.28s/it]
 60%|██████    | 429/711 [40:14<24:33,  5.23s/it]
 60%|██████    | 430/711 [40:19<24:23,  5.21s/it]
                                                 
 60%|██████    | 430/711 [40:19<24:23,  5.21s/it]
 61%|██████    | 431/711 [40:24<24:10,  5.18s/it]
 61%|██████    | 432/711 [40:29<23:56,  5.15s/it]
 61%|██████    | 433/711 [40:34<23:50,  5.14s/it]
 61%|██████    | 434/711 [40:40<23:39,  5.12s/it]
 61%|██████    | 435/711 [40:45<23:32,  5.12s/it]
 61%|██████▏   | 436/711 [40:50<24:14,  5.29s/it]
 61%|██████▏   | 437/711 [40:56<24:11,  5.30s/it]
 62%|██████▏   | 438/711 [41:01<23:49,  5.24s/it]
 62%|██████▏   | 439/711 [41:06<23:33,  5.20s/it]
 62%|██████▏   | 440/711 [41:11<23:37,  5.23s/it]
                             
+0: {'loss': 0.3937, 'grad_norm': 0.8553439901672909, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.37}
+0: {'loss': 0.3873, 'grad_norm': 0.8286249080798415, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.38}
+0:                     
 62%|██████▏   | 440/711 [41:11<23:37,  5.23s/it]
 62%|██████▏   | 441/711 [41:16<23:23,  5.20s/it]
 62%|██████▏   | 442/711 [41:21<23:13,  5.18s/it]
 62%|██████▏   | 443/711 [41:27<23:01,  5.16s/it]
 62%|██████▏   | 444/711 [41:32<22:57,  5.16s/it]
 63%|██████▎   | 445/711 [41:37<23:14,  5.24s/it]
 63%|██████▎   | 446/711 [41:42<22:58,  5.20s/it]
 63%|██████▎   | 447/711 [41:47<22:45,  5.17s/it]
 63%|██████▎   | 448/711 [41:52<22:34,  5.15s/it]
 63%|██████▎   | 449/711 [41:58<22:43,  5.20s/it]
 63%|██████▎   | 450/711 [42:03<22:32,  5.18s/it]
                                                 
 63%|██████▎   | 450/711 [42:03<22:32,  5.18s/it]
 63%|██████▎   | 451/711 [42:08<22:20,  5.16s/it]
 64%|██████▎   | 452/711 [42:13<22:14,  5.15s/it]
 64%|██████▎   | 453/711 [42:18<22:09,  5
+0: {'loss': 0.384, 'grad_norm': 0.8623716385758442, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.39}
+0: .15s/it]
 64%|██████▍   | 454/711 [42:23<22:03,  5.15s/it]
 64%|██████▍   | 455/711 [42:29<21:56,  5.14s/it]
 64%|██████▍   | 456/711 [42:34<21:51,  5.14s/it]
 64%|██████▍   | 457/711 [42:39<21:44,  5.14s/it]
 64%|██████▍   | 458/711 [42:44<21:40,  5.14s/it]
 65%|██████▍   | 459/711 [42:49<21:39,  5.16s/it]
 65%|██████▍   | 460/711 [42:54<21:46,  5.21s/it]
                                                 
 65%|██████▍   | 460/711 [42:54<21:46,  5.21s/it]
 65%|██████▍   | 461/711 [43:00<21:38,  5.19s/it]
 65%|██████▍   | 462/711 [43:05<21:30,  5.18s/it]
 65%|██████▌   | 463/711 [43:10<21:25,  5.18s/it]
 65%|██████▌   | 464/711 [43:15<21:17,  5.17s/it]
 65%|██████▌   | 465/711 [43:20<21:05,  5.15s/it]
 66%|██████▌   | 466/711 [43:25<20:58,  5.14s/it]
 66%|██████▌   | 467/711 [43:30<20:53,  5.14s/it]
 66%
+0: {'loss': 0.3893, 'grad_norm': 0.7980262942281969, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.4}
+0: {'loss': 0.3928, 'grad_norm': 0.9024656134697462, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.4}
+0: |██████▌   | 468/711 [43:36<20:50,  5.14s/it]
 66%|██████▌   | 469/711 [43:41<21:10,  5.25s/it]
 66%|██████▌   | 470/711 [43:46<20:53,  5.20s/it]
                                                 
 66%|██████▌   | 470/711 [43:46<20:53,  5.20s/it]
 66%|██████▌   | 471/711 [43:51<20:45,  5.19s/it]
 66%|██████▋   | 472/711 [43:57<20:40,  5.19s/it]
 67%|██████▋   | 473/711 [44:02<20:28,  5.16s/it]
 67%|██████▋   | 474/711 [44:07<20:23,  5.16s/it]
 67%|██████▋   | 475/711 [44:12<20:12,  5.14s/it]
 67%|██████▋   | 476/711 [44:18<20:43,  5.29s/it]
 67%|██████▋   | 477/711 [44:23<20:23,  5.23s/it]
 67%|██████▋   | 478/711 [44:28<20:24,  5.26s/it]
 67%|██████▋   | 479/711 [44:33<20:10,  5.22s/it]
 68%|██████▊   | 480/711 [44:38<19:59,  5.19s/it]
                                                 
 68%|██████▊   |
+0: {'loss': 0.3747, 'grad_norm': 1.4532167164219425, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.41}
+0:  480/711 [44:38<19:59,  5.19s/it]
 68%|██████▊   | 481/711 [44:44<20:13,  5.28s/it]
 68%|██████▊   | 482/711 [44:49<20:00,  5.24s/it]
 68%|██████▊   | 483/711 [44:54<19:48,  5.21s/it]
 68%|██████▊   | 484/711 [44:59<19:39,  5.20s/it]
 68%|██████▊   | 485/711 [45:04<19:33,  5.19s/it]
 68%|██████▊   | 486/711 [45:10<19:26,  5.18s/it]
 68%|██████▊   | 487/711 [45:15<19:19,  5.18s/it]
 69%|██████▊   | 488/711 [45:20<19:09,  5.15s/it]
 69%|██████▉   | 489/711 [45:25<19:24,  5.25s/it]
 69%|██████▉   | 490/711 [45:30<19:09,  5.20s/it]
                                                 
 69%|██████▉   | 490/711 [45:30<19:09,  5.20s/it]
 69%|██████▉   | 491/711 [45:36<19:16,  5.26s/it]
 69%|██████▉   | 492/711 [45:41<19:00,  5.21s/it]
 69%|██████▉   | 493/711 [45:46<18:47,  5.17s/it]
 69%|██████▉   | 494/711 [45:
+0: {'loss': 0.3797, 'grad_norm': 0.8355553409451639, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.42}
+0: 51<19:02,  5.27s/it]
 70%|██████▉   | 495/711 [45:56<18:45,  5.21s/it]
 70%|██████▉   | 496/711 [46:02<18:34,  5.19s/it]
 70%|██████▉   | 497/711 [46:07<18:38,  5.22s/it]
 70%|███████   | 498/711 [46:12<18:25,  5.19s/it]
 70%|███████   | 499/711 [46:18<18:55,  5.35s/it]
 70%|███████   | 500/711 [46:23<18:34,  5.28s/it]
                                                 
 70%|███████   | 500/711 [46:23<18:34,  5.28s/it]
 70%|███████   | 501/711 [46:28<18:22,  5.25s/it]
 71%|███████   | 502/711 [46:33<18:12,  5.23s/it]
 71%|███████   | 503/711 [46:38<18:04,  5.21s/it]
 71%|███████   | 504/711 [46:44<17:53,  5.19s/it]
 71%|███████   | 505/711 [46:49<17:43,  5.16s/it]
 71%|███████   | 506/711 [46:54<17:36,  5.16s/it]
 71%|███████▏  | 507/711 [46:59<17:29,  5.14s/it]
 71%|███████▏  | 508/711 [47:04<17:21,
+0: {'loss': 0.3917, 'grad_norm': 0.8425621928447311, 'learning_rate': 2e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.43}
+0: {'loss': 0.3825, 'grad_norm': 0.7816311224212293, 'learning_rate': 1.9929032311830302e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.44}
+0:   5.13s/it]
 72%|███████▏  | 509/711 [47:09<17:16,  5.13s/it]
 72%|███████▏  | 510/711 [47:14<17:09,  5.12s/it]
                                                 
 72%|███████▏  | 510/711 [47:14<17:09,  5.12s/it]
 72%|███████▏  | 511/711 [47:20<17:21,  5.21s/it]
 72%|███████▏  | 512/711 [47:25<17:12,  5.19s/it]
 72%|███████▏  | 513/711 [47:30<17:00,  5.16s/it]
 72%|███████▏  | 514/711 [47:35<16:51,  5.14s/it]
 72%|███████▏  | 515/711 [47:40<16:45,  5.13s/it]
 73%|███████▎  | 516/711 [47:45<16:40,  5.13s/it]
 73%|███████▎  | 517/711 [47:50<16:32,  5.12s/it]
 73%|███████▎  | 518/711 [47:55<16:30,  5.13s/it]
 73%|███████▎  | 519/711 [48:01<16:25,  5.13s/it]
 73%|███████▎  | 520/711 [48:06<16:42,  5.25s/it]
                                                 
 73%|███████▎  | 520/711 [48:06<16:4
+0: {'loss': 0.3819, 'grad_norm': 0.8516269037345293, 'learning_rate': 1.9642643171092486e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.45}
+0: 2,  5.25s/it]
 73%|███████▎  | 521/711 [48:11<16:30,  5.21s/it]
 73%|███████▎  | 522/711 [48:16<16:18,  5.18s/it]
 74%|███████▎  | 523/711 [48:22<16:20,  5.22s/it]
 74%|███████▎  | 524/711 [48:27<16:36,  5.33s/it]
 74%|███████▍  | 525/711 [48:32<16:20,  5.27s/it]
 74%|███████▍  | 526/711 [48:37<16:07,  5.23s/it]
 74%|███████▍  | 527/711 [48:43<15:53,  5.18s/it]
 74%|███████▍  | 528/711 [48:48<15:54,  5.22s/it]
 74%|███████▍  | 529/711 [48:53<15:42,  5.18s/it]
 75%|███████▍  | 530/711 [48:58<15:38,  5.18s/it]
                                                 
 75%|███████▍  | 530/711 [48:58<15:38,  5.18s/it]
 75%|███████▍  | 531/711 [49:03<15:27,  5.15s/it]
 75%|███████▍  | 532/711 [49:08<15:19,  5.14s/it]
 75%|███████▍  | 533/711 [49:13<15:11,  5.12s/it]
 75%|███████▌  | 53
+0: {'loss': 0.3918, 'grad_norm': 0.9451621145813273, 'learning_rate': 1.9143443472194176e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.46}
+0: 4/711 [49:18<15:05,  5.12s/it]
 75%|███████▌  | 535/711 [49:24<14:58,  5.11s/it]
 75%|███████▌  | 536/711 [49:29<15:04,  5.17s/it]
 76%|███████▌  | 537/711 [49:34<14:59,  5.17s/it]
 76%|███████▌  | 538/711 [49:39<14:48,  5.14s/it]
 76%|███████▌  | 539/711 [49:44<14:46,  5.16s/it]
 76%|███████▌  | 540/711 [49:49<14:41,  5.15s/it]
                                                 
 76%|███████▌  | 540/711 [49:49<14:41,  5.15s/it]
 76%|███████▌  | 541/711 [49:55<14:33,  5.14s/it]
 76%|███████▌  | 542/711 [50:00<14:26,  5.13s/it]
 76%|███████▋  | 543/711 [50:05<14:23,  5.14s/it]
 77%|███████▋  | 544/711 [50:10<14:17,  5.13s/it]
 77%|███████▋  | 545/711 [50:15<14:10,  5.12s/it]
 77%|███████▋  | 546/711 [50:20<14:04,  5.12s/it]
 77%|███████▋  | 547/711 [50:25<13:59,  5.12s/it]
 77%|████�
+0: {'loss': 0.3907, 'grad_norm': 0.8481507125427856, 'learning_rate': 1.8443725168471053e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.46}
+0: {'loss': 0.3803, 'grad_norm': 0.8683953024369212, 'learning_rate': 1.7560717646792703e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.47}
+0: ��██▋  | 548/711 [50:30<13:52,  5.11s/it]
 77%|███████▋  | 549/711 [50:35<13:47,  5.11s/it]
 77%|███████▋  | 550/711 [50:41<13:42,  5.11s/it]
                                                 
 77%|███████▋  | 550/711 [50:41<13:42,  5.11s/it]
 77%|███████▋  | 551/711 [50:46<13:40,  5.13s/it]
 78%|███████▊  | 552/711 [50:51<13:36,  5.13s/it]
 78%|███████▊  | 553/711 [50:56<13:29,  5.12s/it]
 78%|███████▊  | 554/711 [51:01<13:24,  5.12s/it]
 78%|███████▊  | 555/711 [51:06<13:21,  5.14s/it]
 78%|███████▊  | 556/711 [51:11<13:14,  5.12s/it]
 78%|███████▊  | 557/711 [51:17<13:25,  5.23s/it]
 78%|███████▊  | 558/711 [51:22<13:32,  5.31s/it]
 79%|███████▊  | 559/711 [51:27<13:18,  5.25s/it]
 79%|███████▉  | 560/711 [51:33<13:06,  5.21s/it]
                                                 
 79%|███�
+0: {'loss': 0.3904, 'grad_norm': 0.8636181194234771, 'learning_rate': 1.6516163482876789e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.48}
+0: �███▉  | 560/711 [51:33<13:06,  5.21s/it]
 79%|███████▉  | 561/711 [51:38<12:58,  5.19s/it]
 79%|███████▉  | 562/711 [51:43<13:13,  5.33s/it]
 79%|███████▉  | 563/711 [51:49<13:11,  5.35s/it]
 79%|███████▉  | 564/711 [51:54<12:58,  5.29s/it]
 79%|███████▉  | 565/711 [51:59<12:47,  5.26s/it]
 80%|███████▉  | 566/711 [52:04<12:34,  5.21s/it]
 80%|███████▉  | 567/711 [52:10<12:40,  5.28s/it]
 80%|███████▉  | 568/711 [52:15<12:28,  5.23s/it]
 80%|████████  | 569/711 [52:20<12:27,  5.26s/it]
 80%|████████  | 570/711 [52:25<12:14,  5.21s/it]
                                                 
 80%|████████  | 570/711 [52:25<12:14,  5.21s/it]
 80%|████████  | 571/711 [52:31<12:14,  5.24s/it]
 80%|████████  | 572/711 [52:36<12:04,  5.21s/it]
 81%|████████  | 573/711 [52:41<12:02,  5.24s/it]
+0: {'loss': 0.3828, 'grad_norm': 0.8879170447340103, 'learning_rate': 1.5335783066915436e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 75.77, 'epoch': 0.49}
+0: 
 81%|████████  | 574/711 [52:46<11:53,  5.21s/it]
 81%|████████  | 575/711 [52:51<11:44,  5.18s/it]
 81%|████████  | 576/711 [52:56<11:38,  5.17s/it]
 81%|████████  | 577/711 [53:02<11:46,  5.27s/it]
 81%|████████▏ | 578/711 [53:07<11:49,  5.33s/it]
 81%|████████▏ | 579/711 [53:12<11:34,  5.26s/it]
 82%|████████▏ | 580/711 [53:18<11:25,  5.23s/it]
                                                 
 82%|████████▏ | 580/711 [53:18<11:25,  5.23s/it]
 82%|████████▏ | 581/711 [53:23<11:15,  5.19s/it]
 82%|████████▏ | 582/711 [53:28<11:06,  5.16s/it]
 82%|████████▏ | 583/711 [53:33<10:59,  5.15s/it]
 82%|████████▏ | 584/711 [53:38<10:52,  5.14s/it]
 82%|████████▏ | 585/711 [53:43<10:46,  5.13s/it]
 82%|████████▏ | 586/711 [53:48<10:42,  5.14s/it]
 83%|███████�
+0: {'loss': 0.3763, 'grad_norm': 0.829865464468096, 'learning_rate': 1.4048641282207622e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.5}
+0: �▎ | 587/711 [53:54<11:03,  5.35s/it]
 83%|████████▎ | 588/711 [53:59<10:55,  5.33s/it]
 83%|████████▎ | 589/711 [54:05<10:40,  5.25s/it]
 83%|████████▎ | 590/711 [54:10<10:43,  5.32s/it]
                                                 
 83%|████████▎ | 590/711 [54:10<10:43,  5.32s/it]
 83%|████████▎ | 591/711 [54:15<10:32,  5.27s/it]
 83%|████████▎ | 592/711 [54:20<10:23,  5.24s/it]
 83%|████████▎ | 593/711 [54:25<10:14,  5.21s/it]
 84%|████████▎ | 594/711 [54:31<10:07,  5.19s/it]
 84%|████████▎ | 595/711 [54:36<09:59,  5.17s/it]
 84%|████████▍ | 596/711 [54:41<09:51,  5.14s/it]
 84%|████████▍ | 597/711 [54:46<09:53,  5.21s/it]
 84%|████████▍ | 598/711 [54:51<09:45,  5.18s/it]
 84%|████████▍ | 599/711 [54:57<10:07,  5.42s/it]
 84%|████████▍ | 600/711 [55:02
+0: {'loss': 0.3895, 'grad_norm': 1.0461353857982227, 'learning_rate': 1.2686431831271522e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.51}
+0: {'loss': 0.3726, 'grad_norm': 0.8291006806562558, 'learning_rate': 1.1282696831703153e-06, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.51}
+0: <09:53,  5.35s/it]
                                                 
 84%|████████▍ | 600/711 [55:02<09:53,  5.35s/it]
 85%|████████▍ | 601/711 [55:08<09:48,  5.35s/it]
 85%|████████▍ | 602/711 [55:13<09:35,  5.28s/it]
 85%|████████▍ | 603/711 [55:18<09:24,  5.23s/it]
 85%|████████▍ | 604/711 [55:23<09:22,  5.26s/it]
 85%|████████▌ | 605/711 [55:28<09:12,  5.22s/it]
 85%|████████▌ | 606/711 [55:34<09:27,  5.41s/it]
 85%|████████▌ | 607/711 [55:40<09:29,  5.47s/it]
 86%|████████▌ | 608/711 [55:45<09:17,  5.41s/it]
 86%|████████▌ | 609/711 [55:50<09:03,  5.32s/it]
 86%|████████▌ | 610/711 [55:56<08:56,  5.31s/it]
                                                 
 86%|████████▌ | 610/711 [55:56<08:56,  5.31s/it]
 86%|████████▌ | 611/711 [56:01<08:45,  5.25s/it]
 86%|████�
+0: {'loss': 0.3749, 'grad_norm': 0.8273940149239204, 'learning_rate': 9.87200089792126e-07, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.52}
+0: �███▌ | 612/711 [56:06<08:36,  5.22s/it]
 86%|████████▌ | 613/711 [56:11<08:34,  5.24s/it]
 86%|████████▋ | 614/711 [56:16<08:25,  5.22s/it]
 86%|████████▋ | 615/711 [56:21<08:17,  5.19s/it]
 87%|████████▋ | 616/711 [56:28<08:41,  5.48s/it]
 87%|████████▋ | 617/711 [56:33<08:24,  5.37s/it]
 87%|████████▋ | 618/711 [56:38<08:23,  5.41s/it]
 87%|████████▋ | 619/711 [56:43<08:09,  5.33s/it]
 87%|████████▋ | 620/711 [56:49<07:59,  5.27s/it]
                                                 
 87%|████████▋ | 620/711 [56:49<07:59,  5.27s/it]
 87%|████████▋ | 621/711 [56:54<07:51,  5.24s/it]
 87%|████████▋ | 622/711 [56:59<07:43,  5.21s/it]
 88%|████████▊ | 623/711 [57:04<07:44,  5.28s/it]
 88%|████████▊ | 624/711 [57:09<07:35,  5.23s/it]
 88%|████████▊ | 625/7
+0: {'loss': 0.3717, 'grad_norm': 0.7609373973543032, 'learning_rate': 8.489080045646937e-07, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.53}
+0: 11 [57:14<07:26,  5.19s/it]
 88%|████████▊ | 626/711 [57:20<07:19,  5.17s/it]
 88%|████████▊ | 627/711 [57:25<07:12,  5.14s/it]
 88%|████████▊ | 628/711 [57:30<07:07,  5.15s/it]
 88%|████████▊ | 629/711 [57:35<07:03,  5.16s/it]
 89%|████████▊ | 630/711 [57:41<07:25,  5.49s/it]
                                                 
 89%|████████▊ | 630/711 [57:41<07:25,  5.49s/it]
 89%|████████▊ | 631/711 [57:47<07:13,  5.42s/it]
 89%|████████▉ | 632/711 [57:52<07:01,  5.33s/it]
 89%|████████▉ | 633/711 [57:57<06:51,  5.28s/it]
 89%|████████▉ | 634/711 [58:02<06:42,  5.23s/it]
 89%|████████▉ | 635/711 [58:07<06:35,  5.21s/it]
 89%|████████▉ | 636/711 [58:13<06:36,  5.28s/it]
 90%|████████▉ | 637/711 [58:18<06:26,  5.23s/it]
 90%|████████▉ | 638/711 [58:23<06:19,  5.2
+0: {'loss': 0.3746, 'grad_norm': 0.8664669621801867, 'learning_rate': 7.167986375914345e-07, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.54}
+0: {'loss': 0.3774, 'grad_norm': 0.7890688699500655, 'learning_rate': 5.941249599330827e-07, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.55}
+0: 0s/it]
 90%|████████▉ | 639/711 [58:28<06:19,  5.28s/it]
 90%|█████████ | 640/711 [58:33<06:11,  5.24s/it]
                                                 
 90%|█████████ | 640/711 [58:33<06:11,  5.24s/it]
 90%|█████████ | 641/711 [58:39<06:10,  5.30s/it]
 90%|█████████ | 642/711 [58:44<06:02,  5.25s/it]
 90%|█████████ | 643/711 [58:49<05:53,  5.20s/it]
 91%|█████████ | 644/711 [58:54<05:46,  5.17s/it]
 91%|█████████ | 645/711 [59:00<05:46,  5.26s/it]
 91%|█████████ | 646/711 [59:05<05:38,  5.21s/it]
 91%|█████████ | 647/711 [59:10<05:31,  5.18s/it]
 91%|█████████ | 648/711 [59:15<05:25,  5.17s/it]
 91%|█████████▏| 649/711 [59:20<05:18,  5.14s/it]
 91%|█████████▏| 650/711 [59:25<05:18,  5.23s/it]
                                                 
 91%|███████�
+0: {'loss': 0.3821, 'grad_norm': 0.8161369753902079, 'learning_rate': 4.839076046641801e-07, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.56}
+0: ��█▏| 650/711 [59:25<05:18,  5.23s/it]
 92%|█████████▏| 651/711 [59:31<05:11,  5.20s/it]
 92%|█████████▏| 652/711 [59:36<05:04,  5.16s/it]
 92%|█████████▏| 653/711 [59:41<05:07,  5.31s/it]
 92%|█████████▏| 654/711 [59:46<04:58,  5.24s/it]
 92%|█████████▏| 655/711 [59:52<04:52,  5.22s/it]
 92%|█████████▏| 656/711 [59:57<04:51,  5.30s/it]
 92%|█████████▏| 657/711 [1:00:02<04:43,  5.26s/it]
 93%|█████████▎| 658/711 [1:00:07<04:36,  5.22s/it]
 93%|█████████▎| 659/711 [1:00:13<04:32,  5.23s/it]
 93%|█████████▎| 660/711 [1:00:18<04:24,  5.19s/it]
                                                   
 93%|█████████▎| 660/711 [1:00:18<04:24,  5.19s/it]
 93%|█████████▎| 661/711 [1:00:23<04:23,  5.28s/it]
 93%|█████████▎| 662/711 [1:00:28<04:16,  5.23s/it]
 93%
+0: {'loss': 0.3655, 'grad_norm': 0.8247018856496126, 'learning_rate': 3.888604888618786e-07, 'memory/max_mem_active(gib)': 69.44, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.56}
+0: |█████████▎| 663/711 [1:00:33<04:09,  5.20s/it]
 93%|█████████▎| 664/711 [1:00:39<04:02,  5.17s/it]
 94%|█████████▎| 665/711 [1:00:44<03:57,  5.16s/it]
 94%|█████████▎| 666/711 [1:00:49<03:54,  5.21s/it]
 94%|█████████▍| 667/711 [1:00:54<03:48,  5.19s/it]
 94%|█████████▍| 668/711 [1:00:59<03:41,  5.16s/it]
 94%|█████████▍| 669/711 [1:01:04<03:35,  5.14s/it]
 94%|█████████▍| 670/711 [1:01:10<03:33,  5.22s/it]
                                                   
 94%|█████████▍| 670/711 [1:01:10<03:33,  5.22s/it]
 94%|█████████▍| 671/711 [1:01:15<03:27,  5.18s/it]
 95%|█████████▍| 672/711 [1:01:20<03:23,  5.22s/it]
 95%|█████████▍| 673/711 [1:01:25<03:17,  5.20s/it]
 95%|█████████▍| 674/711 [1:01:30<03:10,  5.16s/it]
 95%|█████████▍| 67
+0: {'loss': 0.3832, 'grad_norm': 0.7788264476641573, 'learning_rate': 3.1132398796052294e-07, 'memory/max_mem_active(gib)': 69.55, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.57}
+0: 5/711 [1:01:36<03:06,  5.19s/it]
 95%|█████████▌| 676/711 [1:01:42<03:13,  5.53s/it]
 95%|█████████▌| 677/711 [1:01:48<03:11,  5.63s/it]
 95%|█████████▌| 678/711 [1:01:53<03:02,  5.52s/it]
 95%|█████████▌| 679/711 [1:01:58<02:52,  5.38s/it]
 96%|█████████▌| 680/711 [1:02:03<02:44,  5.29s/it]
                                                   
 96%|█████████▌| 680/711 [1:02:03<02:44,  5.29s/it]
 96%|█████████▌| 681/711 [1:02:08<02:37,  5.25s/it]
 96%|█████████▌| 682/711 [1:02:14<02:33,  5.31s/it]
 96%|█████████▌| 683/711 [1:02:19<02:27,  5.25s/it]
 96%|█████████▌| 684/711 [1:02:24<02:20,  5.21s/it]
 96%|█████████▋| 685/711 [1:02:29<02:14,  5.18s/it]
 96%|█████████▋| 686/711 [1:02:34<02:09,  5.17s/it]
 97%|█████████▋| 687/711 [1:02:39<02:03,  5.17s/it]
 9
+0: {'loss': 0.3818, 'grad_norm': 0.7875994058840892, 'learning_rate': 2.532073079411971e-07, 'memory/max_mem_active(gib)': 69.55, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.58}
+0: 7%|█████████▋| 688/711 [1:02:45<01:58,  5.15s/it]
 97%|█████████▋| 689/711 [1:02:50<01:53,  5.15s/it]
 97%|█████████▋| 690/711 [1:02:55<01:47,  5.14s/it]
                                                   
 97%|█████████▋| 690/711 [1:02:55<01:47,  5.14s/it]
 97%|█████████▋| 691/711 [1:03:00<01:43,  5.19s/it]
 97%|█████████▋| 692/711 [1:03:05<01:37,  5.16s/it]
 97%|█████████▋| 693/711 [1:03:10<01:32,  5.14s/it]
 98%|█████████▊| 694/711 [1:03:15<01:27,  5.13s/it]
 98%|█████████▊| 695/711 [1:03:21<01:22,  5.13s/it]
 98%|█████████▊| 696/711 [1:03:26<01:17,  5.14s/it]
 98%|█████████▊| 697/711 [1:03:31<01:11,  5.14s/it]
 98%|█████████▊| 698/711 [1:03:36<01:06,  5.14s/it]
 98%|█████████▊| 699/711 [1:03:41<01:02,  5.17s/it]
 98%|█████████▊| 
+0: {'loss': 0.3763, 'grad_norm': 0.7322258611860605, 'learning_rate': 2.1594147434418026e-07, 'memory/max_mem_active(gib)': 69.55, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.59}
+0: {'loss': 0.3734, 'grad_norm': 0.7653492019720352, 'learning_rate': 2.0044409567084156e-07, 'memory/max_mem_active(gib)': 69.55, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.6}
+0: 700/711 [1:03:47<00:57,  5.24s/it]
                                                   
 98%|█████████▊| 700/711 [1:03:47<00:57,  5.24s/it]
 99%|█████████▊| 701/711 [1:03:52<00:52,  5.26s/it]
 99%|█████████▊| 702/711 [1:03:57<00:47,  5.23s/it]
 99%|█████████▉| 703/711 [1:04:02<00:41,  5.20s/it]
 99%|█████████▉| 704/711 [1:04:07<00:36,  5.20s/it]
 99%|█████████▉| 705/711 [1:04:13<00:31,  5.30s/it]
 99%|█████████▉| 706/711 [1:04:18<00:26,  5.24s/it]
 99%|█████████▉| 707/711 [1:04:23<00:21,  5.27s/it]
                                                   
+0: [2025-11-24 01:16:53,191] [INFO] [axolotl.core.trainers.base._save:613] [PID:3081979] [RANK:0] Saving model checkpoint to /lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0/checkpoint-711
+0: [2025-11-24 01:17:11,725] [INFO] [axolotl.core.trainers.base._save:662] [PID:3081979] [RANK:0] Saving Trainer.data_collator.tokenizer by default as Trainer.processing_class is `None`
+0: {'train_runtime': 3910.4069, 'train_samples_per_second': 2.909, 'train_steps_per_second': 0.182, 'train_loss': 0.4125736778295493, 'memory/max_mem_active(gib)': 69.55, 'memory/max_mem_allocated(gib)': 67.66, 'memory/device_mem_reserved(gib)': 76.53, 'epoch': 0.6}
+0: �█████| 711/711 [1:04:44<00:00,  5.22s/it]
                                                   
+0: [2025-11-24 01:17:21,056] [INFO] [axolotl.train.save_trained_model:228] [PID:3081979] [RANK:0] Training completed! Saving trained model to /lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0.
+0: [2025-11-24 01:17:26,694] [INFO] [axolotl.core.trainers.base._save:613] [PID:3081979] [RANK:0] Saving model checkpoint to /lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0
+0: [2025-11-24 01:17:44,493] [INFO] [axolotl.core.trainers.base._save:662] [PID:3081979] [RANK:0] Saving Trainer.data_collator.tokenizer by default as Trainer.processing_class is `None`
+0: [2025-11-24 01:17:44,817] [INFO] [axolotl.train.save_trained_model:350] [PID:3081979] [RANK:0] Model successfully saved to /lustre/fswork/projects/rech/dgo/udv55np/ift/Nemotron-Super-49B-v1_5/gemma-3-12b/0

special_tokens_map.json

@@ -0,0 +1,33 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d94a47440372bd382b8068468e71d951722420260c48543cdcd097f95f9ee7fb
+size 10424