# NeuronTrainer

Training classes for AWS Trainium accelerators.

## NeuronTrainingArguments[[optimum.neuron.NeuronTrainingArguments]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class optimum.neuron.NeuronTrainingArguments</name><anchor>optimum.neuron.NeuronTrainingArguments</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/training_args.py#L51</source><parameters>[{"name": "output_dir", "val": ": str | None = None"}, {"name": "overwrite_output_dir", "val": ": bool = False"}, {"name": "do_train", "val": ": bool = False"}, {"name": "do_eval", "val": ": bool = False"}, {"name": "eval_strategy", "val": ": transformers.trainer_utils.IntervalStrategy | str = 'no'"}, {"name": "per_device_train_batch_size", "val": ": int = 1"}, {"name": "per_device_eval_batch_size", "val": ": int = 1"}, {"name": "gradient_accumulation_steps", "val": ": int = 1"}, {"name": "learning_rate", "val": ": float = 5e-05"}, {"name": "weight_decay", "val": ": float = 0.0"}, {"name": "adam_beta1", "val": ": float = 0.9"}, {"name": "adam_beta2", "val": ": float = 0.999"}, {"name": "adam_epsilon", "val": ": float = 1e-08"}, {"name": "max_grad_norm", "val": ": float = 1.0"}, {"name": "num_train_epochs", "val": ": float = 3.0"}, {"name": "max_steps", "val": ": int = -1"}, {"name": "lr_scheduler_type", "val": ": transformers.trainer_utils.SchedulerType | str = 'linear'"}, {"name": "lr_scheduler_kwargs", "val": ": dict[str, typing.Any] | str | None = <factory>"}, {"name": "warmup_ratio", "val": ": float = 0.0"}, {"name": "warmup_steps", "val": ": int = 0"}, {"name": "log_level", "val": ": str = 'info'"}, {"name": "log_level_replica", "val": ": str = 'silent'"}, {"name": "logging_dir", "val": ": str | None = None"}, {"name": "logging_strategy", "val": ": transformers.trainer_utils.IntervalStrategy | str = 'steps'"}, {"name": "logging_first_step", "val": ": bool = False"}, {"name": "logging_steps", "val": ": float = 500"}, {"name": "save_strategy", "val": ": transformers.trainer_utils.SaveStrategy | str = 'steps'"}, {"name": "save_steps", "val": ": float = 500"}, {"name": "save_total_limit", "val": ": int | None = None"}, {"name": "save_only_model", "val": ": bool = False"}, {"name": "restore_callback_states_from_checkpoint", "val": ": bool = False"}, {"name": "seed", "val": ": int = 42"}, {"name": "bf16", "val": ": bool = False"}, {"name": "dataloader_drop_last", "val": ": bool = False"}, {"name": "eval_steps", "val": ": float | None = None"}, {"name": "dataloader_num_workers", "val": ": int = 0"}, {"name": "dataloader_prefetch_factor", "val": ": int | None = None"}, {"name": "run_name", "val": ": str | None = None"}, {"name": "disable_tqdm", "val": ": bool | None = None"}, {"name": "remove_unused_columns", "val": ": bool | None = True"}, {"name": "label_names", "val": ": list[str] | None = None"}, {"name": "accelerator_config", "val": ": dict | str | None = None"}, {"name": "label_smoothing_factor", "val": ": float = 0.0"}, {"name": "optim", "val": ": transformers.training_args.OptimizerNames | str = 'adamw_torch'"}, {"name": "optim_args", "val": ": str | None = None"}, {"name": "report_to", "val": ": None | str | list[str] = None"}, {"name": "resume_from_checkpoint", "val": ": str | None = None"}, {"name": "gradient_checkpointing", "val": ": bool = False"}, {"name": "gradient_checkpointing_kwargs", "val": ": dict[str, typing.Any] | str | None = None"}, {"name": "use_liger_kernel", "val": ": bool | None = False"}, {"name": "average_tokens_across_devices", "val": ": bool | None = False"}, {"name": "dataloader_prefetch_size", "val": ": int = None"}, {"name": "skip_cache_push", "val": ": bool = False"}, {"name": "use_autocast", "val": ": bool = False"}, {"name": "zero_1", "val": ": bool = True"}, {"name": "stochastic_rounding_enabled", "val": ": bool = True"}, {"name": "optimizer_save_master_weights_in_ckpt", "val": ": bool = False"}, {"name": "tensor_parallel_size", "val": ": int = 1"}, {"name": "disable_sequence_parallel", "val": ": bool = False"}, {"name": "pipeline_parallel_size", "val": ": int = 1"}, {"name": "pipeline_parallel_num_microbatches", "val": ": int = -1"}, {"name": "kv_size_multiplier", "val": ": int | None = None"}, {"name": "num_local_ranks_per_step", "val": ": int = 8"}, {"name": "use_xser", "val": ": bool = True"}, {"name": "async_save", "val": ": bool = False"}, {"name": "fuse_qkv", "val": ": bool = False"}, {"name": "recompute_causal_mask", "val": ": bool = True"}]</parameters></docstring>



<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_process_log_level</name><anchor>optimum.neuron.NeuronTrainingArguments.get_process_log_level</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/training_args.py#L702</source><parameters>[]</parameters></docstring>

Returns the log level to be used depending on whether this process is the main process of node 0, main process
of node non-0, or a non-main process.

For the main process the log level defaults to the logging level set (`logging.WARNING` if you didn't do
anything) unless overridden by `log_level` argument.

For the replica processes the log level defaults to `logging.WARNING` unless overridden by `log_level_replica`
argument.

The choice between the main and replica process settings is made according to the return value of `should_log`.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_warmup_steps</name><anchor>optimum.neuron.NeuronTrainingArguments.get_warmup_steps</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/training_args.py#L724</source><parameters>[{"name": "num_training_steps", "val": ": int"}]</parameters></docstring>

Get number of steps used for a linear warmup.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>to_dict</name><anchor>optimum.neuron.NeuronTrainingArguments.to_dict</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/training_args.py#L744</source><parameters>[]</parameters></docstring>

Serializes this instance while replace `Enum` by their values (for JSON serialization support). It obfuscates
the token values by removing their value.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>to_json_string</name><anchor>optimum.neuron.NeuronTrainingArguments.to_json_string</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/training_args.py#L771</source><parameters>[]</parameters></docstring>

Serializes this instance to a JSON string.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>to_sanitized_dict</name><anchor>optimum.neuron.NeuronTrainingArguments.to_sanitized_dict</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/training_args.py#L777</source><parameters>[]</parameters></docstring>

Sanitized serialization to use with TensorBoard’s hparams


</div></div>

## NeuronTrainer[[optimum.neuron.NeuronTrainer]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class optimum.neuron.NeuronTrainer</name><anchor>optimum.neuron.NeuronTrainer</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L114</source><parameters>[{"name": "model", "val": ": transformers.modeling_utils.PreTrainedModel | torch.nn.modules.module.Module"}, {"name": "args", "val": ": NeuronTrainingArguments"}, {"name": "data_collator", "val": ": typing.Optional[transformers.data.data_collator.DataCollator] = None"}, {"name": "train_dataset", "val": ": Dataset | IterableDataset | datasets.Dataset | None = None"}, {"name": "eval_dataset", "val": ": Dataset | dict[str, Dataset] | datasets.Dataset | None = None"}, {"name": "processing_class", "val": ": transformers.tokenization_utils_base.PreTrainedTokenizerBase | transformers.image_processing_utils.BaseImageProcessor | transformers.feature_extraction_utils.FeatureExtractionMixin | transformers.processing_utils.ProcessorMixin | None = None"}, {"name": "callbacks", "val": ": list[transformers.trainer_callback.TrainerCallback] | None = None"}, {"name": "optimizers", "val": ": tuple[torch.optim.optimizer.Optimizer | None, torch.optim.lr_scheduler.LambdaLR | None] = (None, None)"}, {"name": "optimizer_cls_and_kwargs", "val": ": tuple[type[torch.optim.optimizer.Optimizer], dict[str, typing.Any]] | None = None"}, {"name": "tokenizer", "val": ": transformers.tokenization_utils_base.PreTrainedTokenizerBase | None = None"}]</parameters></docstring>



<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>add_callback</name><anchor>optimum.neuron.NeuronTrainer.add_callback</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L367</source><parameters>[{"name": "callback", "val": ": typing.Union[typing.Type[transformers.trainer_callback.TrainerCallback], transformers.trainer_callback.TrainerCallback]"}]</parameters><paramsdesc>- **callback** (`Type[TrainerCallback] | TrainerCallback`) --
  A `TrainerCallback` class or an instance of a `TrainerCallback`. In the
  first case, will instantiate a member of that class.</paramsdesc><paramgroups>0</paramgroups></docstring>

Add a callback to the current list of `TrainerCallback`.




</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>autocast_smart_context_manager</name><anchor>optimum.neuron.NeuronTrainer.autocast_smart_context_manager</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L727</source><parameters>[{"name": "cache_enabled", "val": ": bool | None = True"}]</parameters></docstring>

A helper wrapper that creates an appropriate context manager for `autocast` while feeding it the desired
arguments, depending on the situation.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_accelerator_and_postprocess</name><anchor>optimum.neuron.NeuronTrainer.create_accelerator_and_postprocess</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L283</source><parameters>[]</parameters></docstring>
Creates NeuronAccelerator instance and prepares model for distributed training.

</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_optimizer</name><anchor>optimum.neuron.NeuronTrainer.create_optimizer</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L561</source><parameters>[]</parameters></docstring>

Setup the optimizer.

We provide a reasonable default that works well. If you want to use something else, you can pass a tuple in the
NeuronTrainer's init through `optimizers`, or subclass and override this method in a subclass.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_optimizer_and_scheduler</name><anchor>optimum.neuron.NeuronTrainer.create_optimizer_and_scheduler</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L539</source><parameters>[{"name": "num_training_steps", "val": ": int"}]</parameters></docstring>

Setup the optimizer and the learning rate scheduler.

We provide a reasonable default that works well. If you want to use something else, you can pass a tuple in the
NeuronTrainer's init through `optimizers`, or subclass and override this method (or `create_optimizer` and/or
`create_scheduler`) in a subclass.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>create_scheduler</name><anchor>optimum.neuron.NeuronTrainer.create_scheduler</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L675</source><parameters>[{"name": "num_training_steps", "val": ": int"}, {"name": "optimizer", "val": ": torch.optim.optimizer.Optimizer | None = None"}]</parameters><paramsdesc>- **num_training_steps** (int) -- The number of training steps to do.</paramsdesc><paramgroups>0</paramgroups></docstring>

Setup the scheduler. The optimizer of the trainer must have been set up either before this method is called or
passed as an argument.




</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_decay_parameter_names</name><anchor>optimum.neuron.NeuronTrainer.get_decay_parameter_names</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L550</source><parameters>[{"name": "model", "val": ""}]</parameters></docstring>

Get all parameter names that weight decay will be applied to.

This function filters out parameters in two ways:
1. By layer type (instances of layers specified in ALL_LAYERNORM_LAYERS)
2. By parameter name patterns (containing 'bias', 'layernorm', or 'rmsnorm')


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_learning_rates</name><anchor>optimum.neuron.NeuronTrainer.get_learning_rates</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L618</source><parameters>[]</parameters></docstring>

Returns the learning rate of each parameter from self.optimizer.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_num_trainable_parameters</name><anchor>optimum.neuron.NeuronTrainer.get_num_trainable_parameters</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L612</source><parameters>[]</parameters></docstring>

Get the number of trainable parameters.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_optimizer_cls_and_kwargs</name><anchor>optimum.neuron.NeuronTrainer.get_optimizer_cls_and_kwargs</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L642</source><parameters>[{"name": "args", "val": ": TrainingArguments"}, {"name": "model", "val": ": transformers.modeling_utils.PreTrainedModel | None = None"}]</parameters><paramsdesc>- **args** (`transformers.training_args.TrainingArguments`) --
  The training arguments for the training session.</paramsdesc><paramgroups>0</paramgroups></docstring>

Returns the optimizer class and optimizer parameters based on the training arguments.




</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_optimizer_group</name><anchor>optimum.neuron.NeuronTrainer.get_optimizer_group</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L626</source><parameters>[{"name": "param", "val": ": str | torch.nn.parameter.Parameter | None = None"}]</parameters><paramsdesc>- **param** (`str | torch.nn.parameter.Parameter | None`, defaults to `None`) --
  The parameter for which optimizer group needs to be returned.</paramsdesc><paramgroups>0</paramgroups></docstring>

Returns optimizer group for a parameter if given, else returns all optimizer groups for params.




</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>get_train_dataloader</name><anchor>optimum.neuron.NeuronTrainer.get_train_dataloader</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L520</source><parameters>[]</parameters></docstring>
Returns the training DataLoader with appropriate sampler and batch size.

</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>is_local_process_zero</name><anchor>optimum.neuron.NeuronTrainer.is_local_process_zero</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L1166</source><parameters>[]</parameters></docstring>

Whether or not this process is the local (e.g., on one machine if training in a distributed fashion on several
machines) main process.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>is_world_process_zero</name><anchor>optimum.neuron.NeuronTrainer.is_world_process_zero</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L1173</source><parameters>[]</parameters></docstring>

Whether or not this process is the global main process (when training in a distributed fashion on several
machines, this is only going to be `True` for one process).


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>log</name><anchor>optimum.neuron.NeuronTrainer.log</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L1233</source><parameters>[{"name": "logs", "val": ": dict[str, float]"}]</parameters></docstring>
Log training metrics to the state history and callbacks.

</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>maybe_log_train_step_metrics</name><anchor>optimum.neuron.NeuronTrainer.maybe_log_train_step_metrics</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L989</source><parameters>[]</parameters></docstring>
Log training step metrics if logging is due.

</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>maybe_save_checkpoint</name><anchor>optimum.neuron.NeuronTrainer.maybe_save_checkpoint</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L1025</source><parameters>[]</parameters></docstring>
Save checkpoint if saving is due.

</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>num_examples</name><anchor>optimum.neuron.NeuronTrainer.num_examples</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L695</source><parameters>[{"name": "dataloader", "val": ": DataLoader"}]</parameters></docstring>

Helper to get number of samples in a `~torch.utils.data.DataLoader` by accessing its dataset. When
dataloader.dataset does not exist or has no length, estimates as best it can


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>num_tokens</name><anchor>optimum.neuron.NeuronTrainer.num_tokens</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L709</source><parameters>[{"name": "train_dl", "val": ": DataLoader"}, {"name": "max_steps", "val": ": int | None = None"}]</parameters></docstring>

Helper to get number of tokens in a `~torch.utils.data.DataLoader` by enumerating dataloader.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>pop_callback</name><anchor>optimum.neuron.NeuronTrainer.pop_callback</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L378</source><parameters>[{"name": "callback", "val": ": typing.Union[typing.Type[transformers.trainer_callback.TrainerCallback], transformers.trainer_callback.TrainerCallback]"}]</parameters><paramsdesc>- **callback** (`Type[TrainerCallback] | TrainerCallback`) --
  A `TrainerCallback` class or an instance of a `TrainerCallback`. In the
  first case, will pop the first member of that class found in the list of callbacks.</paramsdesc><paramgroups>0</paramgroups><rettype>`TrainerCallback | None`</rettype><retdesc>The callback removed, if found.</retdesc></docstring>

Remove a callback from the current list of `TrainerCallback` and returns it.

If the callback is not found, returns `None` (and no error is raised).








</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>remove_callback</name><anchor>optimum.neuron.NeuronTrainer.remove_callback</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L394</source><parameters>[{"name": "callback", "val": ": typing.Union[typing.Type[transformers.trainer_callback.TrainerCallback], transformers.trainer_callback.TrainerCallback]"}]</parameters><paramsdesc>- **callback** (`Type[TrainerCallback] | TrainerCallback`) --
  A `TrainerCallback` class or an instance of a `TrainerCallback`. In the
  first case, will remove the first member of that class found in the list of callbacks.</paramsdesc><paramgroups>0</paramgroups></docstring>

Remove a callback from the current list of `TrainerCallback`.




</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>set_initial_training_values</name><anchor>optimum.neuron.NeuronTrainer.set_initial_training_values</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L738</source><parameters>[{"name": "args", "val": ": NeuronTrainingArguments"}, {"name": "dataloader", "val": ": DataLoader"}, {"name": "total_train_batch_size", "val": ": int"}]</parameters></docstring>

Calculates and returns the following values:
- `num_train_epochs`
- `num_update_steps_per_epoch`
- `num_examples`
- `num_train_samples`
- `epoch_based`
- `len_dataloader`
- `max_steps`


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>setup_training</name><anchor>optimum.neuron.NeuronTrainer.setup_training</anchor><source>https://github.com/huggingface/optimum-neuron/blob/main/optimum/neuron/trainers/transformers.py#L802</source><parameters>[{"name": "train_dataloader", "val": ": DataLoader"}, {"name": "max_steps", "val": ": int"}, {"name": "num_train_epochs", "val": ": int"}, {"name": "num_examples", "val": ": int"}, {"name": "total_train_batch_size", "val": ": int"}]</parameters></docstring>

Setup everything to prepare for the training loop.
This methods does not return anything but initializes many attributes of the class for training.


</div></div>