# FP8

Below are functions and classes relative to the underlying FP8 implementation

## FP8RecipeKwargs[[accelerate.utils.FP8RecipeKwargs]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class accelerate.utils.FP8RecipeKwargs</name><anchor>accelerate.utils.FP8RecipeKwargs</anchor><source>https://github.com/huggingface/accelerate/blob/main/src/accelerate/utils/dataclasses.py#L425</source><parameters>[{"name": "opt_level", "val": ": typing.Literal['O1', 'O2'] = None"}, {"name": "use_autocast_during_eval", "val": ": typing.Optional[bool] = None"}, {"name": "margin", "val": ": typing.Optional[int] = None"}, {"name": "interval", "val": ": typing.Optional[int] = None"}, {"name": "fp8_format", "val": ": typing.Literal['HYBRID', 'E4M3', 'E5M2'] = None"}, {"name": "amax_history_len", "val": ": typing.Optional[int] = None"}, {"name": "amax_compute_algo", "val": ": typing.Literal['max', 'most_recent'] = None"}, {"name": "override_linear_precision", "val": ": tuple = None"}, {"name": "use_mxfp8_block_scaling", "val": ": typing.Optional[bool] = None"}, {"name": "backend", "val": ": typing.Literal['MSAMP', 'TE'] = None"}]</parameters></docstring>

Deprecated. Please use one of the proper FP8 recipe kwargs classes such as `TERecipeKwargs` or `MSAMPRecipeKwargs`
instead.


</div>

## convert_model[[accelerate.utils.convert_model]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>accelerate.utils.convert_model</name><anchor>accelerate.utils.convert_model</anchor><source>https://github.com/huggingface/accelerate/blob/main/src/accelerate/utils/transformer_engine.py#L26</source><parameters>[{"name": "model", "val": ""}, {"name": "to_transformer_engine", "val": " = True"}, {"name": "_convert_linear", "val": " = True"}, {"name": "_convert_ln", "val": " = True"}]</parameters></docstring>

Recursively converts the linear and layernorm layers of a model to their `transformers_engine` counterpart.


</div>

## has_transformer_engine_layers[[accelerate.utils.has_transformer_engine_layers]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>accelerate.utils.has_transformer_engine_layers</name><anchor>accelerate.utils.has_transformer_engine_layers</anchor><source>https://github.com/huggingface/accelerate/blob/main/src/accelerate/utils/transformer_engine.py#L95</source><parameters>[{"name": "model", "val": ""}]</parameters></docstring>

Returns whether a given model has some `transformer_engine` layer or not.


</div>

## contextual_fp8_autocast[[accelerate.utils.contextual_fp8_autocast]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>accelerate.utils.contextual_fp8_autocast</name><anchor>accelerate.utils.contextual_fp8_autocast</anchor><source>https://github.com/huggingface/accelerate/blob/main/src/accelerate/utils/transformer_engine.py#L118</source><parameters>[{"name": "model_forward", "val": ""}, {"name": "fp8_recipe", "val": ""}, {"name": "use_during_eval", "val": " = False"}]</parameters></docstring>

Wrapper for a model's forward method to apply FP8 autocast. Is context aware, meaning that by default it will
disable FP8 autocast during eval mode, which is generally better for more accurate metrics.


</div>

## apply_fp8_autowrap[[accelerate.utils.apply_fp8_autowrap]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>accelerate.utils.apply_fp8_autowrap</name><anchor>accelerate.utils.apply_fp8_autowrap</anchor><source>https://github.com/huggingface/accelerate/blob/main/src/accelerate/utils/transformer_engine.py#L142</source><parameters>[{"name": "model", "val": ""}, {"name": "fp8_recipe_handler", "val": ""}]</parameters></docstring>

Applies FP8 context manager to the model's forward method


</div>

<EditOnGithub source="https://github.com/huggingface/accelerate/blob/main/docs/source/package_reference/fp8.md" />