#
Functions for PEFT integration

A collection of functions that could be useful for non-PeftModel models, e.g. transformers or diffusers integration

The functions provided here can be considered “public API” of PEFT and hence are safe to be used by packages that provide PEFT integrations.

##
Cast the adapter weight dtypes

####
peft.tuners.tuners_utils.cast_adapter_dtype

< source >

( model: nn.Module adapter_name: str autocast_adapter_dtype: bool = True )

Parameters

- adapter_name (`str`) — The adapter name.
- autocast_adapter_dtype (`bool`, optional) — Whether to autocast the adapter dtype. Defaults to `True`.

A helper method to cast the adapter weights to the correct dtype.

Currently, this only upcasts float16 and bfloat16 to float32.

##
Delete the PEFT adapter from model

####
peft.tuners.tuners_utils.delete_adapter

< source >

( model: nn.Module adapter_name: str prefix: str layer_cls: type[BaseTunerLayer] = <class 'peft.tuners.tuners_utils.BaseTunerLayer'> ) →
new_adapter (list[str] | None)

Parameters

- model (`nn.Module`) — The model from which the adapter should be deleted.
- adapter_name (str) — The name of the adapter to be deleted.
- prefix (str) — The prefix of the PEFT method, e.g. “lora_” for LoRA.
- layer_cls (type, optional) — The class of the adapter layer. Defaults to `BaseTunerLayer`.

Returns

new_adapter (list[str] | None)

The name of remaining adapter(s) after deletion, or `None` if there are no active adapters left. Use this to set the new active adapter of the model if necessary.

Delete an existing PEFT adapter.

Note: This function does not delete the PEFT config on the model, if there is one. It will also not completely purge the PEFT layers if the last PEFT adapter is deleted. For this, consider using `model.unload()` if using a PEFT model instance, or just reloading the base model.

##
Get the state dict of the PEFT adapter

####
peft.get_peft_model_state_dict

< source >

( model state_dict = None adapter_name = 'default' unwrap_compiled = False save_embedding_layers = 'auto' )

Parameters

- model (PeftModel) — The Peft model. When using torch.nn.DistributedDataParallel, DeepSpeed or FSDP, the model should be the underlying model/unwrapped model (i.e. model.module).
- state_dict (`dict`, optional, defaults to `None`) — The state dict of the model. If not provided, the state dict of the passed model will be used.
- adapter_name (`str`, optional, defaults to `"default"`) — The name of the adapter whose state dict should be returned.
- unwrap_compiled (`bool`, optional, defaults to `False`) — Whether to unwrap the model if torch.compile was used.
- save_embedding_layers (`Union[bool, str]`, , optional, defaults to `auto`) — If `True`, save the embedding layers in addition to adapter weights. If `auto`, checks the common embedding layers `peft.utils.other.EMBEDDING_LAYER_NAMES` in config’s `target_modules` when available. Based on it sets the boolean flag. This only works for 🤗 transformers models.

Get the state dict of the given adapter of the PEFT model.

This only includes the PEFT parameters, not the parameters of the base model. Thus the returned `state_dict` is generally small compared to the full model size. To retrieve the full `state_dict`, just call `model.state_dict()`.

Note that the adapter name is removed from the `state_dict`, as this is just an arbitrary name that can be changed when loading the adapter. So e.g. if the adapter name is `'default'` and the original key is `'model.q_proj.lora_A.default.weight'`, the returned key will be `'model.q_proj.lora_A.weight'`. Use this function in conjunction with set_peft_model_state_dict() to take care of the adapter name when loading weights.

##
Inject a PEFT adapter into the model based on a PEFT config

####
peft.inject_adapter_in_model

< source >

( peft_config: PeftConfig model: torch.nn.Module adapter_name: str = 'default' low_cpu_mem_usage: bool = False state_dict: Optional[dict[str, torch.Tensor]] = None )

Parameters

- peft_config (`PeftConfig`) — Configuration object containing the parameters of the PEFT model.
- model (`torch.nn.Module`) — The input model where the adapter will be injected.
- adapter_name (`str`, `optional`, defaults to `"default"`) — The name of the adapter to be injected, if not provided, the default adapter name is used (“default”).
- low_cpu_mem_usage (`bool`, `optional`, defaults to `False`) — Create empty adapter weights on meta device. Useful to speed up the loading process.
- state_dict (`dict`, optional, defaults to `None`) — If a `state_dict` is passed here, the adapters will be injected based on the entries of the state_dict. This can be useful when the exact `target_modules` of the PEFT method is unknown, for instance because the checkpoint was created without meta data. Note that the values from the `state_dict` are not used, only the keys are used to determine the correct layers that should be adapted.

Create PEFT layers and inject them into the model in-place.

Currently the API does not support prompt learning methods and adaption prompt.

This function is similar to get_peft_model() but it does not return a PeftModel instance. Instead, it returns the original, mutated instance of the passed model.

##
Set the active PEFT adapter(s) of the model

####
peft.tuners.tuners_utils.set_adapter

< source >

( model adapter_name: str | list[str] inference_mode: bool = False layer_cls: type[BaseTunerLayer] = <class 'peft.tuners.tuners_utils.BaseTunerLayer'> )

Parameters

- model (`nn.Module`) — The model on which the adapter(s) should be set.
- adapter_name (str, list[str]) — The name(s) of the adapter(s) to set as active
- inference_mode (bool, optional) — Whether the activated adapter should be frozen (i.e. `requires_grad=False`). Default is False.
- layer_cls (type, optional) — The class of the adapter layer. Defaults to `BaseTunerLayer`.

Set the active PEFT adapter(s) of the model.

Active adapters are those adapters that participate in the forward pass. Use this function if you want to switch between multiple PEFT adapters.

##
Load the weights of the PEFT state dict into the model

####
peft.set_peft_model_state_dict

< source >

( model peft_model_state_dict adapter_name = 'default' ignore_mismatched_sizes: bool = False low_cpu_mem_usage: bool = False )

Parameters

- model (PeftModel) — The Peft model.
- peft_model_state_dict (`dict`) — The state dict of the Peft model.
- adapter_name (`str`, optional, defaults to `"default"`) — The name of the adapter whose state dict should be set.
- ignore_mismatched_sizes (`bool`, optional, defaults to `False`) — Whether to ignore mismatched in the state dict.
- low_cpu_mem_usage (`bool`, `optional`, defaults to `False`) — This argument must be `True` if the `model` was loaded with adapter weights on the meta device, e.g. after calling `inject_adapter_in_model` with `low_cpu_mem_usage=True`. Otherwise, leave it as `False`.

Set the state dict of the PEFT model.

Given a PEFT `state_dict` (as returned by get_peft_model_state_dict()), insert the weights into the model. The model needs to have the PEFT adapters already in place (e.g. via inject_adapter_in_model()).

Setting the adapter weights also takes care of re-inserting the adapter name. This name may be a different name than the one originally used to train the adapter.

< > Update on GitHub