# Consistency Decoder

Consistency decoder can be used to decode the latents from the denoising UNet in the [StableDiffusionPipeline](/docs/diffusers/main/en/api/pipelines/stable_diffusion/text2img#diffusers.StableDiffusionPipeline). This decoder was introduced in the [DALL-E 3 technical report](https://openai.com/dall-e-3).

The original codebase can be found at [openai/consistencydecoder](https://github.com/openai/consistencydecoder).

> [!WARNING]
> Inference is only supported for 2 iterations as of now.

The pipeline could not have been contributed without the help of [madebyollin](https://github.com/madebyollin) and [mrsteyk](https://github.com/mrsteyk) from [this issue](https://github.com/openai/consistencydecoder/issues/1).

## ConsistencyDecoderVAE[[diffusers.ConsistencyDecoderVAE]]
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class diffusers.ConsistencyDecoderVAE</name><anchor>diffusers.ConsistencyDecoderVAE</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L52</source><parameters>[{"name": "scaling_factor", "val": ": float = 0.18215"}, {"name": "latent_channels", "val": ": int = 4"}, {"name": "sample_size", "val": ": int = 32"}, {"name": "encoder_act_fn", "val": ": str = 'silu'"}, {"name": "encoder_block_out_channels", "val": ": typing.Tuple[int, ...] = (128, 256, 512, 512)"}, {"name": "encoder_double_z", "val": ": bool = True"}, {"name": "encoder_down_block_types", "val": ": typing.Tuple[str, ...] = ('DownEncoderBlock2D', 'DownEncoderBlock2D', 'DownEncoderBlock2D', 'DownEncoderBlock2D')"}, {"name": "encoder_in_channels", "val": ": int = 3"}, {"name": "encoder_layers_per_block", "val": ": int = 2"}, {"name": "encoder_norm_num_groups", "val": ": int = 32"}, {"name": "encoder_out_channels", "val": ": int = 4"}, {"name": "decoder_add_attention", "val": ": bool = False"}, {"name": "decoder_block_out_channels", "val": ": typing.Tuple[int, ...] = (320, 640, 1024, 1024)"}, {"name": "decoder_down_block_types", "val": ": typing.Tuple[str, ...] = ('ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D')"}, {"name": "decoder_downsample_padding", "val": ": int = 1"}, {"name": "decoder_in_channels", "val": ": int = 7"}, {"name": "decoder_layers_per_block", "val": ": int = 3"}, {"name": "decoder_norm_eps", "val": ": float = 1e-05"}, {"name": "decoder_norm_num_groups", "val": ": int = 32"}, {"name": "decoder_num_train_timesteps", "val": ": int = 1024"}, {"name": "decoder_out_channels", "val": ": int = 6"}, {"name": "decoder_resnet_time_scale_shift", "val": ": str = 'scale_shift'"}, {"name": "decoder_time_embedding_type", "val": ": str = 'learned'"}, {"name": "decoder_up_block_types", "val": ": typing.Tuple[str, ...] = ('ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D')"}]</parameters></docstring>

The consistency decoder used with DALL-E 3.

<ExampleCodeBlock anchor="diffusers.ConsistencyDecoderVAE.example">

Examples:
```py
>>> import torch
>>> from diffusers import StableDiffusionPipeline, ConsistencyDecoderVAE

>>> vae = ConsistencyDecoderVAE.from_pretrained("openai/consistency-decoder", torch_dtype=torch.float16)
>>> pipe = StableDiffusionPipeline.from_pretrained(
...     "stable-diffusion-v1-5/stable-diffusion-v1-5", vae=vae, torch_dtype=torch.float16
... ).to("cuda")

>>> image = pipe("horse", generator=torch.manual_seed(0)).images[0]
>>> image
```

</ExampleCodeBlock>



<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>wrapper</name><anchor>diffusers.ConsistencyDecoderVAE.decode</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/utils/accelerate_utils.py#L43</source><parameters>[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]</parameters></docstring>


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>disable_slicing</name><anchor>diffusers.ConsistencyDecoderVAE.disable_slicing</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L196</source><parameters>[]</parameters></docstring>

Disable sliced VAE decoding. If `enable_slicing` was previously enabled, this method will go back to computing
decoding in one step.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>disable_tiling</name><anchor>diffusers.ConsistencyDecoderVAE.disable_tiling</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L180</source><parameters>[]</parameters></docstring>

Disable tiled VAE decoding. If `enable_tiling` was previously enabled, this method will go back to computing
decoding in one step.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>enable_slicing</name><anchor>diffusers.ConsistencyDecoderVAE.enable_slicing</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L188</source><parameters>[]</parameters></docstring>

Enable sliced VAE decoding. When this option is enabled, the VAE will split the input tensor in slices to
compute decoding in several steps. This is useful to save some memory and allow larger batch sizes.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>enable_tiling</name><anchor>diffusers.ConsistencyDecoderVAE.enable_tiling</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L171</source><parameters>[{"name": "use_tiling", "val": ": bool = True"}]</parameters></docstring>

Enable tiled VAE decoding. When this option is enabled, the VAE will split the input tensor into tiles to
compute decoding and encoding in several steps. This is useful for saving a large amount of memory and to allow
processing larger images.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>forward</name><anchor>diffusers.ConsistencyDecoderVAE.forward</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L430</source><parameters>[{"name": "sample", "val": ": Tensor"}, {"name": "sample_posterior", "val": ": bool = False"}, {"name": "return_dict", "val": ": bool = True"}, {"name": "generator", "val": ": typing.Optional[torch._C.Generator] = None"}]</parameters><paramsdesc>- **sample** (`torch.Tensor`) -- Input sample.
- **sample_posterior** (`bool`, *optional*, defaults to `False`) --
  Whether to sample from the posterior.
- **return_dict** (`bool`, *optional*, defaults to `True`) --
  Whether or not to return a `DecoderOutput` instead of a plain tuple.
- **generator** (`torch.Generator`, *optional*, defaults to `None`) --
  Generator to use for sampling.</paramsdesc><paramgroups>0</paramgroups><rettype>`DecoderOutput` or `tuple`</rettype><retdesc>If return_dict is True, a `DecoderOutput` is returned, otherwise a plain `tuple` is returned.</retdesc></docstring>








</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>set_attn_processor</name><anchor>diffusers.ConsistencyDecoderVAE.set_attn_processor</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L229</source><parameters>[{"name": "processor", "val": ": typing.Union[diffusers.models.attention_processor.AttnProcessor, diffusers.models.attention_processor.CustomDiffusionAttnProcessor, diffusers.models.attention_processor.AttnAddedKVProcessor, diffusers.models.attention_processor.AttnAddedKVProcessor2_0, diffusers.models.attention_processor.JointAttnProcessor2_0, diffusers.models.attention_processor.PAGJointAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGJointAttnProcessor2_0, diffusers.models.attention_processor.FusedJointAttnProcessor2_0, diffusers.models.attention_processor.AllegroAttnProcessor2_0, diffusers.models.attention_processor.AuraFlowAttnProcessor2_0, diffusers.models.attention_processor.FusedAuraFlowAttnProcessor2_0, diffusers.models.attention_processor.FluxAttnProcessor2_0, diffusers.models.attention_processor.FluxAttnProcessor2_0_NPU, diffusers.models.attention_processor.FusedFluxAttnProcessor2_0, diffusers.models.attention_processor.FusedFluxAttnProcessor2_0_NPU, diffusers.models.attention_processor.CogVideoXAttnProcessor2_0, diffusers.models.attention_processor.FusedCogVideoXAttnProcessor2_0, diffusers.models.attention_processor.XFormersAttnAddedKVProcessor, diffusers.models.attention_processor.XFormersAttnProcessor, diffusers.models.attention_processor.XLAFlashAttnProcessor2_0, diffusers.models.attention_processor.AttnProcessorNPU, diffusers.models.attention_processor.AttnProcessor2_0, diffusers.models.attention_processor.MochiVaeAttnProcessor2_0, diffusers.models.attention_processor.MochiAttnProcessor2_0, diffusers.models.attention_processor.StableAudioAttnProcessor2_0, diffusers.models.attention_processor.HunyuanAttnProcessor2_0, diffusers.models.attention_processor.FusedHunyuanAttnProcessor2_0, diffusers.models.attention_processor.PAGHunyuanAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGHunyuanAttnProcessor2_0, diffusers.models.attention_processor.LuminaAttnProcessor2_0, diffusers.models.attention_processor.FusedAttnProcessor2_0, diffusers.models.attention_processor.CustomDiffusionXFormersAttnProcessor, diffusers.models.attention_processor.CustomDiffusionAttnProcessor2_0, diffusers.models.attention_processor.SlicedAttnProcessor, diffusers.models.attention_processor.SlicedAttnAddedKVProcessor, diffusers.models.attention_processor.SanaLinearAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGSanaLinearAttnProcessor2_0, diffusers.models.attention_processor.PAGIdentitySanaLinearAttnProcessor2_0, diffusers.models.attention_processor.SanaMultiscaleLinearAttention, diffusers.models.attention_processor.SanaMultiscaleAttnProcessor2_0, diffusers.models.attention_processor.SanaMultiscaleAttentionProjection, diffusers.models.attention_processor.IPAdapterAttnProcessor, diffusers.models.attention_processor.IPAdapterAttnProcessor2_0, diffusers.models.attention_processor.IPAdapterXFormersAttnProcessor, diffusers.models.attention_processor.SD3IPAdapterJointAttnProcessor2_0, diffusers.models.attention_processor.PAGIdentitySelfAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGIdentitySelfAttnProcessor2_0, diffusers.models.attention_processor.LoRAAttnProcessor, diffusers.models.attention_processor.LoRAAttnProcessor2_0, diffusers.models.attention_processor.LoRAXFormersAttnProcessor, diffusers.models.attention_processor.LoRAAttnAddedKVProcessor, typing.Dict[str, typing.Union[diffusers.models.attention_processor.AttnProcessor, diffusers.models.attention_processor.CustomDiffusionAttnProcessor, diffusers.models.attention_processor.AttnAddedKVProcessor, diffusers.models.attention_processor.AttnAddedKVProcessor2_0, diffusers.models.attention_processor.JointAttnProcessor2_0, diffusers.models.attention_processor.PAGJointAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGJointAttnProcessor2_0, diffusers.models.attention_processor.FusedJointAttnProcessor2_0, diffusers.models.attention_processor.AllegroAttnProcessor2_0, diffusers.models.attention_processor.AuraFlowAttnProcessor2_0, diffusers.models.attention_processor.FusedAuraFlowAttnProcessor2_0, diffusers.models.attention_processor.FluxAttnProcessor2_0, diffusers.models.attention_processor.FluxAttnProcessor2_0_NPU, diffusers.models.attention_processor.FusedFluxAttnProcessor2_0, diffusers.models.attention_processor.FusedFluxAttnProcessor2_0_NPU, diffusers.models.attention_processor.CogVideoXAttnProcessor2_0, diffusers.models.attention_processor.FusedCogVideoXAttnProcessor2_0, diffusers.models.attention_processor.XFormersAttnAddedKVProcessor, diffusers.models.attention_processor.XFormersAttnProcessor, diffusers.models.attention_processor.XLAFlashAttnProcessor2_0, diffusers.models.attention_processor.AttnProcessorNPU, diffusers.models.attention_processor.AttnProcessor2_0, diffusers.models.attention_processor.MochiVaeAttnProcessor2_0, diffusers.models.attention_processor.MochiAttnProcessor2_0, diffusers.models.attention_processor.StableAudioAttnProcessor2_0, diffusers.models.attention_processor.HunyuanAttnProcessor2_0, diffusers.models.attention_processor.FusedHunyuanAttnProcessor2_0, diffusers.models.attention_processor.PAGHunyuanAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGHunyuanAttnProcessor2_0, diffusers.models.attention_processor.LuminaAttnProcessor2_0, diffusers.models.attention_processor.FusedAttnProcessor2_0, diffusers.models.attention_processor.CustomDiffusionXFormersAttnProcessor, diffusers.models.attention_processor.CustomDiffusionAttnProcessor2_0, diffusers.models.attention_processor.SlicedAttnProcessor, diffusers.models.attention_processor.SlicedAttnAddedKVProcessor, diffusers.models.attention_processor.SanaLinearAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGSanaLinearAttnProcessor2_0, diffusers.models.attention_processor.PAGIdentitySanaLinearAttnProcessor2_0, diffusers.models.attention_processor.SanaMultiscaleLinearAttention, diffusers.models.attention_processor.SanaMultiscaleAttnProcessor2_0, diffusers.models.attention_processor.SanaMultiscaleAttentionProjection, diffusers.models.attention_processor.IPAdapterAttnProcessor, diffusers.models.attention_processor.IPAdapterAttnProcessor2_0, diffusers.models.attention_processor.IPAdapterXFormersAttnProcessor, diffusers.models.attention_processor.SD3IPAdapterJointAttnProcessor2_0, diffusers.models.attention_processor.PAGIdentitySelfAttnProcessor2_0, diffusers.models.attention_processor.PAGCFGIdentitySelfAttnProcessor2_0, diffusers.models.attention_processor.LoRAAttnProcessor, diffusers.models.attention_processor.LoRAAttnProcessor2_0, diffusers.models.attention_processor.LoRAXFormersAttnProcessor, diffusers.models.attention_processor.LoRAAttnAddedKVProcessor]]]"}]</parameters><paramsdesc>- **processor** (`dict` of `AttentionProcessor` or only `AttentionProcessor`) --
  The instantiated processor class or a dictionary of processor classes that will be set as the processor
  for **all** `Attention` layers.

  If `processor` is a dict, the key needs to define the path to the corresponding cross attention
  processor. This is strongly recommended when setting trainable attention processors.</paramsdesc><paramgroups>0</paramgroups></docstring>

Sets the attention processor to use to compute attention.




</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>set_default_attn_processor</name><anchor>diffusers.ConsistencyDecoderVAE.set_default_attn_processor</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L264</source><parameters>[]</parameters></docstring>

Disables custom attention processors and sets the default attention implementation.


</div>
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>tiled_encode</name><anchor>diffusers.ConsistencyDecoderVAE.tiled_encode</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L375</source><parameters>[{"name": "x", "val": ": Tensor"}, {"name": "return_dict", "val": ": bool = True"}]</parameters><paramsdesc>- **x** (`torch.Tensor`) -- Input batch of images.
- **return_dict** (`bool`, *optional*, defaults to `True`) --
  Whether or not to return a `ConsistencyDecoderVAEOutput`
  instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>`ConsistencyDecoderVAEOutput` or `tuple`</rettype><retdesc>If return_dict is True, a `ConsistencyDecoderVAEOutput`
is returned, otherwise a plain `tuple` is returned.</retdesc></docstring>
Encode a batch of images using a tiled encoder.

When this option is enabled, the VAE will split the input tensor into tiles to compute encoding in several
steps. This is useful to keep memory use constant regardless of image size. The end result of tiled encoding is
different from non-tiled encoding because each tile uses a different encoder. To avoid tiling artifacts, the
tiles overlap and are blended together to form a smooth output. You may still see tile-sized changes in the
output, but they should be much less noticeable.








</div></div>

<EditOnGithub source="https://github.com/huggingface/diffusers/blob/main/docs/source/en/api/models/consistency_decoder_vae.md" />