# Marigold Computer Vision

![marigold](https://marigoldmonodepth.github.io/images/teaser_collage_compressed.jpg)

Marigold was proposed in 
[Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://huggingface.co/papers/2312.02145), 
a CVPR 2024 Oral paper by 
[Bingxin Ke](http://www.kebingxin.com/), 
[Anton Obukhov](https://www.obukhov.ai/), 
[Shengyu Huang](https://shengyuh.github.io/), 
[Nando Metzger](https://nandometzger.github.io/), 
[Rodrigo Caye Daudt](https://rcdaudt.github.io/), and 
[Konrad Schindler](https://scholar.google.com/citations?user=FZuNgqIAAAAJ&hl=en).
The core idea is to **repurpose the generative prior of Text-to-Image Latent Diffusion Models (LDMs) for traditional 
computer vision tasks**.
This approach was explored by fine-tuning Stable Diffusion for **Monocular Depth Estimation**, as demonstrated in the 
teaser above.

Marigold was later extended in the follow-up paper, 
[Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis](https://huggingface.co/papers/2312.02145), 
authored by 
[Bingxin Ke](http://www.kebingxin.com/), 
[Kevin Qu](https://www.linkedin.com/in/kevin-qu-b3417621b/?locale=en_US), 
[Tianfu Wang](https://tianfwang.github.io/), 
[Nando Metzger](https://nandometzger.github.io/), 
[Shengyu Huang](https://shengyuh.github.io/), 
[Bo Li](https://www.linkedin.com/in/bobboli0202/), 
[Anton Obukhov](https://www.obukhov.ai/), and 
[Konrad Schindler](https://scholar.google.com/citations?user=FZuNgqIAAAAJ&hl=en).
This work expanded Marigold to support new modalities such as **Surface Normals** and **Intrinsic Image Decomposition** 
(IID), introduced a training protocol for **Latent Consistency Models** (LCM), and demonstrated **High-Resolution** (HR) 
processing capability.

> [!TIP]
> The early Marigold models (`v1-0` and earlier) were optimized for best results with at least 10 inference steps.
> LCM models were later developed to enable high-quality inference in just 1 to 4 steps.
> Marigold models `v1-1` and later use the DDIM scheduler to achieve optimal 
> results in as few as 1 to 4 steps.

## Available Pipelines

Each pipeline is tailored for a specific computer vision task, processing an input RGB image and generating a 
corresponding prediction.
Currently, the following computer vision tasks are implemented:

| Pipeline                                                                                                                                          | Recommended Model Checkpoints                                                                                                                                                                           |                              Spaces (Interactive Apps)                               | Predicted Modalities                                                                                                                                                               |
|---------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [MarigoldDepthPipeline](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_depth.py)           | [prs-eth/marigold-depth-v1-1](https://huggingface.co/prs-eth/marigold-depth-v1-1)                                                                                                                       |          [Depth Estimation](https://huggingface.co/spaces/prs-eth/marigold)          | [Depth](https://en.wikipedia.org/wiki/Depth_map), [Disparity](https://en.wikipedia.org/wiki/Binocular_disparity)                                                                   |
| [MarigoldNormalsPipeline](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_normals.py)       | [prs-eth/marigold-normals-v1-1](https://huggingface.co/prs-eth/marigold-normals-v1-1)                                                                                                                   | [Surface Normals Estimation](https://huggingface.co/spaces/prs-eth/marigold-normals) | [Surface normals](https://en.wikipedia.org/wiki/Normal_mapping)                                                                                                                    |
| [MarigoldIntrinsicsPipeline](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_intrinsics.py) | [prs-eth/marigold-iid-appearance-v1-1](https://huggingface.co/prs-eth/marigold-iid-appearance-v1-1),<br>[prs-eth/marigold-iid-lighting-v1-1](https://huggingface.co/prs-eth/marigold-iid-lighting-v1-1) | [Intrinsic Image Decomposition](https://huggingface.co/spaces/prs-eth/marigold-iid)  | [Albedo](https://en.wikipedia.org/wiki/Albedo), [Materials](https://www.n.aiq3d.com/wiki/roughnessmetalnessao-map), [Lighting](https://en.wikipedia.org/wiki/Diffuse_reflection)   |

## Available Checkpoints

All original checkpoints are available under the [PRS-ETH](https://huggingface.co/prs-eth/) organization on Hugging Face.
They are designed for use with diffusers pipelines and the [original codebase](https://github.com/prs-eth/marigold), which can also be used to train 
new model checkpoints.
The following is a summary of the recommended checkpoints, all of which produce reliable results with 1 to 4 steps. 

| Checkpoint                                                                                          | Modality     | Comment                                                                                                                                                                              |
|-----------------------------------------------------------------------------------------------------|--------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [prs-eth/marigold-depth-v1-1](https://huggingface.co/prs-eth/marigold-depth-v1-1)                   | Depth        | Affine-invariant depth prediction assigns each pixel a value between 0 (near plane) and 1 (far plane), with both planes determined by the model during inference.                    |
| [prs-eth/marigold-normals-v0-1](https://huggingface.co/prs-eth/marigold-normals-v0-1)               | Normals      | The surface normals predictions are unit-length 3D vectors in the screen space camera, with values in the range from -1 to 1.                                                        |
| [prs-eth/marigold-iid-appearance-v1-1](https://huggingface.co/prs-eth/marigold-iid-appearance-v1-1) | Intrinsics   | InteriorVerse decomposition is comprised of Albedo and two BRDF material properties: Roughness and Metallicity.                                                                      | 
| [prs-eth/marigold-iid-lighting-v1-1](https://huggingface.co/prs-eth/marigold-iid-lighting-v1-1)     | Intrinsics   | HyperSim decomposition of an image &nbsp\\(I\\)&nbsp is comprised of Albedo &nbsp\\(A\\), Diffuse shading &nbsp\\(S\\), and Non-diffuse residual &nbsp\\(R\\): &nbsp\\(I = A*S+R\\). |

> [!TIP]
> Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers) to learn how to explore the tradeoff 
> between scheduler speed and quality, and see the [reuse components across pipelines](../../using-diffusers/loading#reuse-a-pipeline) section to learn how to 
> efficiently load the same components into multiple pipelines. 
> Also, to know more about reducing the memory usage of this pipeline, refer to the ["Reduce memory usage"] section 
> [here](../../using-diffusers/svd#reduce-memory-usage).

> [!WARNING]
> Marigold pipelines were designed and tested with the scheduler embedded in the model checkpoint.
> The optimal number of inference steps varies by scheduler, with no universal value that works best across all cases.
> To accommodate this, the `num_inference_steps` parameter in the pipeline's `__call__` method defaults to `None` (see the 
> API reference).
> Unless set explicitly, it inherits the value from the `default_denoising_steps` field in the checkpoint configuration 
> file (`model_index.json`).
> This ensures high-quality predictions when invoking the pipeline with only the `image` argument.

See also Marigold [usage examples](../../using-diffusers/marigold_usage).

## Marigold Depth Prediction API[[diffusers.MarigoldDepthPipeline]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class diffusers.MarigoldDepthPipeline</name><anchor>diffusers.MarigoldDepthPipeline</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_depth.py#L105</source><parameters>[{"name": "unet", "val": ": UNet2DConditionModel"}, {"name": "vae", "val": ": AutoencoderKL"}, {"name": "scheduler", "val": ": typing.Union[diffusers.schedulers.scheduling_ddim.DDIMScheduler, diffusers.schedulers.scheduling_lcm.LCMScheduler]"}, {"name": "text_encoder", "val": ": CLIPTextModel"}, {"name": "tokenizer", "val": ": CLIPTokenizer"}, {"name": "prediction_type", "val": ": typing.Optional[str] = None"}, {"name": "scale_invariant", "val": ": typing.Optional[bool] = True"}, {"name": "shift_invariant", "val": ": typing.Optional[bool] = True"}, {"name": "default_denoising_steps", "val": ": typing.Optional[int] = None"}, {"name": "default_processing_resolution", "val": ": typing.Optional[int] = None"}]</parameters><paramsdesc>- **unet** (`UNet2DConditionModel`) --
  Conditional U-Net to denoise the depth latent, conditioned on image latent.
- **vae** (`AutoencoderKL`) --
  Variational Auto-Encoder (VAE) Model to encode and decode images and predictions to and from latent
  representations.
- **scheduler** (`DDIMScheduler` or `LCMScheduler`) --
  A scheduler to be used in combination with `unet` to denoise the encoded image latents.
- **text_encoder** (`CLIPTextModel`) --
  Text-encoder, for empty text embedding.
- **tokenizer** (`CLIPTokenizer`) --
  CLIP tokenizer.
- **prediction_type** (`str`, *optional*) --
  Type of predictions made by the model.
- **scale_invariant** (`bool`, *optional*) --
  A model property specifying whether the predicted depth maps are scale-invariant. This value must be set in
  the model config. When used together with the `shift_invariant=True` flag, the model is also called
  "affine-invariant". NB: overriding this value is not supported.
- **shift_invariant** (`bool`, *optional*) --
  A model property specifying whether the predicted depth maps are shift-invariant. This value must be set in
  the model config. When used together with the `scale_invariant=True` flag, the model is also called
  "affine-invariant". NB: overriding this value is not supported.
- **default_denoising_steps** (`int`, *optional*) --
  The minimum number of denoising diffusion steps that are required to produce a prediction of reasonable
  quality with the given model. This value must be set in the model config. When the pipeline is called
  without explicitly setting `num_inference_steps`, the default value is used. This is required to ensure
  reasonable results with various model flavors compatible with the pipeline, such as those relying on very
  short denoising schedules (`LCMScheduler`) and those with full diffusion schedules (`DDIMScheduler`).
- **default_processing_resolution** (`int`, *optional*) --
  The recommended value of the `processing_resolution` parameter of the pipeline. This value must be set in
  the model config. When the pipeline is called without explicitly setting `processing_resolution`, the
  default value is used. This is required to ensure reasonable results with various model flavors trained
  with varying optimal processing resolution values.</paramsdesc><paramgroups>0</paramgroups></docstring>

Pipeline for monocular depth estimation using the Marigold method: https://marigoldmonodepth.github.io.

This model inherits from [DiffusionPipeline](/docs/diffusers/main/en/api/pipelines/overview#diffusers.DiffusionPipeline). Check the superclass documentation for the generic methods the
library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.)





<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>__call__</name><anchor>diffusers.MarigoldDepthPipeline.__call__</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_depth.py#L348</source><parameters>[{"name": "image", "val": ": typing.Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, typing.List[PIL.Image.Image], typing.List[numpy.ndarray], typing.List[torch.Tensor]]"}, {"name": "num_inference_steps", "val": ": typing.Optional[int] = None"}, {"name": "ensemble_size", "val": ": int = 1"}, {"name": "processing_resolution", "val": ": typing.Optional[int] = None"}, {"name": "match_input_resolution", "val": ": bool = True"}, {"name": "resample_method_input", "val": ": str = 'bilinear'"}, {"name": "resample_method_output", "val": ": str = 'bilinear'"}, {"name": "batch_size", "val": ": int = 1"}, {"name": "ensembling_kwargs", "val": ": typing.Optional[typing.Dict[str, typing.Any]] = None"}, {"name": "latents", "val": ": typing.Union[torch.Tensor, typing.List[torch.Tensor], NoneType] = None"}, {"name": "generator", "val": ": typing.Union[torch._C.Generator, typing.List[torch._C.Generator], NoneType] = None"}, {"name": "output_type", "val": ": str = 'np'"}, {"name": "output_uncertainty", "val": ": bool = False"}, {"name": "output_latent", "val": ": bool = False"}, {"name": "return_dict", "val": ": bool = True"}]</parameters><paramsdesc>- **image** (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `List[PIL.Image.Image]`, `List[np.ndarray]`), --
  `List[torch.Tensor]`: An input image or images used as an input for the depth estimation task. For
  arrays and tensors, the expected value range is between `[0, 1]`. Passing a batch of images is possible
  by providing a four-dimensional array or a tensor. Additionally, a list of images of two- or
  three-dimensional arrays or tensors can be passed. In the latter case, all list elements must have the
  same width and height.
- **num_inference_steps** (`int`, *optional*, defaults to `None`) --
  Number of denoising diffusion steps during inference. The default value `None` results in automatic
  selection.
- **ensemble_size** (`int`, defaults to `1`) --
  Number of ensemble predictions. Higher values result in measurable improvements and visual degradation.
- **processing_resolution** (`int`, *optional*, defaults to `None`) --
  Effective processing resolution. When set to `0`, matches the larger input image dimension. This
  produces crisper predictions, but may also lead to the overall loss of global context. The default
  value `None` resolves to the optimal value from the model config.
- **match_input_resolution** (`bool`, *optional*, defaults to `True`) --
  When enabled, the output prediction is resized to match the input dimensions. When disabled, the longer
  side of the output will equal to `processing_resolution`.
- **resample_method_input** (`str`, *optional*, defaults to `"bilinear"`) --
  Resampling method used to resize input images to `processing_resolution`. The accepted values are:
  `"nearest"`, `"nearest-exact"`, `"bilinear"`, `"bicubic"`, or `"area"`.
- **resample_method_output** (`str`, *optional*, defaults to `"bilinear"`) --
  Resampling method used to resize output predictions to match the input resolution. The accepted values
  are `"nearest"`, `"nearest-exact"`, `"bilinear"`, `"bicubic"`, or `"area"`.
- **batch_size** (`int`, *optional*, defaults to `1`) --
  Batch size; only matters when setting `ensemble_size` or passing a tensor of images.
- **ensembling_kwargs** (`dict`, *optional*, defaults to `None`) --
  Extra dictionary with arguments for precise ensembling control. The following options are available:
  - reduction (`str`, *optional*, defaults to `"median"`): Defines the ensembling function applied in
    every pixel location, can be either `"median"` or `"mean"`.
  - regularizer_strength (`float`, *optional*, defaults to `0.02`): Strength of the regularizer that
    pulls the aligned predictions to the unit range from 0 to 1.
  - max_iter (`int`, *optional*, defaults to `2`): Maximum number of the alignment solver steps. Refer to
    `scipy.optimize.minimize` function, `options` argument.
  - tol (`float`, *optional*, defaults to `1e-3`): Alignment solver tolerance. The solver stops when the
    tolerance is reached.
  - max_res (`int`, *optional*, defaults to `None`): Resolution at which the alignment is performed;
    `None` matches the `processing_resolution`.
- **latents** (`torch.Tensor`, or `List[torch.Tensor]`, *optional*, defaults to `None`) --
  Latent noise tensors to replace the random initialization. These can be taken from the previous
  function call's output.
- **generator** (`torch.Generator`, or `List[torch.Generator]`, *optional*, defaults to `None`) --
  Random number generator object to ensure reproducibility.
- **output_type** (`str`, *optional*, defaults to `"np"`) --
  Preferred format of the output's `prediction` and the optional `uncertainty` fields. The accepted
  values are: `"np"` (numpy array) or `"pt"` (torch tensor).
- **output_uncertainty** (`bool`, *optional*, defaults to `False`) --
  When enabled, the output's `uncertainty` field contains the predictive uncertainty map, provided that
  the `ensemble_size` argument is set to a value above 2.
- **output_latent** (`bool`, *optional*, defaults to `False`) --
  When enabled, the output's `latent` field contains the latent codes corresponding to the predictions
  within the ensemble. These codes can be saved, modified, and used for subsequent calls with the
  `latents` argument.
- **return_dict** (`bool`, *optional*, defaults to `True`) --
  Whether or not to return a [MarigoldDepthOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldDepthOutput) instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>[MarigoldDepthOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldDepthOutput) or `tuple`</rettype><retdesc>If `return_dict` is `True`, [MarigoldDepthOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldDepthOutput) is returned, otherwise a
`tuple` is returned where the first element is the prediction, the second element is the uncertainty
(or `None`), and the third is the latent (or `None`).</retdesc></docstring>

Function invoked when calling the pipeline.



<ExampleCodeBlock anchor="diffusers.MarigoldDepthPipeline.__call__.example">

Examples:
```py
>>> import diffusers
>>> import torch

>>> pipe = diffusers.MarigoldDepthPipeline.from_pretrained(
...     "prs-eth/marigold-depth-v1-1", variant="fp16", torch_dtype=torch.float16
... ).to("cuda")

>>> image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg")
>>> depth = pipe(image)

>>> vis = pipe.image_processor.visualize_depth(depth.prediction)
>>> vis[0].save("einstein_depth.png")

>>> depth_16bit = pipe.image_processor.export_depth_to_16bit_png(depth.prediction)
>>> depth_16bit[0].save("einstein_depth_16bit.png")
```

</ExampleCodeBlock>







</div></div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class diffusers.pipelines.marigold.MarigoldDepthOutput</name><anchor>diffusers.pipelines.marigold.MarigoldDepthOutput</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_depth.py#L83</source><parameters>[{"name": "prediction", "val": ": typing.Union[numpy.ndarray, torch.Tensor]"}, {"name": "uncertainty", "val": ": typing.Union[NoneType, numpy.ndarray, torch.Tensor]"}, {"name": "latent", "val": ": typing.Optional[torch.Tensor]"}]</parameters><paramsdesc>- **prediction** (`np.ndarray`, `torch.Tensor`) --
  Predicted depth maps with values in the range [0, 1]. The shape is $numimages 	imes 1 	imes height 	imes
  width$ for `torch.Tensor` or $numimages 	imes height 	imes width 	imes 1$ for `np.ndarray`.
- **uncertainty** (`None`, `np.ndarray`, `torch.Tensor`) --
  Uncertainty maps computed from the ensemble, with values in the range [0, 1]. The shape is $numimages
  imes 1 	imes height 	imes width$ for `torch.Tensor` or $numimages 	imes height 	imes width 	imes 1$
  for `np.ndarray`.
- **latent** (`None`, `torch.Tensor`) --
  Latent features corresponding to the predictions, compatible with the `latents` argument of the pipeline.
  The shape is $numimages * numensemble 	imes 4 	imes latentheight 	imes latentwidth$.</paramsdesc><paramgroups>0</paramgroups></docstring>

Output class for Marigold monocular depth prediction pipeline.




</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>diffusers.pipelines.marigold.MarigoldImageProcessor.visualize_depth</name><anchor>diffusers.pipelines.marigold.MarigoldImageProcessor.visualize_depth</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/marigold_image_processing.py#L387</source><parameters>[{"name": "depth", "val": ": typing.Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, typing.List[PIL.Image.Image], typing.List[numpy.ndarray], typing.List[torch.Tensor]]"}, {"name": "val_min", "val": ": float = 0.0"}, {"name": "val_max", "val": ": float = 1.0"}, {"name": "color_map", "val": ": str = 'Spectral'"}]</parameters><paramsdesc>- **depth** (`Union[PIL.Image.Image, np.ndarray, torch.Tensor, List[PIL.Image.Image], List[np.ndarray], --
  List[torch.Tensor]]`): Depth maps.
- **val_min** (`float`, *optional*, defaults to `0.0`) -- Minimum value of the visualized depth range.
- **val_max** (`float`, *optional*, defaults to `1.0`) -- Maximum value of the visualized depth range.
- **color_map** (`str`, *optional*, defaults to `"Spectral"`) -- Color map used to convert a single-channel
  depth prediction into colored representation.</paramsdesc><paramgroups>0</paramgroups></docstring>

Visualizes depth maps, such as predictions of the `MarigoldDepthPipeline`.



Returns: `List[PIL.Image.Image]` with depth maps visualization.


</div>

## Marigold Normals Estimation API[[diffusers.MarigoldNormalsPipeline]]
<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class diffusers.MarigoldNormalsPipeline</name><anchor>diffusers.MarigoldNormalsPipeline</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_normals.py#L100</source><parameters>[{"name": "unet", "val": ": UNet2DConditionModel"}, {"name": "vae", "val": ": AutoencoderKL"}, {"name": "scheduler", "val": ": typing.Union[diffusers.schedulers.scheduling_ddim.DDIMScheduler, diffusers.schedulers.scheduling_lcm.LCMScheduler]"}, {"name": "text_encoder", "val": ": CLIPTextModel"}, {"name": "tokenizer", "val": ": CLIPTokenizer"}, {"name": "prediction_type", "val": ": typing.Optional[str] = None"}, {"name": "use_full_z_range", "val": ": typing.Optional[bool] = True"}, {"name": "default_denoising_steps", "val": ": typing.Optional[int] = None"}, {"name": "default_processing_resolution", "val": ": typing.Optional[int] = None"}]</parameters><paramsdesc>- **unet** (`UNet2DConditionModel`) --
  Conditional U-Net to denoise the normals latent, conditioned on image latent.
- **vae** (`AutoencoderKL`) --
  Variational Auto-Encoder (VAE) Model to encode and decode images and predictions to and from latent
  representations.
- **scheduler** (`DDIMScheduler` or `LCMScheduler`) --
  A scheduler to be used in combination with `unet` to denoise the encoded image latents.
- **text_encoder** (`CLIPTextModel`) --
  Text-encoder, for empty text embedding.
- **tokenizer** (`CLIPTokenizer`) --
  CLIP tokenizer.
- **prediction_type** (`str`, *optional*) --
  Type of predictions made by the model.
- **use_full_z_range** (`bool`, *optional*) --
  Whether the normals predicted by this model utilize the full range of the Z dimension, or only its positive
  half.
- **default_denoising_steps** (`int`, *optional*) --
  The minimum number of denoising diffusion steps that are required to produce a prediction of reasonable
  quality with the given model. This value must be set in the model config. When the pipeline is called
  without explicitly setting `num_inference_steps`, the default value is used. This is required to ensure
  reasonable results with various model flavors compatible with the pipeline, such as those relying on very
  short denoising schedules (`LCMScheduler`) and those with full diffusion schedules (`DDIMScheduler`).
- **default_processing_resolution** (`int`, *optional*) --
  The recommended value of the `processing_resolution` parameter of the pipeline. This value must be set in
  the model config. When the pipeline is called without explicitly setting `processing_resolution`, the
  default value is used. This is required to ensure reasonable results with various model flavors trained
  with varying optimal processing resolution values.</paramsdesc><paramgroups>0</paramgroups></docstring>

Pipeline for monocular normals estimation using the Marigold method: https://marigoldmonodepth.github.io.

This model inherits from [DiffusionPipeline](/docs/diffusers/main/en/api/pipelines/overview#diffusers.DiffusionPipeline). Check the superclass documentation for the generic methods the
library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.)





<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>__call__</name><anchor>diffusers.MarigoldNormalsPipeline.__call__</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_normals.py#L333</source><parameters>[{"name": "image", "val": ": typing.Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, typing.List[PIL.Image.Image], typing.List[numpy.ndarray], typing.List[torch.Tensor]]"}, {"name": "num_inference_steps", "val": ": typing.Optional[int] = None"}, {"name": "ensemble_size", "val": ": int = 1"}, {"name": "processing_resolution", "val": ": typing.Optional[int] = None"}, {"name": "match_input_resolution", "val": ": bool = True"}, {"name": "resample_method_input", "val": ": str = 'bilinear'"}, {"name": "resample_method_output", "val": ": str = 'bilinear'"}, {"name": "batch_size", "val": ": int = 1"}, {"name": "ensembling_kwargs", "val": ": typing.Optional[typing.Dict[str, typing.Any]] = None"}, {"name": "latents", "val": ": typing.Union[torch.Tensor, typing.List[torch.Tensor], NoneType] = None"}, {"name": "generator", "val": ": typing.Union[torch._C.Generator, typing.List[torch._C.Generator], NoneType] = None"}, {"name": "output_type", "val": ": str = 'np'"}, {"name": "output_uncertainty", "val": ": bool = False"}, {"name": "output_latent", "val": ": bool = False"}, {"name": "return_dict", "val": ": bool = True"}]</parameters><paramsdesc>- **image** (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `List[PIL.Image.Image]`, `List[np.ndarray]`), --
  `List[torch.Tensor]`: An input image or images used as an input for the normals estimation task. For
  arrays and tensors, the expected value range is between `[0, 1]`. Passing a batch of images is possible
  by providing a four-dimensional array or a tensor. Additionally, a list of images of two- or
  three-dimensional arrays or tensors can be passed. In the latter case, all list elements must have the
  same width and height.
- **num_inference_steps** (`int`, *optional*, defaults to `None`) --
  Number of denoising diffusion steps during inference. The default value `None` results in automatic
  selection.
- **ensemble_size** (`int`, defaults to `1`) --
  Number of ensemble predictions. Higher values result in measurable improvements and visual degradation.
- **processing_resolution** (`int`, *optional*, defaults to `None`) --
  Effective processing resolution. When set to `0`, matches the larger input image dimension. This
  produces crisper predictions, but may also lead to the overall loss of global context. The default
  value `None` resolves to the optimal value from the model config.
- **match_input_resolution** (`bool`, *optional*, defaults to `True`) --
  When enabled, the output prediction is resized to match the input dimensions. When disabled, the longer
  side of the output will equal to `processing_resolution`.
- **resample_method_input** (`str`, *optional*, defaults to `"bilinear"`) --
  Resampling method used to resize input images to `processing_resolution`. The accepted values are:
  `"nearest"`, `"nearest-exact"`, `"bilinear"`, `"bicubic"`, or `"area"`.
- **resample_method_output** (`str`, *optional*, defaults to `"bilinear"`) --
  Resampling method used to resize output predictions to match the input resolution. The accepted values
  are `"nearest"`, `"nearest-exact"`, `"bilinear"`, `"bicubic"`, or `"area"`.
- **batch_size** (`int`, *optional*, defaults to `1`) --
  Batch size; only matters when setting `ensemble_size` or passing a tensor of images.
- **ensembling_kwargs** (`dict`, *optional*, defaults to `None`) --
  Extra dictionary with arguments for precise ensembling control. The following options are available:
  - reduction (`str`, *optional*, defaults to `"closest"`): Defines the ensembling function applied in
    every pixel location, can be either `"closest"` or `"mean"`.
- **latents** (`torch.Tensor`, *optional*, defaults to `None`) --
  Latent noise tensors to replace the random initialization. These can be taken from the previous
  function call's output.
- **generator** (`torch.Generator`, or `List[torch.Generator]`, *optional*, defaults to `None`) --
  Random number generator object to ensure reproducibility.
- **output_type** (`str`, *optional*, defaults to `"np"`) --
  Preferred format of the output's `prediction` and the optional `uncertainty` fields. The accepted
  values are: `"np"` (numpy array) or `"pt"` (torch tensor).
- **output_uncertainty** (`bool`, *optional*, defaults to `False`) --
  When enabled, the output's `uncertainty` field contains the predictive uncertainty map, provided that
  the `ensemble_size` argument is set to a value above 2.
- **output_latent** (`bool`, *optional*, defaults to `False`) --
  When enabled, the output's `latent` field contains the latent codes corresponding to the predictions
  within the ensemble. These codes can be saved, modified, and used for subsequent calls with the
  `latents` argument.
- **return_dict** (`bool`, *optional*, defaults to `True`) --
  Whether or not to return a [MarigoldNormalsOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldNormalsOutput) instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>[MarigoldNormalsOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldNormalsOutput) or `tuple`</rettype><retdesc>If `return_dict` is `True`, [MarigoldNormalsOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldNormalsOutput) is returned, otherwise a
`tuple` is returned where the first element is the prediction, the second element is the uncertainty
(or `None`), and the third is the latent (or `None`).</retdesc></docstring>

Function invoked when calling the pipeline.



<ExampleCodeBlock anchor="diffusers.MarigoldNormalsPipeline.__call__.example">

Examples:
```py
>>> import diffusers
>>> import torch

>>> pipe = diffusers.MarigoldNormalsPipeline.from_pretrained(
...     "prs-eth/marigold-normals-v1-1", variant="fp16", torch_dtype=torch.float16
... ).to("cuda")

>>> image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg")
>>> normals = pipe(image)

>>> vis = pipe.image_processor.visualize_normals(normals.prediction)
>>> vis[0].save("einstein_normals.png")
```

</ExampleCodeBlock>







</div></div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class diffusers.pipelines.marigold.MarigoldNormalsOutput</name><anchor>diffusers.pipelines.marigold.MarigoldNormalsOutput</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_normals.py#L78</source><parameters>[{"name": "prediction", "val": ": typing.Union[numpy.ndarray, torch.Tensor]"}, {"name": "uncertainty", "val": ": typing.Union[NoneType, numpy.ndarray, torch.Tensor]"}, {"name": "latent", "val": ": typing.Optional[torch.Tensor]"}]</parameters><paramsdesc>- **prediction** (`np.ndarray`, `torch.Tensor`) --
  Predicted normals with values in the range [-1, 1]. The shape is $numimages 	imes 3 	imes height 	imes
  width$ for `torch.Tensor` or $numimages 	imes height 	imes width 	imes 3$ for `np.ndarray`.
- **uncertainty** (`None`, `np.ndarray`, `torch.Tensor`) --
  Uncertainty maps computed from the ensemble, with values in the range [0, 1]. The shape is $numimages
  imes 1 	imes height 	imes width$ for `torch.Tensor` or $numimages 	imes height 	imes width 	imes 1$
  for `np.ndarray`.
- **latent** (`None`, `torch.Tensor`) --
  Latent features corresponding to the predictions, compatible with the `latents` argument of the pipeline.
  The shape is $numimages * numensemble 	imes 4 	imes latentheight 	imes latentwidth$.</paramsdesc><paramgroups>0</paramgroups></docstring>

Output class for Marigold monocular normals prediction pipeline.




</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>diffusers.pipelines.marigold.MarigoldImageProcessor.visualize_normals</name><anchor>diffusers.pipelines.marigold.MarigoldImageProcessor.visualize_normals</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/marigold_image_processing.py#L488</source><parameters>[{"name": "normals", "val": ": typing.Union[numpy.ndarray, torch.Tensor, typing.List[numpy.ndarray], typing.List[torch.Tensor]]"}, {"name": "flip_x", "val": ": bool = False"}, {"name": "flip_y", "val": ": bool = False"}, {"name": "flip_z", "val": ": bool = False"}]</parameters><paramsdesc>- **normals** (`Union[np.ndarray, torch.Tensor, List[np.ndarray], List[torch.Tensor]]`) --
  Surface normals.
- **flip_x** (`bool`, *optional*, defaults to `False`) -- Flips the X axis of the normals frame of reference.
  Default direction is right.
- **flip_y** (`bool`, *optional*, defaults to `False`) --  Flips the Y axis of the normals frame of reference.
  Default direction is top.
- **flip_z** (`bool`, *optional*, defaults to `False`) -- Flips the Z axis of the normals frame of reference.
  Default direction is facing the observer.</paramsdesc><paramgroups>0</paramgroups></docstring>

Visualizes surface normals, such as predictions of the `MarigoldNormalsPipeline`.



Returns: `List[PIL.Image.Image]` with surface normals visualization.


</div>

## Marigold Intrinsic Image Decomposition API[[diffusers.MarigoldIntrinsicsPipeline]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class diffusers.MarigoldIntrinsicsPipeline</name><anchor>diffusers.MarigoldIntrinsicsPipeline</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_intrinsics.py#L120</source><parameters>[{"name": "unet", "val": ": UNet2DConditionModel"}, {"name": "vae", "val": ": AutoencoderKL"}, {"name": "scheduler", "val": ": typing.Union[diffusers.schedulers.scheduling_ddim.DDIMScheduler, diffusers.schedulers.scheduling_lcm.LCMScheduler]"}, {"name": "text_encoder", "val": ": CLIPTextModel"}, {"name": "tokenizer", "val": ": CLIPTokenizer"}, {"name": "prediction_type", "val": ": typing.Optional[str] = None"}, {"name": "target_properties", "val": ": typing.Optional[typing.Dict[str, typing.Any]] = None"}, {"name": "default_denoising_steps", "val": ": typing.Optional[int] = None"}, {"name": "default_processing_resolution", "val": ": typing.Optional[int] = None"}]</parameters><paramsdesc>- **unet** (`UNet2DConditionModel`) --
  Conditional U-Net to denoise the targets latent, conditioned on image latent.
- **vae** (`AutoencoderKL`) --
  Variational Auto-Encoder (VAE) Model to encode and decode images and predictions to and from latent
  representations.
- **scheduler** (`DDIMScheduler` or `LCMScheduler`) --
  A scheduler to be used in combination with `unet` to denoise the encoded image latents.
- **text_encoder** (`CLIPTextModel`) --
  Text-encoder, for empty text embedding.
- **tokenizer** (`CLIPTokenizer`) --
  CLIP tokenizer.
- **prediction_type** (`str`, *optional*) --
  Type of predictions made by the model.
- **target_properties** (`Dict[str, Any]`, *optional*) --
  Properties of the predicted modalities, such as `target_names`, a `List[str]` used to define the number,
  order and names of the predicted modalities, and any other metadata that may be required to interpret the
  predictions.
- **default_denoising_steps** (`int`, *optional*) --
  The minimum number of denoising diffusion steps that are required to produce a prediction of reasonable
  quality with the given model. This value must be set in the model config. When the pipeline is called
  without explicitly setting `num_inference_steps`, the default value is used. This is required to ensure
  reasonable results with various model flavors compatible with the pipeline, such as those relying on very
  short denoising schedules (`LCMScheduler`) and those with full diffusion schedules (`DDIMScheduler`).
- **default_processing_resolution** (`int`, *optional*) --
  The recommended value of the `processing_resolution` parameter of the pipeline. This value must be set in
  the model config. When the pipeline is called without explicitly setting `processing_resolution`, the
  default value is used. This is required to ensure reasonable results with various model flavors trained
  with varying optimal processing resolution values.</paramsdesc><paramgroups>0</paramgroups></docstring>

Pipeline for Intrinsic Image Decomposition (IID) using the Marigold method:
https://marigoldcomputervision.github.io.

This model inherits from [DiffusionPipeline](/docs/diffusers/main/en/api/pipelines/overview#diffusers.DiffusionPipeline). Check the superclass documentation for the generic methods the
library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.)





<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>__call__</name><anchor>diffusers.MarigoldIntrinsicsPipeline.__call__</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_intrinsics.py#L359</source><parameters>[{"name": "image", "val": ": typing.Union[PIL.Image.Image, numpy.ndarray, torch.Tensor, typing.List[PIL.Image.Image], typing.List[numpy.ndarray], typing.List[torch.Tensor]]"}, {"name": "num_inference_steps", "val": ": typing.Optional[int] = None"}, {"name": "ensemble_size", "val": ": int = 1"}, {"name": "processing_resolution", "val": ": typing.Optional[int] = None"}, {"name": "match_input_resolution", "val": ": bool = True"}, {"name": "resample_method_input", "val": ": str = 'bilinear'"}, {"name": "resample_method_output", "val": ": str = 'bilinear'"}, {"name": "batch_size", "val": ": int = 1"}, {"name": "ensembling_kwargs", "val": ": typing.Optional[typing.Dict[str, typing.Any]] = None"}, {"name": "latents", "val": ": typing.Union[torch.Tensor, typing.List[torch.Tensor], NoneType] = None"}, {"name": "generator", "val": ": typing.Union[torch._C.Generator, typing.List[torch._C.Generator], NoneType] = None"}, {"name": "output_type", "val": ": str = 'np'"}, {"name": "output_uncertainty", "val": ": bool = False"}, {"name": "output_latent", "val": ": bool = False"}, {"name": "return_dict", "val": ": bool = True"}]</parameters><paramsdesc>- **image** (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `List[PIL.Image.Image]`, `List[np.ndarray]`), --
  `List[torch.Tensor]`: An input image or images used as an input for the intrinsic decomposition task.
  For arrays and tensors, the expected value range is between `[0, 1]`. Passing a batch of images is
  possible by providing a four-dimensional array or a tensor. Additionally, a list of images of two- or
  three-dimensional arrays or tensors can be passed. In the latter case, all list elements must have the
  same width and height.
- **num_inference_steps** (`int`, *optional*, defaults to `None`) --
  Number of denoising diffusion steps during inference. The default value `None` results in automatic
  selection.
- **ensemble_size** (`int`, defaults to `1`) --
  Number of ensemble predictions. Higher values result in measurable improvements and visual degradation.
- **processing_resolution** (`int`, *optional*, defaults to `None`) --
  Effective processing resolution. When set to `0`, matches the larger input image dimension. This
  produces crisper predictions, but may also lead to the overall loss of global context. The default
  value `None` resolves to the optimal value from the model config.
- **match_input_resolution** (`bool`, *optional*, defaults to `True`) --
  When enabled, the output prediction is resized to match the input dimensions. When disabled, the longer
  side of the output will equal to `processing_resolution`.
- **resample_method_input** (`str`, *optional*, defaults to `"bilinear"`) --
  Resampling method used to resize input images to `processing_resolution`. The accepted values are:
  `"nearest"`, `"nearest-exact"`, `"bilinear"`, `"bicubic"`, or `"area"`.
- **resample_method_output** (`str`, *optional*, defaults to `"bilinear"`) --
  Resampling method used to resize output predictions to match the input resolution. The accepted values
  are `"nearest"`, `"nearest-exact"`, `"bilinear"`, `"bicubic"`, or `"area"`.
- **batch_size** (`int`, *optional*, defaults to `1`) --
  Batch size; only matters when setting `ensemble_size` or passing a tensor of images.
- **ensembling_kwargs** (`dict`, *optional*, defaults to `None`) --
  Extra dictionary with arguments for precise ensembling control. The following options are available:
  - reduction (`str`, *optional*, defaults to `"median"`): Defines the ensembling function applied in
    every pixel location, can be either `"median"` or `"mean"`.
- **latents** (`torch.Tensor`, *optional*, defaults to `None`) --
  Latent noise tensors to replace the random initialization. These can be taken from the previous
  function call's output.
- **generator** (`torch.Generator`, or `List[torch.Generator]`, *optional*, defaults to `None`) --
  Random number generator object to ensure reproducibility.
- **output_type** (`str`, *optional*, defaults to `"np"`) --
  Preferred format of the output's `prediction` and the optional `uncertainty` fields. The accepted
  values are: `"np"` (numpy array) or `"pt"` (torch tensor).
- **output_uncertainty** (`bool`, *optional*, defaults to `False`) --
  When enabled, the output's `uncertainty` field contains the predictive uncertainty map, provided that
  the `ensemble_size` argument is set to a value above 2.
- **output_latent** (`bool`, *optional*, defaults to `False`) --
  When enabled, the output's `latent` field contains the latent codes corresponding to the predictions
  within the ensemble. These codes can be saved, modified, and used for subsequent calls with the
  `latents` argument.
- **return_dict** (`bool`, *optional*, defaults to `True`) --
  Whether or not to return a [MarigoldIntrinsicsOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldIntrinsicsOutput) instead of a plain tuple.</paramsdesc><paramgroups>0</paramgroups><rettype>[MarigoldIntrinsicsOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldIntrinsicsOutput) or `tuple`</rettype><retdesc>If `return_dict` is `True`, [MarigoldIntrinsicsOutput](/docs/diffusers/main/en/api/pipelines/marigold#diffusers.pipelines.marigold.MarigoldIntrinsicsOutput) is returned, otherwise a
`tuple` is returned where the first element is the prediction, the second element is the uncertainty
(or `None`), and the third is the latent (or `None`).</retdesc></docstring>

Function invoked when calling the pipeline.



<ExampleCodeBlock anchor="diffusers.MarigoldIntrinsicsPipeline.__call__.example">

Examples:
```py
>>> import diffusers
>>> import torch

>>> pipe = diffusers.MarigoldIntrinsicsPipeline.from_pretrained(
...     "prs-eth/marigold-iid-appearance-v1-1", variant="fp16", torch_dtype=torch.float16
... ).to("cuda")

>>> image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg")
>>> intrinsics = pipe(image)

>>> vis = pipe.image_processor.visualize_intrinsics(intrinsics.prediction, pipe.target_properties)
>>> vis[0]["albedo"].save("einstein_albedo.png")
>>> vis[0]["roughness"].save("einstein_roughness.png")
>>> vis[0]["metallicity"].save("einstein_metallicity.png")
```

</ExampleCodeBlock>
<ExampleCodeBlock anchor="diffusers.MarigoldIntrinsicsPipeline.__call__.example-2">

```py
>>> import diffusers
>>> import torch

>>> pipe = diffusers.MarigoldIntrinsicsPipeline.from_pretrained(
...     "prs-eth/marigold-iid-lighting-v1-1", variant="fp16", torch_dtype=torch.float16
... ).to("cuda")

>>> image = diffusers.utils.load_image("https://marigoldmonodepth.github.io/images/einstein.jpg")
>>> intrinsics = pipe(image)

>>> vis = pipe.image_processor.visualize_intrinsics(intrinsics.prediction, pipe.target_properties)
>>> vis[0]["albedo"].save("einstein_albedo.png")
>>> vis[0]["shading"].save("einstein_shading.png")
>>> vis[0]["residual"].save("einstein_residual.png")
```

</ExampleCodeBlock>







</div></div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>class diffusers.pipelines.marigold.MarigoldIntrinsicsOutput</name><anchor>diffusers.pipelines.marigold.MarigoldIntrinsicsOutput</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/pipeline_marigold_intrinsics.py#L96</source><parameters>[{"name": "prediction", "val": ": typing.Union[numpy.ndarray, torch.Tensor]"}, {"name": "uncertainty", "val": ": typing.Union[NoneType, numpy.ndarray, torch.Tensor]"}, {"name": "latent", "val": ": typing.Optional[torch.Tensor]"}]</parameters><paramsdesc>- **prediction** (`np.ndarray`, `torch.Tensor`) --
  Predicted image intrinsics with values in the range [0, 1]. The shape is $(numimages * numtargets) 	imes 3
  imes height 	imes width$ for `torch.Tensor` or $(numimages * numtargets) 	imes height 	imes width
  imes 3$ for `np.ndarray`, where `numtargets` corresponds to the number of predicted target modalities of
  the intrinsic image decomposition.
- **uncertainty** (`None`, `np.ndarray`, `torch.Tensor`) --
  Uncertainty maps computed from the ensemble, with values in the range [0, 1]. The shape is $(numimages *
  numtargets) 	imes 3 	imes height 	imes width$ for `torch.Tensor` or $(numimages * numtargets) 	imes
  height 	imes width 	imes 3$ for `np.ndarray`.
- **latent** (`None`, `torch.Tensor`) --
  Latent features corresponding to the predictions, compatible with the `latents` argument of the pipeline.
  The shape is $(numimages * numensemble) 	imes (numtargets * 4) 	imes latentheight 	imes latentwidth$.</paramsdesc><paramgroups>0</paramgroups></docstring>

Output class for Marigold Intrinsic Image Decomposition pipeline.




</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>diffusers.pipelines.marigold.MarigoldImageProcessor.visualize_intrinsics</name><anchor>diffusers.pipelines.marigold.MarigoldImageProcessor.visualize_intrinsics</anchor><source>https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/marigold/marigold_image_processing.py#L549</source><parameters>[{"name": "prediction", "val": ": typing.Union[numpy.ndarray, torch.Tensor, typing.List[numpy.ndarray], typing.List[torch.Tensor]]"}, {"name": "target_properties", "val": ": typing.Dict[str, typing.Any]"}, {"name": "color_map", "val": ": typing.Union[str, typing.Dict[str, str]] = 'binary'"}]</parameters><paramsdesc>- **prediction** (`Union[np.ndarray, torch.Tensor, List[np.ndarray], List[torch.Tensor]]`) --
  Intrinsic image decomposition.
- **target_properties** (`Dict[str, Any]`) --
  Decomposition properties. Expected entries: `target_names: List[str]` and a dictionary with keys
  `prediction_space: str`, `sub_target_names: List[Union[str, Null]]` (must have 3 entries, null for
  missing modalities), `up_to_scale: bool`, one for each target and sub-target.
- **color_map** (`Union[str, Dict[str, str]]`, *optional*, defaults to `"Spectral"`) --
  Color map used to convert a single-channel predictions into colored representations. When a dictionary
  is passed, each modality can be colored with its own color map.</paramsdesc><paramgroups>0</paramgroups></docstring>

Visualizes intrinsic image decomposition, such as predictions of the `MarigoldIntrinsicsPipeline`.



Returns: `List[Dict[str, PIL.Image.Image]]` with intrinsic image decomposition visualization.


</div>

<EditOnGithub source="https://github.com/huggingface/diffusers/blob/main/docs/source/en/api/pipelines/marigold.md" />