#
🧨 Diffusers

🤗 Diffusers 是一个值得首选用于生成图像、音频甚至 3D 分子结构的，最先进的预训练扩散模型库。 无论您是在寻找简单的推理解决方案，还是想训练自己的扩散模型，🤗 Diffusers 这一模块化工具箱都能对其提供支持。 本库的设计更偏重于可用而非高性能、简明而非简单以及易用而非抽象。

本库包含三个主要组件：

- 最先进的扩散管道 diffusion pipelines，只需几行代码即可进行推理。
- 可交替使用的各种噪声调度器 noise schedulers，用于平衡生成速度和质量。
- 预训练模型 models，可作为构建模块，并与调度程序结合使用，来创建您自己的端到端扩散系统。

Tutorials

Learn the fundamental skills you need to start generating outputs, build your own diffusion system, and train a diffusion model. We recommend starting here if you're using 🤗 Diffusers for the first time!

How-to guides

Practical guides for helping you load pipelines, models, and schedulers. You'll also learn how to use pipelines for specific tasks, control how outputs are generated, optimize for inference speed, and different training techniques.

Conceptual guides

Understand why the library was designed the way it was, and learn more about the ethical guidelines and safety implementations for using the library.

Reference

Technical descriptions of how 🤗 Diffusers classes and methods work.

##
🧨 Diffusers pipelines

下表汇总了当前所有官方支持的pipelines及其对应的论文.

管道 | 论文/仓库 | 任务
--- | --- | ---
alt_diffusion | AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities | Image-to-Image Text-Guided Generation
audio_diffusion | Audio Diffusion | Unconditional Audio Generation
controlnet | Adding Conditional Control to Text-to-Image Diffusion Models | Image-to-Image Text-Guided Generation
cycle_diffusion | Unifying Diffusion Models’ Latent Space, with Applications to CycleDiffusion and Guidance | Image-to-Image Text-Guided Generation
dance_diffusion | Dance Diffusion | Unconditional Audio Generation
ddpm | Denoising Diffusion Probabilistic Models | Unconditional Image Generation
ddim | Denoising Diffusion Implicit Models | Unconditional Image Generation
if | IF | Image Generation
if_img2img | IF | Image-to-Image Generation
if_inpainting | IF | Image-to-Image Generation
latent_diffusion | High-Resolution Image Synthesis with Latent Diffusion Models | Text-to-Image Generation
latent_diffusion | High-Resolution Image Synthesis with Latent Diffusion Models | Super Resolution Image-to-Image
latent_diffusion_uncond | High-Resolution Image Synthesis with Latent Diffusion Models | Unconditional Image Generation
paint_by_example | Paint by Example: Exemplar-based Image Editing with Diffusion Models | Image-Guided Image Inpainting
pndm | Pseudo Numerical Methods for Diffusion Models on Manifolds | Unconditional Image Generation
score_sde_ve | Score-Based Generative Modeling through Stochastic Differential Equations | Unconditional Image Generation
score_sde_vp | Score-Based Generative Modeling through Stochastic Differential Equations | Unconditional Image Generation
semantic_stable_diffusion | Semantic Guidance | Text-Guided Generation
stable_diffusion_text2img | Stable Diffusion | Text-to-Image Generation
stable_diffusion_img2img | Stable Diffusion | Image-to-Image Text-Guided Generation
stable_diffusion_inpaint | Stable Diffusion | Text-Guided Image Inpainting
stable_diffusion_panorama | MultiDiffusion | Text-to-Panorama Generation
stable_diffusion_pix2pix | InstructPix2Pix: Learning to Follow Image Editing Instructions | Text-Guided Image Editing
stable_diffusion_pix2pix_zero | Zero-shot Image-to-Image Translation | Text-Guided Image Editing
stable_diffusion_attend_and_excite | Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models | Text-to-Image Generation
stable_diffusion_self_attention_guidance | Improving Sample Quality of Diffusion Models Using Self-Attention Guidance | Text-to-Image Generation Unconditional Image Generation
stable_diffusion_image_variation | Stable Diffusion Image Variations | Image-to-Image Generation
stable_diffusion_latent_upscale | Stable Diffusion Latent Upscaler | Text-Guided Super Resolution Image-to-Image
stable_diffusion_model_editing | Editing Implicit Assumptions in Text-to-Image Diffusion Models | Text-to-Image Model Editing
stable_diffusion_2 | Stable Diffusion 2 | Text-to-Image Generation
stable_diffusion_2 | Stable Diffusion 2 | Text-Guided Image Inpainting
stable_diffusion_2 | Depth-Conditional Stable Diffusion | Depth-to-Image Generation
stable_diffusion_2 | Stable Diffusion 2 | Text-Guided Super Resolution Image-to-Image
stable_diffusion_safe | Safe Stable Diffusion | Text-Guided Generation
stable_unclip | Stable unCLIP | Text-to-Image Generation
stable_unclip | Stable unCLIP | Image-to-Image Text-Guided Generation
stochastic_karras_ve | Elucidating the Design Space of Diffusion-Based Generative Models | Unconditional Image Generation
text_to_video_sd | Modelscope’s Text-to-video-synthesis Model in Open Domain | Text-to-Video Generation
unclip | Hierarchical Text-Conditional Image Generation with CLIP Latents(implementation by kakaobrain) | Text-to-Image Generation
versatile_diffusion | Versatile Diffusion: Text, Images and Variations All in One Diffusion Model | Text-to-Image Generation
versatile_diffusion | Versatile Diffusion: Text, Images and Variations All in One Diffusion Model | Image Variations Generation
versatile_diffusion | Versatile Diffusion: Text, Images and Variations All in One Diffusion Model | Dual Image and Text Guided Generation
vq_diffusion | Vector Quantized Diffusion Model for Text-to-Image Synthesis | Text-to-Image Generation

Update on GitHub