Update README.md to include a diffusers example (#2)
Browse files- Update README.md (0f5619591d52be126abc24912cd5ffc762202c4d)
- Upload darth_vader_grid.png (b4fc6df76673d2093963f6db6ed3080d6866ce39)
- Update README.md (c124ff14968f416ae86fa73fca31329649e788a5)
Co-authored-by: Sayak Paul <[email protected]>
- .gitattributes +1 -0
- README.md +52 -0
- darth_vader_grid.png +3 -0
.gitattributes
CHANGED
|
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
out_ballerina.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
out_ballerina.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
darth_vader_grid.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -26,6 +26,58 @@ prompt: a ballerina, romantic sunset, 4k photo
|
|
| 26 |
|
| 27 |
License: refers to the OpenPose's one.
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
### Training
|
| 31 |
|
|
|
|
| 26 |
|
| 27 |
License: refers to the OpenPose's one.
|
| 28 |
|
| 29 |
+
### Using in 🧨 diffusers
|
| 30 |
+
|
| 31 |
+
First, install all the libraries:
|
| 32 |
+
|
| 33 |
+
```bash
|
| 34 |
+
pip install -q controlnet_aux transformers accelerate
|
| 35 |
+
pip install -q git+https://github.com/huggingface/diffusers
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
Now, we're ready to make Darth Vader dance:
|
| 39 |
+
|
| 40 |
+
```python
|
| 41 |
+
from diffusers import AutoencoderKL, StableDiffusionXLControlNetPipeline, ControlNetModel, UniPCMultistepScheduler
|
| 42 |
+
import torch
|
| 43 |
+
from controlnet_aux import OpenposeDetector
|
| 44 |
+
from diffusers.utils import load_image
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
# Compute openpose conditioning image.
|
| 48 |
+
openpose = OpenposeDetector.from_pretrained("lllyasviel/ControlNet")
|
| 49 |
+
|
| 50 |
+
image = load_image(
|
| 51 |
+
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/person.png"
|
| 52 |
+
)
|
| 53 |
+
openpose_image = openpose(image)
|
| 54 |
+
|
| 55 |
+
# Initialize ControlNet pipeline.
|
| 56 |
+
controlnet = ControlNetModel.from_pretrained("thibaud/controlnet-openpose-sdxl-1.0", torch_dtype=torch.float16)
|
| 57 |
+
pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
|
| 58 |
+
"stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, torch_dtype=torch.float16
|
| 59 |
+
)
|
| 60 |
+
pipe.enable_model_cpu_offload()
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
# Infer.
|
| 64 |
+
prompt = "Darth vader dancing in a desert, high quality"
|
| 65 |
+
negative_prompt = "low quality, bad quality"
|
| 66 |
+
images = pipe(
|
| 67 |
+
prompt,
|
| 68 |
+
negative_prompt=negative_prompt,
|
| 69 |
+
num_inference_steps=25,
|
| 70 |
+
num_images_per_prompt=4,
|
| 71 |
+
image=openpose_image.resize((1024, 1024)),
|
| 72 |
+
generator=torch.manual_seed(97),
|
| 73 |
+
).images
|
| 74 |
+
images[0]
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
Here are some gemerated examples:
|
| 78 |
+
|
| 79 |
+

|
| 80 |
+
|
| 81 |
|
| 82 |
### Training
|
| 83 |
|
darth_vader_grid.png
ADDED
|
Git LFS Details
|