Update README.md
Browse files
README.md
CHANGED
|
@@ -25,7 +25,7 @@ tags:
|
|
| 25 |
|
| 26 |
<div align="center">
|
| 27 |
|
| 28 |
-
[[π€ <strong><span style="color:hotpink">Diffusers</span></strong> Implementation](https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/visualcloze)]
|
| 29 |
|
| 30 |
</div>
|
| 31 |
|
|
@@ -41,6 +41,7 @@ If you find VisualCloze is helpful, please consider to star β the [<strong><sp
|
|
| 41 |
|
| 42 |
## π° News
|
| 43 |
- [2025-5-15] π€π€π€ VisualCloze has been merged into the [<strong><span style="color:hotpink">official pipelines of diffusers</span></strong>](https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/visualcloze).
|
|
|
|
| 44 |
|
| 45 |
## π Key Features
|
| 46 |
|
|
@@ -65,9 +66,13 @@ pip install git+https://github.com/huggingface/diffusers.git
|
|
| 65 |
|
| 66 |
[](https://huggingface.co/spaces/VisualCloze/VisualCloze)
|
| 67 |
|
| 68 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
while this model uses the `resolution` of 384. The `resolution` means that each image will be resized to it before being
|
| 70 |
-
concatenated to avoid the out-of-memory error. To generate high-resolution images, we use the
|
| 71 |
|
| 72 |
#### Example with Depth-to-Image:
|
| 73 |
|
|
@@ -106,6 +111,11 @@ high contrast, photorealistic, intimate, elegant, visually balanced, serene atmo
|
|
| 106 |
pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, torch_dtype=torch.bfloat16)
|
| 107 |
pipe.to("cuda")
|
| 108 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
# Run the pipeline
|
| 110 |
image_result = pipe(
|
| 111 |
task_prompt=task_prompt,
|
|
@@ -160,6 +170,11 @@ content_prompt = None
|
|
| 160 |
pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, torch_dtype=torch.bfloat16)
|
| 161 |
pipe.to("cuda")
|
| 162 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 163 |
# Run the pipeline
|
| 164 |
image_result = pipe(
|
| 165 |
task_prompt=task_prompt,
|
|
|
|
| 25 |
|
| 26 |
<div align="center">
|
| 27 |
|
| 28 |
+
[[π€ <strong><span style="color:hotpink">Diffusers</span></strong> Implementation](https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/visualcloze)]   [[π€ LoRA Model Card for Diffusers]](https://huggingface.co/VisualCloze/VisualClozePipeline-LoRA-384)
|
| 29 |
|
| 30 |
</div>
|
| 31 |
|
|
|
|
| 41 |
|
| 42 |
## π° News
|
| 43 |
- [2025-5-15] π€π€π€ VisualCloze has been merged into the [<strong><span style="color:hotpink">official pipelines of diffusers</span></strong>](https://github.com/huggingface/diffusers/tree/main/src/diffusers/pipelines/visualcloze).
|
| 44 |
+
- [2025-5-18] π₯³π₯³π₯³ We have released the LoRA weights supporting diffusers at [LoRA Model Card 384](https://huggingface.co/VisualCloze/VisualClozePipeline-LoRA-384) and [LoRA Model Card 512](https://huggingface.co/VisualCloze/VisualClozePipeline-LoRA-512).
|
| 45 |
|
| 46 |
## π Key Features
|
| 47 |
|
|
|
|
| 66 |
|
| 67 |
[](https://huggingface.co/spaces/VisualCloze/VisualCloze)
|
| 68 |
|
| 69 |
+
This model provides the full parameters of our VisualCloze.
|
| 70 |
+
If you find the download size too large, you can use the [LoRA version](https://huggingface.co/VisualCloze/VisualClozePipeline-LoRA-384)
|
| 71 |
+
with the FLUX.1-Fill-dev as the base model.
|
| 72 |
+
|
| 73 |
+
A model trained with the `resolution` of 512 is released at [Full Model Card 512](https://huggingface.co/VisualCloze/VisualClozePipeline-512) and [LoRA Model Card 512](https://huggingface.co/VisualCloze/VisualClozePipeline-LoRA-512),
|
| 74 |
while this model uses the `resolution` of 384. The `resolution` means that each image will be resized to it before being
|
| 75 |
+
concatenated to avoid the out-of-memory error. To generate high-resolution images, we use the SDEdit technology for upsampling the generated results.
|
| 76 |
|
| 77 |
#### Example with Depth-to-Image:
|
| 78 |
|
|
|
|
| 111 |
pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, torch_dtype=torch.bfloat16)
|
| 112 |
pipe.to("cuda")
|
| 113 |
|
| 114 |
+
# Loading the VisualClozePipeline via LoRA
|
| 115 |
+
# pipe = VisualClozePipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", resolution=384, torch_dtype=torch.bfloat16)
|
| 116 |
+
# pipe.load_lora_weights('VisualCloze/VisualClozePipeline-LoRA-384', weight_name='visualcloze-lora-384.safetensors')
|
| 117 |
+
# pipe.to("cuda")
|
| 118 |
+
|
| 119 |
# Run the pipeline
|
| 120 |
image_result = pipe(
|
| 121 |
task_prompt=task_prompt,
|
|
|
|
| 170 |
pipe = VisualClozePipeline.from_pretrained("VisualCloze/VisualClozePipeline-384", resolution=384, torch_dtype=torch.bfloat16)
|
| 171 |
pipe.to("cuda")
|
| 172 |
|
| 173 |
+
# Loading the VisualClozePipeline via LoRA
|
| 174 |
+
# pipe = VisualClozePipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", resolution=384, torch_dtype=torch.bfloat16)
|
| 175 |
+
# pipe.load_lora_weights('VisualCloze/VisualClozePipeline-LoRA-384', weight_name='visualcloze-lora-384.safetensors')
|
| 176 |
+
# pipe.to("cuda")
|
| 177 |
+
|
| 178 |
# Run the pipeline
|
| 179 |
image_result = pipe(
|
| 180 |
task_prompt=task_prompt,
|