  We present a comprehensive experimental study on image-level conditioning for
diffusion models using cluster assignments. We elucidate how individual
components regarding image clustering impact image synthesis across three
datasets. By combining recent advancements from image clustering and diffusion
models, we show that, given the optimal cluster granularity with respect to
image synthesis (visual groups), cluster-conditioning can achieve
state-of-the-art FID (i.e. 1.67, 2.17 on CIFAR10 and CIFAR100 respectively),
while attaining a strong training sample efficiency. Finally, we propose a
novel method to derive an upper cluster bound that reduces the search space of
the visual groups using solely feature-based clustering. Unlike existing
approaches, we find no significant connection between clustering and
cluster-conditional image generation. The code and cluster assignments will be
released.
