  Token-based masked generative models are gaining popularity for their fast
inference time with parallel decoding. While recent token-based approaches
achieve competitive performance to diffusion-based models, their generation
performance is still suboptimal as they sample multiple tokens simultaneously
without considering the dependence among them. We empirically investigate this
problem and propose a learnable sampling model, Text-Conditioned Token
Selection (TCTS), to select optimal tokens via localized supervision with text
information. TCTS improves not only the image quality but also the semantic
alignment of the generated images with the given texts. To further improve the
image quality, we introduce a cohesive sampling strategy, Frequency Adaptive
Sampling (FAS), to each group of tokens divided according to the self-attention
maps. We validate the efficacy of TCTS combined with FAS with various
generative tasks, demonstrating that it significantly outperforms the baselines
in image-text alignment and image quality. Our text-conditioned sampling
framework further reduces the original inference time by more than 50% without
modifying the original generative model.
