  The synthesis of high-resolution remote sensing images based on text
descriptions has great potential in many practical application scenarios.
Although deep neural networks have achieved great success in many important
remote sensing tasks, generating realistic remote sensing images from text
descriptions is still very difficult. To address this challenge, we propose a
novel text-to-image modern Hopfield network (Txt2Img-MHN). The main idea of
Txt2Img-MHN is to conduct hierarchical prototype learning on both text and
image embeddings with modern Hopfield layers. Instead of directly learning
concrete but highly diverse text-image joint feature representations for
different semantics, Txt2Img-MHN aims to learn the most representative
prototypes from text-image embeddings, achieving a coarse-to-fine learning
strategy. These learned prototypes can then be utilized to represent more
complex semantics in the text-to-image generation task. To better evaluate the
realism and semantic consistency of the generated images, we further conduct
zero-shot classification on real remote sensing data using the classification
model trained on synthesized images. Despite its simplicity, we find that the
overall accuracy in the zero-shot classification may serve as a good metric to
evaluate the ability to generate an image from text. Extensive experiments on
the benchmark remote sensing text-image dataset demonstrate that the proposed
Txt2Img-MHN can generate more realistic remote sensing images than existing
methods. Code and pre-trained models are available online
(https://github.com/YonghaoXu/Txt2Img-MHN).
