  Event cameras are advantageous for tasks that require vision sensors with
low-latency and sparse output responses. However, the development of deep
network algorithms using event cameras has been slow because of the lack of
large labelled event camera datasets for network training. This paper reports a
method for creating new labelled event datasets by using a text-to-X model,
where X is one or multiple output modalities, in the case of this work, events.
Our proposed text-to-events model produces synthetic event frames directly from
text prompts. It uses an autoencoder which is trained to produce sparse event
frames representing event camera outputs. By combining the pretrained
autoencoder with a diffusion model architecture, the new text-to-events model
is able to generate smooth synthetic event streams of moving objects. The
autoencoder was first trained on an event camera dataset of diverse scenes. In
the combined training with the diffusion model, the DVS gesture dataset was
used. We demonstrate that the model can generate realistic event sequences of
human gestures prompted by different text statements. The classification
accuracy of the generated sequences, using a classifier trained on the real
dataset, ranges between 42% to 92%, depending on the gesture group. The results
demonstrate the capability of this method in synthesizing event datasets.
