  Generative models hold promise for revolutionizing medical education,
robot-assisted surgery, and data augmentation for medical AI development.
Diffusion models can now generate realistic images from text prompts, while
recent advancements have demonstrated their ability to create diverse,
high-quality videos. However, these models often struggle with generating
accurate representations of medical procedures and detailed anatomical
structures. This paper introduces Bora, the first spatio-temporal diffusion
probabilistic model designed for text-guided biomedical video generation. Bora
leverages Transformer architecture and is pre-trained on general-purpose video
generation tasks. It is fine-tuned through model alignment and instruction
tuning using a newly established medical video corpus, which includes paired
text-video data from various biomedical fields. To the best of our knowledge,
this is the first attempt to establish such a comprehensive annotated
biomedical video dataset. Bora is capable of generating high-quality video data
across four distinct biomedical domains, adhering to medical expert standards
and demonstrating consistency and diversity. This generalist video generative
model holds significant potential for enhancing medical consultation and
decision-making, particularly in resource-limited settings. Additionally, Bora
could pave the way for immersive medical training and procedure planning.
Extensive experiments on distinct medical modalities such as endoscopy,
ultrasound, MRI, and cell tracking validate the effectiveness of our model in
understanding biomedical instructions and its superior performance across
subjects compared to state-of-the-art generation models.
