  Echocardiography (ECHO) video is widely used for cardiac examination. In
clinical, this procedure heavily relies on operator experience, which needs
years of training and maybe the assistance of deep learning-based systems for
enhanced accuracy and efficiency. However, it is challenging since acquiring
sufficient customized data (e.g., abnormal cases) for novice training and deep
model development is clinically unrealistic. Hence, controllable ECHO video
synthesis is highly desirable. In this paper, we propose a novel
diffusion-based framework named HeartBeat towards controllable and
high-fidelity ECHO video synthesis. Our highlight is three-fold. First,
HeartBeat serves as a unified framework that enables perceiving multimodal
conditions simultaneously to guide controllable generation. Second, we
factorize the multimodal conditions into local and global ones, with two
insertion strategies separately provided fine- and coarse-grained controls in a
composable and flexible manner. In this way, users can synthesize ECHO videos
that conform to their mental imagery by combining multimodal control signals.
Third, we propose to decouple the visual concepts and temporal dynamics
learning using a two-stage training scheme for simplifying the model training.
One more interesting thing is that HeartBeat can easily generalize to
mask-guided cardiac MRI synthesis in a few shots, showcasing its scalability to
broader applications. Extensive experiments on two public datasets show the
efficacy of the proposed HeartBeat.
