  Radiation hazards associated with standard-dose positron emission tomography
(SPET) images remain a concern, whereas the quality of low-dose PET (LPET)
images fails to meet clinical requirements. Therefore, there is great interest
in reconstructing SPET images from LPET images. However, prior studies focus
solely on image data, neglecting vital complementary information from other
modalities, e.g., patients' clinical tabular, resulting in compromised
reconstruction with limited diagnostic utility. Moreover, they often overlook
the semantic consistency between real SPET and reconstructed images, leading to
distorted semantic contexts. To tackle these problems, we propose a novel
Multi-modal Conditioned Adversarial Diffusion model (MCAD) to reconstruct SPET
images from multi-modal inputs, including LPET images and clinical tabular.
Specifically, our MCAD incorporates a Multi-modal conditional Encoder
(Mc-Encoder) to extract multi-modal features, followed by a conditional
diffusion process to blend noise with multi-modal features and gradually map
blended features to the target SPET images. To balance multi-modal inputs, the
Mc-Encoder embeds Optimal Multi-modal Transport co-Attention (OMTA) to narrow
the heterogeneity gap between image and tabular while capturing their
interactions, providing sufficient guidance for reconstruction. In addition, to
mitigate semantic distortions, we introduce the Multi-Modal Masked Text
Reconstruction (M3TRec), which leverages semantic knowledge extracted from
denoised PET images to restore the masked clinical tabular, thereby compelling
the network to maintain accurate semantics during reconstruction. To expedite
the diffusion process, we further introduce an adversarial diffusive network
with a reduced number of diffusion steps. Experiments show that our method
achieves the state-of-the-art performance both qualitatively and
quantitatively.
