  In this paper, we review physics- and data-driven reconstruction techniques
for simultaneous positron emission tomography (PET) / magnetic resonance
imaging (MRI) systems, which have significant advantages for clinical imaging
of cancer, neurological disorders, and heart disease. These reconstruction
approaches utilize priors, either structural or statistical, together with a
physics-based description of the PET system response. However, due to the
nested representation of the forward problem, direct PET/MRI reconstruction is
a nonlinear problem. We elucidate how a multi-faceted approach accommodates
hybrid data- and physics-driven machine learning for reconstruction of 3D
PET/MRI, summarizing important deep learning developments made in the last 5
years to address attenuation correction, scattering, low photon counts, and
data consistency. We also describe how applications of these multi-modality
approaches extend beyond PET/MRI to improving accuracy in radiation therapy
planning. We conclude by discussing opportunities for extending the current
state-of-the-art following the latest trends in physics- and deep
learning-based computational imaging and next-generation detector hardware.
