  Reconstruction of PET images is an ill-posed inverse problem and often
requires iterative algorithms to achieve good image quality for reliable
clinical use in practice, at huge computational costs. In this paper, we
consider the PET reconstruction a dense prediction problem where the large
scale contextual information is essential, and propose a novel architecture of
multi-scale fully convolutional neural networks (msfCNN) for fast PET image
reconstruction. The proposed msfCNN gains large receptive fields with both
memory and computational efficiency, by using a downscaling-upscaling structure
and dilated convolutions. Instead of pooling and deconvolution, we propose to
use the periodic shuffling operation from sub-pixel convolution and its inverse
to scale the size of feature maps without losing resolution. Residual
connections were added to improve training. We trained the proposed msfCNN
model with simulated data, and applied it to clinical PET data acquired on a
Siemens mMR scanner. The results from real oncological and neurodegenerative
cases show that the proposed msfCNN-based reconstruction outperforms the
iterative approaches in terms of computational time while achieving comparable
image quality for quantification. The proposed msfCNN model can be applied to
other dense prediction tasks, and fast msfCNN-based PET reconstruction could
facilitate the potential use of molecular imaging in interventional/surgical
procedures, where cancer surgery can particularly benefit.
