  This paper newly introduces multi-modality loss function for GAN-based
super-resolution that can maintain image structure and intensity on unpaired
training dataset of clinical CT and micro CT volumes. Precise non-invasive
diagnosis of lung cancer mainly utilizes 3D multidetector computed-tomography
(CT) data. On the other hand, we can take micro CT images of resected lung
specimen in 50 micro meter or higher resolution. However, micro CT scanning
cannot be applied to living human imaging. For obtaining highly detailed
information such as cancer invasion area from pre-operative clinical CT volumes
of lung cancer patients, super-resolution (SR) of clinical CT volumes to
$\mu$CT level might be one of substitutive solutions. While most SR methods
require paired low- and high-resolution images for training, it is infeasible
to obtain precisely paired clinical CT and micro CT volumes. We aim to propose
unpaired SR approaches for clincial CT using micro CT images based on unpaired
image translation methods such as CycleGAN or UNIT. Since clinical CT and micro
CT are very different in structure and intensity, direct application of
GAN-based unpaired image translation methods in super-resolution tends to
generate arbitrary images. Aiming to solve this problem, we propose new loss
function called multi-modality loss function to maintain the similarity of
input images and corresponding output images in super-resolution task.
Experimental results demonstrated that the newly proposed loss function made
CycleGAN and UNIT to successfully perform SR of clinical CT images of lung
cancer patients into micro CT level resolution, while original CycleGAN and
UNIT failed in super-resolution.
