  Although chest X-ray (CXR) offers a 2D projection with overlapped anatomies,
it is widely used for clinical diagnosis. There is clinical evidence supporting
that decomposing an X-ray image into different components (e.g., bone, lung and
soft tissue) improves diagnostic value. We hereby propose a decomposition
generative adversarial network (DecGAN) to anatomically decompose a CXR image
but with unpaired data. We leverage the anatomy knowledge embedded in CT, which
features a 3D volume with clearly visible anatomies. Our key idea is to embed
CT priori decomposition knowledge into the latent space of unpaired CXR
autoencoder. Specifically, we train DecGAN with a decomposition loss,
adversarial losses, cycle-consistency losses and a mask loss to guarantee that
the decomposed results of the latent space preserve realistic body structures.
Extensive experiments demonstrate that DecGAN provides superior unsupervised
CXR bone suppression results and the feasibility of modulating CXR components
by latent space disentanglement. Furthermore, we illustrate the diagnostic
value of DecGAN and demonstrate that it outperforms the state-of-the-art
approaches in terms of predicting 11 out of 14 common lung diseases.
