  We propose Deep-Motion-Net: an end-to-end graph neural network (GNN)
architecture that enables 3D (volumetric) organ shape reconstruction from a
single in-treatment kV planar X-ray image acquired at any arbitrary projection
angle. Estimating and compensating for true anatomical motion during
radiotherapy is essential for improving the delivery of planned radiation dose
to target volumes while sparing organs-at-risk, and thereby improving the
therapeutic ratio. Achieving this using only limited imaging available during
irradiation and without the use of surrogate signals or invasive fiducial
markers is attractive. The proposed model learns the mesh regression from a
patient-specific template and deep features extracted from kV images at
arbitrary projection angles. A 2D-CNN encoder extracts image features, and four
feature pooling networks fuse these features to the 3D template organ mesh. A
ResNet-based graph attention network then deforms the feature-encoded mesh. The
model is trained using synthetically generated organ motion instances and
corresponding kV images. The latter is generated by deforming a reference CT
volume aligned with the template mesh, creating digitally reconstructed
radiographs (DRRs) at required projection angles, and DRR-to-kV style
transferring with a conditional CycleGAN model. The overall framework was
tested quantitatively on synthetic respiratory motion scenarios and
qualitatively on in-treatment images acquired over full scan series for liver
cancer patients. Overall mean prediction errors for synthetic motion test
datasets were 0.16$\pm$0.13 mm, 0.18$\pm$0.19 mm, 0.22$\pm$0.34 mm, and
0.12$\pm$0.11 mm. Mean peak prediction errors were 1.39 mm, 1.99 mm, 3.29 mm,
and 1.16 mm.
