  Adaptive radiotherapy is a growing field of study in cancer treatment due to
it's objective in sparing healthy tissue. The standard of care in several
institutions includes longitudinal cone-beam computed tomography (CBCT)
acquisitions to monitor changes, but have yet to be used to improve tumor
control while managing side-effects. The aim of this study is to demonstrate
the clinical value of pre-treatment CBCT acquired daily during radiation
therapy treatment for head and neck cancers for the downstream task of
predicting severe toxicity occurrence: reactive feeding tube (NG),
hospitalization and radionecrosis. For this, we propose a deformable 3D
classification pipeline that includes a component analyzing the Jacobian matrix
of the deformation between planning CT and longitudinal CBCT, as well as
clinical data. The model is based on a multi-branch 3D residual convolutional
neural network, while the CT to CBCT registration is based on a pair of
VoxelMorph architectures. Accuracies of 85.8% and 75.3% was found for
radionecrosis and hospitalization, respectively, with similar performance as
early as after the first week of treatment. For NG tube risk, performance
improves with increasing the timing of the CBCT fraction, reaching 83.1% after
the $5_{th}$ week of treatment.
