  The purpose of this work is to advance fair and consistent comparisons of
dose prediction methods for knowledge-based planning (KBP) in radiation therapy
research. We hosted OpenKBP, a 2020 AAPM Grand Challenge, and challenged
participants to develop the best method for predicting the dose of contoured CT
images. The models were evaluated according to two separate scores: (1) dose
score, which evaluates the full 3D dose distributions, and (2) dose-volume
histogram (DVH) score, which evaluates a set DVH metrics. Participants were
given the data of 340 patients who were treated for head-and-neck cancer with
radiation therapy. The data was partitioned into training (n=200), validation
(n=40), and testing (n=100) datasets. All participants performed training and
validation with the corresponding datasets during the validation phase of the
Challenge, and we ranked the models in the testing phase based on out-of-sample
performance. The Challenge attracted 195 participants from 28 countries, and 73
of those participants formed 44 teams in the validation phase, which received a
total of 1750 submissions. The testing phase garnered submissions from 28
teams. On average, over the course of the validation phase, participants
improved the dose and DVH scores of their models by a factor of 2.7 and 5.7,
respectively. In the testing phase one model achieved significantly better dose
and DVH score than the runner-up models. Lastly, many of the top performing
teams reported using generalizable techniques (e.g., ensembles) to achieve
higher performance than their competition. This is the first competition for
knowledge-based planning research, and it helped launch the first platform for
comparing KBP prediction methods fairly and consistently. The OpenKBP datasets
are available publicly to help benchmark future KBP research, which has also
democratized KBP research by making it accessible to everyone.
