  Measuring airways in chest computed tomography (CT) scans is important for
characterizing diseases such as cystic fibrosis, yet very time-consuming to
perform manually. Machine learning algorithms offer an alternative, but need
large sets of annotated scans for good performance. We investigate whether
crowdsourcing can be used to gather airway annotations. We generate image
slices at known locations of airways in 24 subjects and request the crowd
workers to outline the airway lumen and airway wall. After combining multiple
crowd workers, we compare the measurements to those made by the experts in the
original scans. Similar to our preliminary study, a large portion of the
annotations were excluded, possibly due to workers misunderstanding the
instructions. After excluding such annotations, moderate to strong correlations
with the expert can be observed, although these correlations are slightly lower
than inter-expert correlations. Furthermore, the results across subjects in
this study are quite variable. Although the crowd has potential in annotating
airways, further development is needed for it to be robust enough for gathering
annotations in practice. For reproducibility, data and code are available
online: \url{http://github.com/adriapr/crowdairway.git}.
