  Wikipedia articles aim to be definitive sources of encyclopedic content. Yet,
only 0.6% of Wikipedia articles have high quality according to its quality
scale due to insufficient number of Wikipedia editors and enormous number of
articles. Supervised Machine Learning (ML) quality improvement approaches that
can automatically identify and fix content issues rely on manual labels of
individual Wikipedia sentence quality. However, current labeling approaches are
tedious and produce noisy labels. Here, we propose an automated labeling
approach that identifies the semantic category (e.g., adding citations,
clarifications) of historic Wikipedia edits and uses the modified sentences
prior to the edit as examples that require that semantic improvement.
Highest-rated article sentences are examples that no longer need semantic
improvements. We show that training existing sentence quality classification
algorithms on our labels improves their performance compared to training them
on existing labels. Our work shows that editing behaviors of Wikipedia editors
provide better labels than labels generated by crowdworkers who lack the
context to make judgments that the editors would agree with.
