  Peer grading systems make large courses more scalable, provide students with
faster and more detailed feedback, and help students to learn by thinking
critically about the work of others. A key obstacle to the broader adoption of
peer grading systems is motivating students to provide accurate grades. The
literature has explored many different approaches to incentivizing accurate
grading (which we survey in detail), but the strongest incentive guarantees
have been offered by mechanisms that compare peer grades to trusted TA grades
with a fixed probability. In this work, we show that less TA work is required
when these probabilities are allowed to depend on the grades that students
report. We prove this result in a model with two possible grades, arbitrary
numbers of agents, no requirement that students grade multiple assignments,
arbitrary but homogeneous noisy observation of the ground truth which students
can pay a fixed cost to observe, and the possibility of misreporting grades
before or after observing this signal. We give necessary and sufficient
conditions for our new mechanism's feasibility, prove its optimality under
these assumptions, and characterize its improvement over the previous state of
the art both analytically and empirically. Finally, we relax our homogeneity
assumption, allowing each student and TA to observe the ground truth according
to a different noise model.
