  In crowdsourcing when there is a lack of verification for contributed
answers, output agreement mechanisms are often used to incentivize participants
to provide truthful answers when the correct answer is hold by the majority. In
this paper, we focus on using output agreement mechanisms to elicit effort, in
addition to eliciting truthful answers, from a population of workers. We
consider a setting where workers have heterogeneous cost of effort exertion and
examine the data requester's problem of deciding the reward level in output
agreement for optimal elicitation. In particular, when the requester knows the
cost distribution, we derive the optimal reward level for output agreement
mechanisms. This is achieved by first characterizing Bayesian Nash equilibria
of output agreement mechanisms for a given reward level. When the requester
does not know the cost distribution, we develop sequential mechanisms that
combine learning the cost distribution with incentivizing effort exertion to
approximately determine the optimal reward level.
