  For decades, the crowdsourcing has gained much attention from both academia
and industry, which outsources a number of tasks to human workers. Existing
works considered improving the task accuracy through voting or learning
methods, they usually did not fully take into account reducing the latency of
the task completion. When a task requester posts a group of tasks (e.g.,
sentiment analysis), and one can only obtain answers of all tasks after the
last task is accomplished. As a consequence, the time delay of even one task in
this group could delay the next step of the task requester's work from minutes
to days, which is quite undesirable for the task requester.
  Inspired by the importance of the task accuracy and latency, in this paper,
we will propose a novel crowdsourcing framework, namely Fast and Reliable
crOwdsourcinG framework (FROG), which intelligently assigns tasks to workers,
such that the latencies of tasks are reduced and the expected accuracies of
tasks are met. Specifically, our FROG framework consists of two important
components, task scheduler and notification modules. For the task scheduler
module, we formalize a FROG task scheduling (FROG-TS) problem, in which the
server actively assigns workers with tasks with high reliability and low
latency. We prove that the FROG-TS problem is NP-hard. Thus, we design two
heuristic approaches, request-based and batch-based scheduling. For the
notification module, we define an efficient worker notifying (EWN) problem,
which only sends task invitations to those workers with high probabilities of
accepting the tasks. To tackle the EWN problem, we propose a smooth kernel
density estimation approach to estimate the probability that a worker accepts
the task invitation. Through extensive experiments, we demonstrate the
effectiveness and efficiency of our proposed FROG platform on both real and
synthetic data sets.
