  Trade-offs such as "how much testing is enough" are critical yet challenging
project decisions in software engineering. Most existing approaches adopt
risk-driven or value-based analysis to prioritize test cases and minimize test
runs. However, none of these is applicable to the emerging crowd testing
paradigm where task requesters typically have no control over online
crowdworkers's dynamic behavior and uncertain performance.
  In current practice, deciding when to close a crowdtesting task is largely
done by guesswork due to lack of decision support. This paper intends to fill
this gap by introducing automated decision support for monitoring and
determining appropriate time to close the crowdtesting tasks.
  First, this paper investigates the necessity and feasibility of close
prediction of crowdtesting tasks based on industrial dataset. Then,it designs 8
methods for close prediction, based on various models including the bug trend,
bug arrival model, capture-recapture model.Finally, the evaluation is conducted
on 218 crowdtesting tasks from one of the largest crowdtesting platforms in
China, and the results show that a median of 91% bugs can be detected with 49%
saved cost.
