  Software crowdsourcing platforms employ extrinsic rewards such as rating or
ranking systems to motivate workers. Such rating systems are noisy and provide
limited knowledge about workers' preferences and performance. To develop better
understanding of worker reliability and trustworthiness in software
crowdsourcing, this paper reports an empirical study conducted on more than one
year's real-world data from TopCoder, one of the leading software crowdsourcing
platforms.
  To do so, first, we create a bipartite network of active workers based on
common task registrations. Then, we use the Clauset-Newman-Moore graph
clustering algorithm to identify worker clusters in the network. Finally, we
conduct an empirical evaluation to measure and analyze workers' behavior per
identified community in the platform by workers' rating. More specifically,
workers' behavior is analyzed based on their performances in terms of
reliability, trustworthiness, and success; their preferences in terms of
efficiency and elasticity; and strategies in terms of comfort, confidence, and
deceitfulness. The main result of this study identified four communities of
active workers: mixed-ranked, high-ranked, mid-ranked, and low-ranked. This
study shows that the low-ranked community associates with the highest reliable
workers with an average reliability of 25%, while the mixed-ranked community
contains the most trustworthy workers with average trustworthiness of 16%. Such
empirical evidence is beneficial to help exploring resourcing options while
understanding the relations among unknown resources to improve task success.
