  Crowds can often make better decisions than individuals or small groups of
experts by leveraging their ability to aggregate diverse information. Question
answering sites, such as Stack Exchange, rely on the "wisdom of crowds" effect
to identify the best answers to questions asked by users. We analyze data from
250 communities on the Stack Exchange network to pinpoint factors affecting
which answers are chosen as the best answers. Our results suggest that, rather
than evaluate all available answers to a question, users rely on simple
cognitive heuristics to choose an answer to vote for or accept. These cognitive
heuristics are linked to an answer's salience, such as the order in which it is
listed and how much screen space it occupies. While askers appear to depend
more on heuristics, compared to voting users, when choosing an answer to accept
as the most helpful one, voters use acceptance itself as a heuristic: they are
more likely to choose the answer after it is accepted than before that very
same answer was accepted. These heuristics become more important in explaining
and predicting behavior as the number of available answers increases. Our
findings suggest that crowd judgments may become less reliable as the number of
answers grow.
