  The TREC 2009 web ad hoc and relevance feedback tasks used a new document
collection, the ClueWeb09 dataset, which was crawled from the general Web in
early 2009. This dataset contains 1 billion web pages, a substantial fraction
of which are spam --- pages designed to deceive search engines so as to deliver
an unwanted payload. We examine the effect of spam on the results of the TREC
2009 web ad hoc and relevance feedback tasks, which used the ClueWeb09 dataset.
We show that a simple content-based classifier with minimal training is
efficient enough to rank the "spamminess" of every page in the dataset using a
standard personal computer in 48 hours, and effective enough to yield
significant and substantive improvements in the fixed-cutoff precision (estP10)
as well as rank measures (estR-Precision, StatMAP, MAP) of nearly all submitted
runs. Moreover, using a set of "honeypot" queries the labeling of training data
may be reduced to an entirely automatic process. The results of classical
information retrieval methods are particularly enhanced by filtering --- from
among the worst to among the best.
