  We develop an abstract model of information acquisition from redundant data.
We assume a random sampling process from data which provide information with
bias and are interested in the fraction of information we expect to learn as
function of (i) the sampled fraction (recall) and (ii) varying bias of
information (redundancy distributions). We develop two rules of thumb with
varying robustness. We first show that, when information bias follows a Zipf
distribution, the 80-20 rule or Pareto principle does surprisingly not hold,
and we rather expect to learn less than 40% of the information when randomly
sampling 20% of the overall data. We then analytically prove that for large
data sets, randomized sampling from power-law distributions leads to "truncated
distributions" with the same power-law exponent. This second rule is very
robust and also holds for distributions that deviate substantially from a
strict power law. We further give one particular family of powerlaw functions
that remain completely invariant under sampling. Finally, we validate our model
with two large Web data sets: link distributions to domains and tag
distributions on delicious.com.
