  One of the most crucial issues in data mining is to model human behaviour in
order to provide personalisation, adaptation and recommendation. This usually
involves implicit or explicit knowledge, either by observing user interactions,
or by asking users directly. But these sources of information are always
subject to the volatility of human decisions, making utilised data uncertain to
a particular extent. In this contribution, we elaborate on the impact of this
human uncertainty when it comes to comparative assessments of different data
mining approaches. In particular, we reveal two problems: (1) biasing effects
on various metrics of model-based prediction and (2) the propagation of
uncertainty and its thus induced error probabilities for algorithm rankings.
For this purpose, we introduce a probabilistic view and prove the existence of
those problems mathematically, as well as provide possible solution strategies.
We exemplify our theory mainly in the context of recommender systems along with
the metric RMSE as a prominent example of precision quality measures.
