  We study the task of learning from non-i.i.d. data. In particular, we aim at
learning predictors that minimize the conditional risk for a stochastic
process, i.e. the expected loss of the predictor on the next point conditioned
on the set of training samples observed so far. For non-i.i.d. data, the
training set contains information about the upcoming samples, so learning with
respect to the conditional distribution can be expected to yield better
predictors than one obtains from the classical setting of minimizing the
marginal risk. Our main contribution is a practical estimator for the
conditional risk based on the theory of non-parametric time-series prediction,
and a finite sample concentration bound that establishes uniform convergence of
the estimator to the true conditional risk under certain regularity assumptions
on the process.
