  We study the problems of sequential nonparametric two-sample and independence
testing. Sequential tests process data online and allow using observed data to
decide whether to stop and reject the null hypothesis or to collect more data,
while maintaining type I error control. We build upon the principle of
(nonparametric) testing by betting, where a gambler places bets on future
observations and their wealth measures evidence against the null hypothesis.
While recently developed kernel-based betting strategies often work well on
simple distributions, selecting a suitable kernel for high-dimensional or
structured data, such as images, is often nontrivial. To address this drawback,
we design prediction-based betting strategies that rely on the following fact:
if a sequentially updated predictor starts to consistently determine (a) which
distribution an instance is drawn from, or (b) whether an instance is drawn
from the joint distribution or the product of the marginal distributions (the
latter produced by external randomization), it provides evidence against the
two-sample or independence nulls respectively. We empirically demonstrate the
superiority of our tests over kernel-based approaches under structured
settings. Our tests can be applied beyond the case of independent and
identically distributed data, remaining valid and powerful even when the data
distribution drifts over time.
