  We study approaches for compressing the empirical measure in the context of
finite dimensional reproducing kernel Hilbert spaces (RKHSs).In this context,
the empirical measure is contained within a natural convex set and can be
approximated using convex optimization methods.Such an approximation gives
under certain conditions rise to a coreset of data points. A key quantity that
controls how large such a coreset has to be is the size of the largest ball
around the empirical measure that is contained within the empirical convex set.
The bulk of our work is concerned with deriving high probability lower bounds
on the size of such a ball under various conditions. We complement this
derivation of the lower bound by developing techniques that allow us to apply
the compression approach to concrete inference problems such as kernel ridge
regression. We conclude with a construction of an infinite dimensional RKHS for
which the compression is poor, highlighting some of the difficulties one faces
when trying to move to infinite dimensional RKHSs.
