  The vanishing ideal is a set of polynomials that takes zero value on the
given data points. Originally proposed in computer algebra, the vanishing ideal
has been recently exploited for extracting the nonlinear structures of data in
many applications. To avoid overfitting to noisy data, the polynomials are
often designed to approximately rather than exactly equal zero on the
designated data. Although such approximations empirically demonstrate high
performance, the sound algebraic structure of the vanishing ideal is lost. The
present paper proposes a vanishing ideal that is tolerant to noisy data and
also pursued to have a better algebraic structure. As a new problem, we
simultaneously find a set of polynomials and data points for which the
polynomials approximately vanish on the input data points, and almost exactly
vanish on the discovered data points. In experimental classification tests, our
method discovered much fewer and lower-degree polynomials than an existing
state-of-the-art method. Consequently, our method accelerated the runtime of
the classification tasks without degrading the classification accuracy.
