  Making an adaptive prediction based on one's input is an important ability
for general artificial intelligence. In this work, we step forward in this
direction and propose a semi-parametric method, Meta-Neighborhoods, where
predictions are made adaptively to the neighborhood of the input. We show that
Meta-Neighborhoods is a generalization of $k$-nearest-neighbors. Due to the
simpler manifold structure around a local neighborhood, Meta-Neighborhoods
represent the predictive distribution $p(y \mid x)$ more accurately. To reduce
memory and computation overhead, we propose induced neighborhoods that
summarize the training data into a much smaller dictionary. A meta-learning
based training mechanism is then exploited to jointly learn the induced
neighborhoods and the model. Extensive studies demonstrate the superiority of
our method.
