  Neural networks successfully capture the computational power of the human
brain for many tasks. Similarly inspired by the brain architecture, Nearest
Neighbor (NN) representations is a novel approach of computation. We establish
a firmer correspondence between NN representations and neural networks.
Although it was known how to represent a single neuron using NN
representations, there were no results even for small depth neural networks.
Specifically, for depth-2 threshold circuits, we provide explicit constructions
for their NN representation with an explicit bound on the number of bits to
represent it. Example functions include NN representations of convex polytopes
(AND of threshold gates), IP2, OR of threshold gates, and linear or exact
decision lists.
