  We deal with two complementary questions about approximation properties of
ReLU networks. First, we study how the uniform quantization of ReLU networks
with real-valued weights impacts their approximation properties. We establish
an upper-bound on the minimal number of bits per coordinate needed for
uniformly quantized ReLU networks to keep the same polynomial asymptotic
approximation speeds as unquantized ones. We also characterize the error of
nearest-neighbour uniform quantization of ReLU networks. This is achieved using
a new lower-bound on the Lipschitz constant of the map that associates the
parameters of ReLU networks to their realization, and an upper-bound
generalizing classical results. Second, we investigate when ReLU networks can
be expected, or not, to have better approximation properties than other
classical approximation families. Indeed, several approximation families share
the following common limitation: their polynomial asymptotic approximation
speed of any set is bounded from above by the encoding speed of this set. We
introduce a new abstract property of approximation families, called
infinite-encodability, which implies this upper-bound. Many classical
approximation families, defined with dictionaries or ReLU networks, are shown
to be infinite-encodable. This unifies and generalizes several situations where
this upper-bound is known.
