  We study the natural function space for infinitely wide two-layer neural
networks with ReLU activation (Barron space) and establish different
representation formulae. In two cases, we describe the space explicitly up to
isomorphism.
  Using a convenient representation, we study the pointwise properties of
two-layer networks and show that functions whose singular set is fractal or
curved (for example distance functions from smooth submanifolds) cannot be
represented by infinitely wide two-layer networks with finite path-norm. We use
this structure theorem to show that the only $C^1$-diffeomorphisms which Barron
space are affine.
  Furthermore, we show that every Barron function can be decomposed as the sum
of a bounded and a positively one-homogeneous function and that there exist
Barron functions which decay rapidly at infinity and are globally
Lebesgue-integrable. This result suggests that two-layer neural networks may be
able to approximate a greater variety of functions than commonly believed.
