  We prove some new results concerning the approximation rate of neural
networks with general activation functions. Our first result concerns the rate
of approximation of a two layer neural network with a polynomially-decaying
non-sigmoidal activation function. We extend the dimension independent
approximation rates previously obtained to this new class of activation
functions. Our second result gives a weaker, but still dimension independent,
approximation rate for a larger class of activation functions, removing the
polynomial decay assumption. This result applies to any bounded, integrable
activation function. Finally, we show that a stratified sampling approach can
be used to improve the approximation rate for polynomially decaying activation
functions under mild additional assumptions.
