  We introduce so-called functional input neural networks defined on a possibly
infinite dimensional weighted space with values also in a possibly infinite
dimensional output space. To this end, we use an additive family to map the
input weighted space to the hidden layer, on which a non-linear scalar
activation function is applied to each neuron, and finally return the output
via some linear readouts. Relying on Stone-Weierstrass theorems on weighted
spaces, we can prove a global universal approximation result on weighted spaces
for continuous functions going beyond the usual approximation on compact sets.
This then applies in particular to approximation of (non-anticipative) path
space functionals via functional input neural networks. As a further
application of the weighted Stone-Weierstrass theorem we prove a global
universal approximation result for linear functions of the signature. We also
introduce the viewpoint of Gaussian process regression in this setting and
emphasize that the reproducing kernel Hilbert space of the signature kernels
are Cameron-Martin spaces of certain Gaussian processes. This paves a way
towards uncertainty quantification for signature kernel regression.
