  We present experimental results highlighting two key differences resulting
from the choice of training algorithm for two-layer neural networks. The
spectral bias of neural networks is well known, while the spectral bias
dependence on the choice of training algorithm is less studied. Our experiments
demonstrate that an adaptive random Fourier features algorithm (ARFF) can yield
a spectral bias closer to zero compared to the stochastic gradient descent
optimizer (SGD). Additionally, we train two identically structured classifiers,
employing SGD and ARFF, to the same accuracy levels and empirically assess
their robustness against adversarial noise attacks.
