  The computationally cheap machine learning architecture of random feature
maps can be viewed as a single-layer feedforward network in which the weights
of the hidden layer are random but fixed and only the outer weights are learned
via linear regression. The internal weights are typically chosen from a
prescribed distribution. The choice of the internal weights significantly
impacts the accuracy of random feature maps. We address here the task of how to
best select the internal weights. In particular, we consider the forecasting
problem whereby random feature maps are used to learn a one-step propagator map
for a dynamical system. We provide a computationally cheap hit-and-run
algorithm to select good internal weights which lead to good forecasting skill.
We show that the number of good features is the main factor controlling the
forecasting skill of random feature maps and acts as an effective feature
dimension. Lastly, we compare random feature maps with single-layer feedforward
neural networks in which the internal weights are now learned using gradient
descent. We find that random feature maps have superior forecasting
capabilities whilst having several orders of magnitude lower computational
cost.
