  We consider the problem of neural network training in a time-varying context.
Machine learning algorithms have excelled in problems that do not change over
time. However, problems encountered in financial markets are often
time-varying. We propose the online early stopping algorithm and show that a
neural network trained using this algorithm can track a function changing with
unknown dynamics. We compare the proposed algorithm to current approaches on
predicting monthly U.S. stock returns and show its superiority. We also show
that prominent factors (such as the size and momentum effects) and industry
indicators, exhibit time varying stock return predictiveness. We find that
during market distress, industry indicators experience an increase in
importance at the expense of firm level features. This indicates that
industries play a role in explaining stock returns during periods of heightened
risk.
