  We study the benefits of reinforcement learning (RL) environments based on
agent-based models (ABM). While ABMs are known to offer microfoundational
simulations at the cost of computational complexity, we empirically show in
this work that their non-deterministic dynamics can improve the generalization
of RL agents. To this end, we examine the control of an epidemic SIR
environments based on either differential equations or ABMs. Numerical
simulations demonstrate that the intrinsic noise in the ABM-based dynamics of
the SIR model not only improve the average reward but also allow the RL agent
to generalize on a wider ranges of epidemic parameters.
