  In this work, we revisit the marking decisions made in the standard adaptive
finite element method (AFEM). Experience shows that a na\"{i}ve marking policy
leads to inefficient use of computational resources for adaptive mesh
refinement (AMR). Consequently, using AFEM in practice often involves ad-hoc or
time-consuming offline parameter tuning to set appropriate parameters for the
marking subroutine. To address these practical concerns, we recast AMR as a
Markov decision process in which refinement parameters can be selected
on-the-fly at run time, without the need for pre-tuning by expert users. In
this new paradigm, the refinement parameters are also chosen adaptively via a
marking policy that can be optimized using methods from reinforcement learning.
We use the Poisson equation to demonstrate our techniques on $h$- and
$hp$-refinement benchmark problems, and our experiments suggest that superior
marking policies remain undiscovered for many classical AFEM applications.
Furthermore, an unexpected observation from this work is that marking policies
trained on one family of PDEs are sometimes robust enough to perform well on
problems far outside the training family. For illustration, we show that a
simple $hp$-refinement policy trained on 2D domains with only a single
re-entrant corner can be deployed on far more complicated 2D domains, and even
3D domains, without significant performance loss. For reproduction and broader
adoption, we accompany this work with an open-source implementation of our
methods.
