  ExaGRyPE describes a suite of solvers for numerical relativity, based upon
ExaHyPE 2, the second generation of our Exascale Hyperbolic PDE Engine. The
presented generation of ExaGRyPE solves the Einstein equations in the CCZ4
formulation under a 3+1 foliation and focuses on black hole spacetimes. It
employs a block-structured Cartesian grid carrying a higher-order order Finite
Difference scheme with adaptive mesh refinement, it facilitates massive
parallelism combining message passing, domain decomposition and task
parallelism, and it supports the injection of particles into the grid as data
probes or tracers. We introduce the ExaGRyPE-specific building blocks within
ExaHyPE 2, and discuss its software architecture and compute-n-feel:
  For this, we formalize the creation of any specific simulation with ExaGRyPE
as a sequence of lowering operations, where abstract logical tasks are
successively broken into finer tasks until we obtain an abstraction level that
can be mapped onto a C++ executable. The overall program logic is fully
specified via a domain-specific Python interface, we map this logic onto a more
detailed set of numerical tasks, subsequently lower this representation onto
technical tasks that the underlying ExaHyPE engine uses to parallelize the
application, before eventually the technical tasks are mapped onto task graphs
including the actual PDE term evaluations, initial conditions, boundary
conditions, and so forth. We end up with a rigorous separation of concerns
which shields ExaGRyPE users from technical details and hence simplifies the
development of novel physical models. We present the simulations and data for
the gauge wave, static single black holes and rotating binary black hole
systems, demonstrating that the code base is mature and usable. However, we
also uncover domain-specific numerical challenges that need further study by
the community in future work.
