  Engineering is an important domain for supercomputing, with the Alya model
being a popular code for undertaking such simulations. With ever increasing
demand from users to model larger, more complex systems at reduced time to
solution it is important to explore the role that novel hardware technologies,
such as FPGAs, can play in accelerating these workloads on future exascale
systems.
  In this paper we explore the porting of Alya's incompressible flow matrix
assembly kernel, which accounts for a large proportion of the model runtime,
onto FPGAs. After describing in detail successful strategies for optimisation
at the kernel level, we then explore sharing the workload between the FPGA and
host CPU, mapping most appropriate parts of the kernel between these
technologies, enabling us to more effectively exploit the FPGA. We then compare
the performance of our approach on a Xilinx Alveo U280 against a 24-core Xeon
Platinum CPU and Nvidia V100 GPU, with the FPGA significantly out-performing
the CPU and performing comparably against the GPU, whilst drawing substantially
less power. The result of this work is both an experience report describing
appropriate dataflow optimisations which we believe can be applied more widely
as a case-study across HPC codes, and a performance comparison for this
specific workload that demonstrates the potential for FPGAs in accelerating HPC
engineering simulations.
