  With FPGAs now being deployed in the cloud and at the edge, there is a need
for scalable design methods which can incorporate the heterogeneity present in
the hardware and software components of FPGA systems. Moreover, these FPGA
systems need to be maintainable and adaptable to changing workloads while
improving accessibility for the application developers. However, current FPGA
systems fail to achieve modularity and support for multi-tenancy due to
dependencies between system components and lack of standardised abstraction
layers. To solve this, we introduce a modular FPGA operating system -- FOS,
which adopts a modular FPGA development flow to allow each system component to
be changed and be agnostic to the heterogeneity of EDA tool versions, hardware
and software layers. Further, to dynamically maximise the utilisation
transparently from the users, FOS employs resource-elastic scheduling to
arbitrate the FPGA resources in both time and spatial domain for any type of
accelerators. Our evaluation on different FPGA boards shows that FOS can
provide performance improvements in both single-tenant and multi-tenant
environments while substantially reducing the development time and, at the same
time, improving flexibility.
