  The demise of Moore's Law has led to the rise of hardware acceleration.
However, the focus on accelerating stable algorithms in their entirety neglects
the abundant fine-grained acceleration opportunities available in broader
domains and squanders host processors' compute power. This paper presents Duet,
a scalable, manycore-FPGA architecture that promotes embedded FPGAs (eFPGA) to
be equal peers with processors through non-intrusive, bi-directionally
cache-coherent integration. In contrast to existing CPU-FPGA hybrid systems in
which the processors play a supportive role, Duet unleashes the full potential
of both the processors and the eFPGAs with two classes of post-fabrication
enhancements: fine-grained acceleration, which partitions an application into
small tasks and offloads the frequently-invoked, compute-intensive ones onto
various small accelerators, leveraging the processors to handle dynamic control
flow and less accelerable tasks; hardware augmentation, which employs
eFPGA-emulated hardware widgets to improve processor efficiency or mitigate
software overheads in certain execution models. An RTL-level implementation of
Duet is developed to evaluate the architecture with high fidelity. Experiments
using synthetic benchmarks show that Duet can reduce the processor-accelerator
communication latency by up to 82% and increase the bandwidth by up to 9.5x.
The RTL implementation is further evaluated with seven application benchmarks,
achieving 1.5-24.9x speedup.
