  The dominance of machine learning and the ending of Moore's law have renewed
interests in Processor in Memory (PIM) architectures. This interest has
produced several recent proposals to modify an FPGA's BRAM architecture to form
a next-generation PIM reconfigurable fabric. PIM architectures can also be
realized within today's FPGAs as overlays without the need to modify the
underlying FPGA architecture. To date, there has been no study to understand
the comparative advantages of the two approaches. In this paper, we present a
study that explores the comparative advantages between two proposed custom
architectures and a PIM overlay running on a commodity FPGA. We created PiCaSO,
a Processor in/near Memory Scalable and Fast Overlay architecture as a
representative PIM overlay. The results of this study show that the PiCaSO
overlay achieves up to 80% of the peak throughput of the custom designs with
2.56x shorter latency and 25% - 43% better BRAM memory utilization efficiency.
We then show how several key features of the PiCaSO overlay can be integrated
into the custom PIM designs to further improve their throughput by 18%, latency
by 19.5%, and memory efficiency by 6.2%.
