  The Coherent Accelerator Processor Interface (CAPI) is a general term for the
infrastructure that provides high throughput and low latency path to the flash
storage connected to the IBM POWER 8+ System. CAPI accelerator card is attached
coherently as a peer to the Power8+ processor. This removes the overhead and
complexity of the IO subsystem and allows the accelerator to operate as part of
an application. In this paper, we present the results of experiments on IBM
FlashSystem900 (FS900) with CAPI accelerator card using the "CAPI-Flash IBM
Data Engine for NoSQL Software" Library. This library provides the application,
a direct access to the underlying flash storage through user space APIs, to
manage and access the data in flash. This offloads kernel IO driver
functionality to dedicated CAPI FPGA accelerator hardware. We conducted
experiments to analyze the performance of FS900 with CAPI accelerator card,
using the Key Value Layer APIs, employing NASA's MODIS Land Surface Reflectance
dataset as a large dataset use case. We performed Read and Write operations on
datasets of size ranging from 1MB to 3TB by varying the number of threads. We
then compared this performance with other heterogeneous storage and memory
devices such as NVM, SSD and RAM, without using the CAPI Accelerator in
synchronous and asynchronous file IO modes of operations. The results indicate
that FS900 & CAPI, together with the metadata cache in RAM, delivers the
highest IO/s and OP/s for read operations. This was higher than just using RAM,
along with utilizing lesser CPU resources. Among FS900, SSD and NVM, FS900 had
the highest write IO/s. Another important observation is that, when the size of
the input dataset exceeds the capacity of RAM, and when the data access is
non-uniform and sparse, FS900 with CAPI would be a cost-effective alternative.
