  Current generation solid-state storage devices are exposing a new bottlenecks
in the SCSI and block layers of the Linux kernel, where IO throughput is
limited by lock contention, inefficient interrupt handling, and poor memory
locality. To address these limitations, the Linux kernel block layer underwent
a major rewrite with the blk-mq project to move from a single request queue to
a multi-queue model. The Linux SCSI subsystem rework to make use of this new
model, known as scsi-mq, has been merged into the Linux kernel and work is
underway for dm-multipath support in the upcoming Linux 4.0 kernel. These
pieces were necessary to make use of the multi-queue block layer in a Lustre
parallel filesystem with high availability requirements. We undertook adding
support of the 3.18 kernel to Lustre with scsi-mq and dm-multipath patches to
evaluate the potential of these efficiency improvements. In this paper we
evaluate the block-level performance of scsi-mq with backing storage hardware
representative of a HPC-targerted Lustre filesystem. Our findings show that
SCSI write request latency is reduced by as much as 13.6%. Additionally, when
profiling the CPU usage of our prototype Lustre filesystem, we found that CPU
idle time increased by a factor of 7 with Linux 3.18 and blk-mq as compared to
a standard 2.6.32 Linux kernel. Our findings demonstrate increased efficiency
of the multi-queue block layer even with disk-based caching storage arrays used
in existing parallel filesystems.
