  Industrial, metrological, and medical applications provide a strong
technological pull for advanced nanoscale sensors exploiting the unique
sensitivity of quantum coherent systems to their environments. Essential to the
functionality of these devices is the availability of control protocols which
shape the sensor's response to the environment in frequency space. However, a
key challenge in these applications is that common control routines result in
out-of-band spectral leakage which complicates interpretation of the sensor's
signal. In this work we demonstrate provably optimal narrowband control
protocols ideally suited to quantum sensing. Our results, based on experiments
with trapped ions using modulation in the form of discrete prolate spheroidal
sequences (aka Slepian functions), demonstrate reduction of spectral leakage by
orders of magnitude over conventional controls. We tune the narrowband
sensitivity using concepts from RF engineering and experimentally reconstruct
complex noise spectra using engineered noise for quantitative performance
evaluation. We then deploy these techniques to identify previously immeasurable
frequency-resolved amplitude noise in our qubit synthesis chain with calibrated
sensitivity better than 0.001 dB.
