  The $\delta f$ particle-in-cell algorithm has been a useful tool in studying
the physics of plasmas, particularly turbulent magnetized plasmas in the
context of gyrokinetics. The reduction in noise due to not having to resolve
the full distribution function indicates an efficiency advantage over standard
("full-$f$") particle-in-cell. Despite its successes, the algorithm behaves
strangely in some circumstances. In this work, we document a fully-resolved
numerical instability that occurs in the simplest of multiple-species test
cases: the electrostatic $\Omega_H$ mode. There is also a poorly-understood
numerical instability that occurs when one is under-resolved in particle
number, which may require a prohibitively large number of particles to
stabilize. Both of these are independent of the time-stepping scheme, and we
conclude that they exist if the time advancement were exact. The exact analytic
form of the algorithm is presented, and several schemes for mitigating these
instabilities are presented.
